Manual vs. automated software testing — choosing the right approach

Therac-25 was a radiation therapy machine that became famous for the wrong reason. Between 1985 and 1987 it gave massive overdoses of radiation to several patients. Three patients died after receiving around 100 times the intended dose; many more were badly injured.

The Mars climate orbiter built by NASA, crashed in 1999, almost immediately after arriving on the destination planet. $327 million and a one-year journey were wasted in a single second.

These two accidents have one thing in common. They were both caused by a software error, or bug, which wasn’t discovered in time.

The U.S. economy loses an estimated $59.5 billion annually due to software errors, according to a study by the Department of Commerce’s National Institute of Standards and Technology (NIST). The same report states that $22.2 billion could have been saved with improved testing.

Even if your software won’t potentially kill people, every single bug can worsen the user experience and cause you to lose dollar after dollar. To minimize the negative impact, you should consider software testing.

What is software testing and why is it important?

According to Wikipedia, “software testing is an investigation conducted to provide stakeholders with information about the quality of the software product or service under test.”
In other words, testing is the process of checking the software for any mistakes, or bugs.

In reality, software can’t be written without any bugs, — and this doesn’t reflect on the professionalism of the developers. Bugs are a natural “by-product” of software development. The only way to eliminate them is to reduce the functionality of the application.

However, this is not a reason to just accept bugs, and stand by while they kill your application. Most bugs can be found and eliminated during the development stage, before the application gets into the hands of actual users.

Our message is clear. Don’t aim to build an entirely bug-free product. Aim to minimize bugs that will significantly affect the user experience.

Manual testing vs. automated testing

There are two ways to ensure that the application will work without any critical errors: manual testing and automated testing.

Manual testing is a process where the tester plays the role of the end user, and manually executes all the test cases. It is important to foresee all the possible actions the end user can take, and check whether the software will respond correctly to these actions.

Automated testing doesn’t require the tester to manually click through the application. On the contrary, an automated testing tool will execute all the predefined test cases, comparing the expected response of the software with the actual response.

Any type of project can involve both manual and automated testing, however, the correct choice of testing strategy depends on the project itself. It is important to find a balance between manual and automated testing.

Features of manual software testing

Manual testing requires the tester to act as the target software user. The tester’s behavior with the app should closely resemble the behavior of the user. Therefore, a deep understanding of the target audience is critical. Besides, there are several things you should consider when choosing manual testing.

• The real human behavior
  Manual testing enables to take certain actions with the software that the testing tool will never predict or simulate. At the same time, they are likely to be taken by the users. You will also receive actual user feedback concerning UX of the software, which is quite valuable.
  At the same time, the human factor is an important disadvantage. The monotonous work can cause the tester to be less attentive, resulting in some minor (or even major) bugs being missed.
  

  

• Time and cost efficiency
  Manual testing is perfect when you have to test the app only once, as this does not require any preparation such as writing test cases. Moreover, you save a lot on the cost of testing tools.
  On the other hand, if the software is being modified frequently, it requires running the same test cases over and over again. Doing this manually would require a significant amount of time. Automated test cases, once written, can be reused for each update testing.
• Flexibility
  Testing the software manually allows a lot of flexibility. The tester can quickly change the testing strategy or test any idea straight away. Compared to this, automated testing is less flexible, since changes to the testing procedure can be quite time-consuming.
  This is impossible during automated testing, as all the test cases are written in advance and can’t be changed or added during the testing process.
  
  

  

Features of automated software testing

Prior to automated testing, the tester must predefine the use cases the tool is going to audit. As you may have guessed, automated testing software imitates the actions of reals user and compares the app’s actual response with the expected one.

• More exact than a human tester
  One of the disadvantages of manual testing is the human factor. Mistakes due to large amounts of testing or a lack of attention will never happen to software (unless there is a bug :) ). That’s why automated testing is more exact and enables more bugs to be found.
  However, the automated testing cannot test the user-friendliness of a website or app. Manual testing, on the contrary, shows more exact results when it comes to testing the user interface.
  
  

• Higher productivity
  Automated test cases are executed several times faster than manual tests. They can also run unattended, which enables testing to occur overnight, or several applications can be tested at the same time. However, automated testing takes longer to set up.
  
  

  

• Reusable test cases
  This is important for software that is frequently modified. Once written, the test scripts can be reused as many times as needed
  
  
• Expenses for automated testing tools
  Automated testing tools are usually expensive, but are a reasonable long-term investment. They work perfectly with software that needs to be tested frequently.
  At the same time, they aren’t a good solution for one-time testing. Preparing the use-cases can take longer than manual testing. Add to this the price of automated testing tools, and you’ll see how much resources can be wasted if you don’t analyze the case before choosing the testing strategy.
  
  
• Results sharing
  Manual testing usually involves one person who executes the test, and only this person sees the results. Automated testing is better for larger teams since the system retains all the test results and enables these to be shared with everyone involved in testing and bug fixing.  
  
  

Different testing approaches for different tasks

Which is better, manual or automated testing? I’m afraid this question will remain unanswered. Both types of testing are good, but both satisfy completely different requirements.

When to use manual testing?

• User interface testing
  As already mentioned, a computer isn’t able to test some subjective things. It won’t tell you something is wrong with your website or app usability. Is the font size comfortable for reading? Is it intuitively clear which button to click? These and many other similar questions can be answered only by manual testing. For subjective tests, you need the answer of a human, not a machine.
  
  
• One-time testing
  Got a final version of the app and want to test it before release? Writing test cases for automated testing would require too much time and resources. Besides, you won’t ever need to use them again. Manual testing, on the contrary, is efficient, doesn’t require any preparation and enables you to test the app from the point of view of the end user.
  
  
• Lack of specialized software or skilled quality assurance specialist
  Automated testing requires experience and certain programming skills, as the QA engineer has to write code to execute the test cases. Additionally, specialized tools are required. If either one of these is missing - choose manual testing. It can be done without any additional tools, and the tester doesn’t need development experience.

When to use automated testing?

• Functionality testing
  Software features, speed, loading can be tested manually, however, automated testing may be more efficient and more exact.
  
  
• Regression testing
  Automated testing is a must for software that is in development and being constantly updated. The test cases, once written, can be reused without significant changes after each software update, which dramatically saves time and, of course, reduces the cost of testing.
  
  
• Load testing and performance testing of complex software
  Software that is intended to be used by millions of users simultaneously is difficult to test manually. Automated testing, on the contrary, is an efficient tool since it allows to imitate any number of users at the same time.
  
  
• Working in a large or remote team
  Teamwork often requires constant communication and sharing of current progress, which is often impossible with manual testing. Automated testing results are saved by the system and can be shared with anyone in the team, even if the person is on another continent. This is perfect for remote cooperation.

Conclusion

Even if you hire the top-rated developers, this won’t guarantee that your code will be bug-free. Software development is a domain where mistakes are an inevitable part of the process.

Manual and automated software testing can be used to eliminate programming errors. These are not competing approaches, and neither is one better than the other. In fact, manual and automated testing can complement each other, allowing to select the best approach for solving certain problems.

Manual testing is the best solution for UI audition since it allows the tester to act as a potential user of the software, and discover all the nuances and faults that automated testing software may not identify.

Functionality requires testing at a deeper level, and automated testing is the right choice for this due to the extra precision and speed. However, preparing test cases for a certain piece of functionality may take up to three times longer than testing manually. Therefore, automated testing is a preferred option in cases where a recurring test is required.

It is important to find the right correlation between automated and manual testing. Together they form a powerful weapon against costly bugs that encroach on your software quality.