Manual vs. automated software testing — choosing the right approach

Article header
Article header

Therac-25 was a radiation therapy machine that became famous for the wrong reason. Between 1985 and 1987 it gave massive overdoses of radiation to several patients. Three patients died after receiving around 100 times the intended dose; many more were badly injured.

The Mars climate orbiter built by NASA, crashed in 1999, almost immediately after arriving on the destination planet. $327 million and a one-year journey were wasted in a single second.

These two accidents have one thing in common. They were both caused by a software error, or bug, which wasn’t discovered in time.

The U.S. economy loses an estimated $59.5 billion annually due to software errors, according to a study by the Department of Commerce’s National Institute of Standards and Technology (NIST). The same report states that $22.2 billion could have been saved with improved testing.

Even if your software won’t potentially kill people, every single bug can worsen the user experience and cause you to lose dollar after dollar. To minimize the negative impact, you should consider software testing.

What is software testing and why is it important?

According to Wikipedia, “software testing is an investigation conducted to provide stakeholders with information about the quality of the software product or service under test.”
In other words, testing is the process of checking the software for any mistakes, or bugs.

In reality, software can’t be written without any bugs, — and this doesn’t reflect on the professionalism of the developers. Bugs are a natural “by-product” of software development. The only way to eliminate them is to reduce the functionality of the application.

However, this is not a reason to just accept bugs, and stand by while they kill your application. Most bugs can be found and eliminated during the development stage, before the application gets into the hands of actual users.

Our message is clear. Don’t aim to build an entirely bug-free product. Aim to minimize bugs that will significantly affect the user experience.

Manual testing vs. automated testing

There are two ways to ensure that the application will work without any critical errors: manual testing and automated testing.

Manual testing is a process where the tester plays the role of the end user, and manually executes all the test cases. It is important to foresee all the possible actions the end user can take, and check whether the software will respond correctly to these actions.

Automated testing doesn’t require the tester to manually click through the application. On the contrary, an automated testing tool will execute all the predefined test cases, comparing the expected response of the software with the actual response.

Any type of project can involve both manual and automated testing, however, the correct choice of testing strategy depends on the project itself. It is important to find a balance between manual and automated testing.

Features of manual software testing

Manual testing requires the tester to act as the target software user. The tester’s behavior with the app should closely resemble the behavior of the user. Therefore, a deep understanding of the target audience is critical. Besides, there are several things you should consider when choosing manual testing.

  • The real human behavior



    Manual testing enables to take certain actions with the software that the testing tool will never predict or simulate. At the same time, they are likely to be taken by the users. You will also receive actual user feedback concerning UX of the software, which is quite valuable.
    At the same time, the human factor is an important disadvantage. The monotonous work can cause the tester to be less attentive, resulting in some minor (or even major) bugs being missed.

  • Time and cost efficiency
    Manual testing is perfect when you have to test the app only once, as this does not require any preparation such as writing test cases. Moreover, you save a lot on the cost of testing tools.
    On the other hand, if the software is being modified frequently, it requires running the same test cases over and over again. Doing this manually would require a significant amount of time. Automated test cases, once written, can be reused for each update testing.

  • Flexibility



    Testing the software manually allows a lot of flexibility. The tester can quickly change the testing strategy or test any idea straight away. Compared to this, automated testing is less flexible, since changes to the testing procedure can be quite time-consuming.
    This is impossible during automated testing, as all the test cases are written in advance and can’t be changed or added during the testing process.

Features of automated software testing

Prior to automated testing, the tester must predefine the use cases the tool is going to audit. As you may have guessed, automated testing software imitates the actions of reals user and compares the app’s actual response with the expected one.

  • More exact than a human tester
    One of the disadvantages of manual testing is the human factor. Mistakes due to large amounts of testing or a lack of attention will never happen to software (unless there is a bug :) ). That’s why automated testing is more exact and enables more bugs to be found.
    However, the automated testing cannot test the user-friendliness of a website or app. Manual testing, on the contrary, shows more exact results when it comes to testing the user interface.

  • Higher productivity



    Automated test cases are executed several times faster than manual tests. They can also run unattended, which enables testing to occur overnight, or several applications can be tested at the same time. However, automated testing takes longer to set up.

  • Reusable test cases
    This is important for software that is frequently modified. Once written, the test scripts can be reused as many times as needed

  • Expenses for automated testing tools
    Automated testing tools are usually expensive, but are a reasonable long-term investment. They work perfectly with software that needs to be tested frequently.
    At the same time, they aren’t a good solution for one-time testing. Preparing the use-cases can take longer than manual testing. Add to this the price of automated testing tools, and you’ll see how much resources can be wasted if you don’t analyze the case before choosing the testing strategy.

  • Results sharing
    Manual testing usually involves one person who executes the test, and only this person sees the results. Automated testing is better for larger teams since the system retains all the test results and enables these to be shared with everyone involved in testing and bug fixing.  

Different testing approaches for different tasks

Which is better, manual or automated testing? I’m afraid this question will remain unanswered. Both types of testing are good, but both satisfy completely different requirements.

When to use manual testing?

  • User interface testing
    As already mentioned, a computer isn’t able to test some subjective things. It won’t tell you something is wrong with your website or app usability. Is the font size comfortable for reading? Is it intuitively clear which button to click? These and many other similar questions can be answered only by manual testing. For subjective tests, you need the answer of a human, not a machine.

  • One-time testing
    Got a final version of the app and want to test it before release? Writing test cases for automated testing would require too much time and resources. Besides, you won’t ever need to use them again. Manual testing, on the contrary, is efficient, doesn’t require any preparation and enables you to test the app from the point of view of the end user.

  • Lack of specialized software or skilled quality assurance specialist
    Automated testing requires experience and certain programming skills, as the QA engineer has to write code to execute the test cases. Additionally, specialized tools are required. If either one of these is missing - choose manual testing. It can be done without any additional tools, and the tester doesn’t need development experience.

When to use automated testing?

  • Functionality testing
    Software features, speed, loading can be tested manually, however, automated testing may be more efficient and more exact.

  • Regression testing
    Automated testing is a must for software that is in development and being constantly updated. The test cases, once written, can be reused without significant changes after each software update, which dramatically saves time and, of course, reduces the cost of testing.

  • Load testing and performance testing of complex software
    Software that is intended to be used by millions of users simultaneously is difficult to test manually. Automated testing, on the contrary, is an efficient tool since it allows to imitate any number of users at the same time.

  • Working in a large or remote team
    Teamwork often requires constant communication and sharing of current progress, which is often impossible with manual testing. Automated testing results are saved by the system and can be shared with anyone in the team, even if the person is on another continent. This is perfect for remote cooperation.

Conclusion

Even if you hire the top rated developers, this won’t guarantee that your code will be bug-free. Software development is a domain where mistakes are an inevitable part of the process.

Manual and automated software testing can be used to eliminate programming errors. These are not competing approaches, and neither is one better than the other. In fact, manual and automated testing can complement each other, allowing to select the best approach for solving certain problems.

Manual testing is the best solution for UI audition since it allows the tester to act as a potential user of the software, and discover all the nuances and faults that automated testing software may not identify.

Functionality requires testing at a deeper level, and automated testing is the right choice for this due to the extra precision and speed. However, preparing test cases for a certain piece of functionality may take up to three times longer than testing manually. Therefore, automated testing is a preferred option in cases where a recurring test is required.

It is important to find the right correlation between automated and manual testing. Together they form a powerful weapon against costly bugs that encroach on your software quality.



Liked this article? Sign up to be notified of new blog posts.

You might also like

Web Development Trends in 2018

Web vs. native vs. hybrid applications. Finding a compromise between price and performance

Frontend frameworks showdown: Angular vs. React vs. Vue

Web development trends are changing rapidly. Yesterday the multi-page websites were popular, today the trend for simplicity brought single-page apps back on the top. Read about other tendencies in web development for 2018.
Web and hybrid apps a bad, native apps are expensive. If you keep hearing this cliche, let us dispel the myth. You will be surprised how many world’s top companies have “bad” apps and are still adored by their users. With this article you will make the wise choice about your future app development.
Starting a new project you definitely have to face the hard choice of the technologies it will be based on. This comparison of three most powerful front end frameworks will help you making the right choice according to your real needs, and not just current trends. Check it out!

Web Development Trends in 2018

Web development trends are changing rapidly. Yesterday the multi-page websites were popular, today the trend for simplicity brought single-page apps back on the top. Read about other tendencies in web development for 2018.

Web vs. native vs. hybrid applications. Finding a compromise between price and performance

Web and hybrid apps a bad, native apps are expensive. If you keep hearing this cliche, let us dispel the myth. You will be surprised how many world’s top companies have “bad” apps and are still adored by their users. With this article you will make the wise choice about your future app development.

Frontend frameworks showdown: Angular vs. React vs. Vue

Starting a new project you definitely have to face the hard choice of the technologies it will be based on. This comparison of three most powerful front end frameworks will help you making the right choice according to your real needs, and not just current trends. Check it out!