Source: https://dzone.com/articles/dzone-research-automated-testing-vs-manual-testing
[DZone Research] Automated Testing vs. Manual Testing
We take a look at the data we gathered as part of our research for the 2018 Automated Testing Guide, discussing how developers test their software. By Jordan Baker
This article is part of the Key Research Findings from the DZone Guide to Automated Testing: Your End-to-End Ecosystem.
Introduction
For this year's Automated Testing Guide Survey, we received
821 responses with a 64% completion rate. Based on these responses,
we've compiled a quick article on the uses of automated and manual
testing.
Automated Testing
Only 11% of respondents told us that their organization does
not practice any kind of automated testing. Out of the 89% who do
automate tests somewhere within their organization and SDLC, there were
varying opinions as to when it's best to automate tests, and which tests
should be automated. When asked at what stage they begin automating
their tests, 54% of respondents told us development, 31% in
staging/QA/testing, and 4% in deployment.
The most popular tests to automate in the development
pipeline were integration tests (77%), component tests (52%),
and performance tests (51%). Comparing this data to last year's
automated testing survey, we see that integration tests are on the rise,
while the frequency of component and performance testing is falling. In
last year's survey, 61% of respondents automated their integration
tests, which, when compared with this year's data, constitutes a 16%
increase. Automated performance tests, however, suffered a 5% decrease
year-over-year, and automated component tests saw a 6% year-over-over
decrease.
Manual Testing
Since manual testing still has its place in the development
pipeline, and there's a small, but not insignificant, faction of
organizations who do not use automated testing, let's quickly look at
where and how manual tests are used and how this compares to automated
tests. When asked at what stage they begin manual testing, 51% reported
development, 40% said staging/QA/testing, and 7% told us deployment. Not
too surprisingly, manual tests are used in much the same places in the
pipeline as automated tests. When it comes the types of tests, however,
we see some divergence between manual and automated. When asked, "Which
tests in your organization’s pipeline(s) are currently performed
manually?", 60% said user acceptance tests, 50% usability tests, 41%
story-level tests, and 38% integration tests.
Interestingly, despite the year-over-year decrease in
automated component and performance tests noted above, the instances of
manual components, performance, and integration tests all decreased as
well. In 2017, 41% of respondents performed manual integration
tests, while in 2018 this number fell to 38%; manual performance
tests decreased from 45% in 2017 to 34% in 2018;
and manual component tests fell from 40% in 2017 to 29% in 2018.
Conclusion
Despite the numbers presented here, whether your team goes
with automated or manual testing truly depends on the situation. A large
portion of this year's respondents to the Automated Testing survey work
in enterprise-level environments, and, in such development and testing
environments, automated testing is a great resource that can decrease
the amount of repetitive and monotonous work. While automated testing
has become the more popular testing method, automating everything in
your SDLC is impossible and some tests are better off done manually ("From 0 to 100: How to Get Into Automated Testing as a Manual Tester," by Alex McPeak). When it comes to ad-hoc, usability, and exploratory tests, manual tests may be the way to go ("What's the Difference Between Automated Testing and Manual Testing?" by Whitney Donaldson).
This article is part of the Key Research Findings from the DZone Guide to Automated Testing: Your End-to-End Ecosystem.
