Test Automation Is Not a Tooling Story – InApps 2022

Test automation tools are not used very often. Only 16 percent of performance test cases are executed with test automation tools, and security tests are being completed at the same frequency according to the World Quality Report (WQR) 2018-2019, which surveyed 1,700 IT decision makers (ITDMs) at companies with more than a thousand employees. Although the QA and testing job roles have been adapting to agile development practices, remember that even if one test is automated, the majority of tests are still done manually.

Source: World Quality Report 2018-2019

Of course, automation does not have to be enabled by a specialized testing tool. Whether you’re looking at software QA or information security, the actual level of automation is actually higher because discrete components of often homegrown processes are handled in a programmatic way. For example, the same survey found that 40 percent automate their delivery pipeline (automated deployment) with integrated test cases. Another developer-focused survey conducted by DZone found that 77 percent of their organization’s pipelines have automated integration testing. Automating usability testing is much harder and only occurs 25 percent of the time.

Source: The 2018 DZone Guide to Automated Testing

There is agreement that testing can’t be completely automated. Part of this is because humans are still needed, but there are other obstacles as well. When asked about test automation challenges, 61 percent of WQR respondents said their applications change too much with every release. Thus, while release frequency has increased due to release and management automation, testing may be a choke-point for future SLDC automation. The second most common obstacle to test automation is the availability of test data and environment availability.

Test environments can be automatically generated, which may address the increasingly noted challenges of the costs of test environments and the number of defects arising due to inaccurate test configurations of test environments. Vendors like Plutora and Enov8 provide automation tools that can alleviate these types of testing challenges. However, they do not address the specific concerns about the number and type of tests being conducted within these environments.

Feature image via Pixabay.