Why You Must Reinvent Testing for Agile and DevOps – InApps Technology 2022

Tricentis sponsored this post.

Think back just five years ago. In 2014:

Times have changed — a lot. If the way you test hasn’t already transformed dramatically, it will soon. And the pace and scope of disruption will continue to escalate throughout the foreseeable future.

On the one hand, testing is all too often the roadblock that stands between highly accelerated Dev processes and highly automated ops-driven delivery processes. But on the other hand, testing is essential for ensuring that the release doesn’t place the business at risk — undermining the very “customer experience” that digital transformation is dedicating to enhancing.

How can you achieve the optimal balance between speed and risk to deliver engaging customer experiences faster than competitors? Enter Continuous Testing.

Continuous Testing is the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release as rapidly as possible.

Continuous Testing really boils down to providing the right feedback to the right stakeholder at the right time. For decades, testing was traditionally deferred until the end of the cycle. At that point, testers would provide all sorts of important feedback… but nobody really wanted to hear it then.  It was too late, and there was little the team could feasibly do, except delay the release. With Continuous Testing, the focus is on providing actionable feedback to people who really care about it and when they are truly prepared to act on it.

Contrary to popular belief, Continuous Testing can happen at any point in the application delivery lifecycle. At some point, the concept of Continuous Testing was inappropriately conflated with the trend of “Shift Left.” To deliver the right feedback to the right stakeholder at the right time, Continuous Testing needs to occur throughout the software delivery lifecycle — and even beyond that to production (e.g., monitoring information from production and feeding that back from the quality perspective).  Just as the name indicates, Continuous Testing involves testing continuously. Simply starting and finishing testing earlier is not, by definition, Continuous Testing.

Of course, testing continuously in a way that provides the right feedback to the right stakeholders at the right time is no easy feat. The short answer to why it’s so challenging is that the time available to test is constantly decreasing while the complexity of what we need to test is increasing. But it’s more than that. To better understand why “reinventing testing” is now so essential, let’s take a quick look at some of the core forces influencing modern application delivery.

Decentralization of Architectures and Teams

System architectures have oscillated from a centralized to a much more decentralized set of integrated systems. From the monolithic mainframe applications of the past, we’ve evolved into a set of integrated systems that are becoming significantly more distributed. Cloud native applications, the advent of microservices which require an extreme degree of orchestration — this is all part of the decentralization trend in system architectures.

The Development team structure has traditionally followed the system architecture trends. Development team structure went from large centralized waterfall type teams to much smaller and more decentralized teams.

As Development teams decentralize and grow more fragmented, it becomes increasingly difficult to maintain control over epics and deliverables.

Just imagine: instead of unified groups working on monolithic applications, we shifted to many small autonomous groups each working on smaller parts of a highly distributed system. How do you ensure a coordinated and compelling user experience across the overarching application?

Now, teams are oscillating back a bit: they’re not necessarily looking for ways to centralize the Dev team structure, but rather to collaborate more effectively within the decentralized Dev team structure.

Then there’s the Test team structure — following the same general pattern, but lagging a bit behind.

The Test team structure has moved from a highly centralized group responsible for all testing, to testing centers of excellence, to digital testing centers of excellence, to testers being embedded within Agile Dev team structures.  Now, Test teams (like Dev teams) are shifting toward more of a mixed model that’s largely decentralized but has centralized components available to test end-to-end interactions.

Proliferation of Tools

One impact of this team decentralization is that different teams select different tools to drive their quality processes. That selection of independent tools might fulfill the needs of individual teams — but what happens when you need to make a fast (often automated) release decision regarding the broader application, which probably involves multiple teams and toolsets?

Such a decision requires an accurate assessment about how the latest changes impact the user experience. However, the data required to understand that change impact is scattered across different teams and tools — open source, custom built and commercial.

Highly Interdependent Applications

Next, let’s shift to the extreme interdependency that inevitably stems from decentralized system architectures. Obviously, this interdependency is going to be a much more pressing pain for a QA team responsible for end-to-end business transactions that involve SAP, legacy apps, mainframes, web UIs, APIs, etc. than it is for a DevTest team building microservices for cloud native apps.  For the former, you can’t just grab an open source tool, play around with it, and succeed. You need to understand all the different technologies well enough to automate them, you need a way to feed secure, reliable compliant data into (and across) the technologies, and you need on-demand access (real or simulated) to all the different components associated with each of those transactions.

However, implementing test automation is only the start. Assume you have a series of transactions that might involve applications “A through E” — and you have automated regression tests built out for these core transactions. Now, imagine all the times that one of the many associated application components changes. For each change, you need to review and potentially update all of the associated regression tests.

Unless your test automation is resilient (built to accommodate frequent change), it’s just a matter of time before the results become ridden with false positives and the test suite fails to detect critical failures or negative change impacts. As the application components continue to evolve, technical debt accumulates and teams feel that an oppressive amount of work is required just to catch up (let alone keep up).

At this point, there’s certainly a temptation to fall back on manual testing. However, regardless of how many people you throw at the problem, there’s simply no way manual testing can deliver the right feedback to the right stakeholder at the right time in a modern delivery process.

Decentralization of Performance Testing

Finally, let’s look at one more impact of decentralized architectures and teams: decentralization of performance testing. This shift has naturally led to the proliferation and expansion of APIs, web services and microservices. A performance issue in any of these distributed components could have a ripple effect across the entire application. And now that new functionality is being released weekly, daily, or hourly, each decentralized team needs instant insight into whether their incremental changes negatively impact performance.

In-depth load testing by performance testing specialists remains critical — but it doesn’t provide the level of fast, on-demand load testing that’s critical for Agile and DevOps. Developers and testers need a way to expose critical performance issues before new functionality progresses through the delivery pipeline. To achieve this, they must be able to easily create load tests that provide fast feedback on the functionality they’re evolving. Beyond that, they must also be able to execute and scale those load tests as needed — without the exorbitant costs and efforts traditionally required to establish, configure and maintain a performance test lab.

The Path Forward

Every team is different. Some might be focused on automating traditionally manual processes while others might be wrestling with the orchestration and correlation of all the various test automation tools they’ve come to master. The challenge is getting to the point where you can report on whether an overarching application or project involving all these different teams — with different cadences, architectures, tool stacks, structures, and challenges — has an acceptable level of risk.

That’s why Tricentis provides a Continuous Testing platform which is optimized for all the various challenges that modern QA and DevTest teams face. Start by tackling your most pressing pains, then easily expand as expectations shift and new challenges emerge.

Tricentis helps enterprise testing teams:

• Expose change impacts in minutes with advanced, resilient test automation optimized for 150+ technologies;
• Modernize testing across SAP and packaged apps with the most comprehensive testing solution for the intelligent enterprise;
• Accelerate release cycles by orchestrating and scaling testing efforts across your teams, projects, applications, and tools (including open source);
• Reduce risks with centralized visibility/traceability, predictive analytics, and “release readiness” dashboards.

Tricentis features several core components that can be applied individually or in concert across your teams and projects:

If you’re intrigued and would like to learn more about the platform and its specific components,  we encourage you to read Gartner’s independent analysis, get a free trial or sign up for a quick demo.

Feature image via Pixabay.

InApps Technology is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Tricentis.