Key Metrics for Successful Feedback Loops – InApps 2022

This post is part of a series, sponsored by CloudBees, exploring the emerging concept of  ‘cloud native DevOps.’ Check back each Monday for future installments, and keep an eye out for the eBook in early 2019.

It wasn’t that long ago that software updates came in intervals of months and years. Big companies built big pieces of software and if anything went wrong from one version to the next, it was just something the user had to deal with while teams of developers went through monolithic piles of code to troubleshoot where they went wrong along the way. Software bugs aside, even unpopular features and customer complaints were met with months, if not years, of wait time before new features could be released to address the issue.

Nowadays, by comparison, companies like Facebook and Google deliver software updates on the order of hundreds if not thousands of times per day across their numerous products. Feedback loops have shrunk from years and months to days, hours and minutes.

At the core of this change is the DevOps approach to software, which relies on closely aligning developers with operations teams to create feedback loops that help to shorten the time between bugs and their fixes, feature requests and their release, and changed goals and their completion. The DevOps approach to software development, however, is not some mystical or out-of-reach technology relegated only to enterprises with deep pockets. Instead, with the confluence of many features of modern software design and deployment, most any company can use DevOps methodologies to create ever-tighter feedback loops in order to increase customer satisfaction and overall success.

First, let’s take a look at how we got to where we are today, what it means, and how we can use these conditions to better serve our customers and ourselves.

A Natural Progression from Agile

A key change in software development, as you might be aware, occurred around the turn of the century upon the introduction of a new software development methodology: agile. Before agile, companies used the waterfall methodology, wherein software was carefully planned from the outset and then implemented according to that rigid plan. With agile, instead of careful planning from the beginning, developers code in short iterations called sprints. During each sprint — usually a week or two at most — developers work to hit certain goals and then deploy their work. Upon deployment, they measure their success, re-evaluate their goals and adjust from there.

As Brian Dawson, DevOps Evangelist at CloudBees, explains, DevOps is merely a natural progression of several environmental conditions coalescing with the agile methodology. “DevOps is a practice that is largely focused on culture and emphasizes the alignment of developers and operations around the shared objective of releasing software repeatedly, reliably and rapidly,” Dawson said.

Over time, he explains, we moved away from boxes of CDs used to install software to constantly connected devices that updated software in the background. Software was no longer installed on the user’s machine, but instead run as Software as a Service, that could be updated and deployed in the cloud and immediately provided in rolling waves to millions of users. And at the same time, technologies like Infrastructure as a Service and containerization made it possible to quickly spin up servers in a matter of minutes and seconds instead of hours and days. With all of these conditions, the expectation for constant updates not only appeared, but the ability to deliver on that promise also became available to software teams both large and small, with continuous integration and continuous delivery (CI/CD) systems.

“With SaaS and PaaS, we don’t need to wait on Ops teams anymore to stand up a server,” says Dawson. “The cloud provided developers and QA teams rapid access to compute resources and infrastructure in a way that revolutionized things.”

Demystifying DevOps

Much like agile, using the DevOps methodology isn’t as simple as hiring someone or implementing a solution. Instead, DevOps is an approach to software development that starts at ideation and continues through to deployment, using monitoring along the way to identify where improvements can be made.

“A clear and unifying measure of DevOps is to show how work ultimately impacts the customer and the business,” explains Ravi Tharisayi, Director of Solutions Strategy at New Relic. “Whether you’ve adopted containers, deployed 2x faster, or migrated to the public cloud, if the results can’t be measurably felt by the customer and the business, it’s hard to argue success.”

In other words, deploying new technologies and adopting new methods is only as good as the final result. It can be easy to get wrapped up in meeting abstract goals when employing a new methodology, but the ultimate goal of DevOps is still to better serve your customers.

“While increased velocity is a huge imperative for organizations today,” Tharisayi further explains, “that velocity can be useless or even counterproductive without the right data to direct that velocity to the right problem with measurable results for everyone to see.”

So what does all of this mean for the modern software development team looking to increase performance?

Feedback Loops and Key KPIs for Success

While key performance indicators (KPIs) can vary from company to company, or even project to project, there does appear to be certain ones that comprise a sort of industry standard. For example, the 2018 DevOps Research & Assessment (DORA) report cites five metrics as key indicators of success. The group has tracked deployment frequency, lead time for changes, time to restore service, change failure rate, and overall product availability, over the last five years. According to the report, the companies that perform well on those five indicators “do significantly less manual work across all vectors, spend more time doing new work, and spend less time remediating security issues or defects than their low-performing counterparts.”

Meanwhile, a CircleCI report on “Three Critical Development Metrics for Engineering Velocity” similarly finds that stability, commit-to-deploy time (CDT) and deploy frequency are “the best predictors of an organization’s velocity and growth.”

Whatever initial measures you choose to observe, however, they are just the beginning. Tharisayi cautions that the key to DevOps is to customize your monitoring and tighten your feedback loops to focus on the key ingredients of your particular success.

“KPIs on a dashboard are certainly important for building a shared understanding among other uses,” Tharisayi says, “but we see high performing teams diving deep into data — filtering, faceting, and iteratively exploring — to understand what to focus on and where reliability issues lurk.”

In addition to the basic KPIs used in the aforementioned reports, Dawson offered numerous others that could be used to dive deeper into measuring success, from code complexity to commit frequency to customer satisfaction. Indeed, KPIs can extend beyond the basics of the CI/CD pipeline to help determine what adjustments may be required.

“It is really about visibility,” Dawson says. “We have a new level of visibility and insight into how we are delivering software — and a new ability to share and compare those insights — that provides us a new level of clarity when it comes to optimizing what we do.”

New Relic is a sponsor of InApps.

Feature image via Pixabay.

InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Velocity.