A New Way to Track the DevOps Supply Chain – InApps 2022

The newly-launched open source initiative Grafeas defines an open, unified metadata exchange format and API that will provide a consistent way to aggregate and consume metadata from software components.

This central store of metadata promises to provide a whole new layer of visibility into the software supply chain — and to enhance auditing and governance capabilities.

Grafeas grew out of work at Google, but JFrog, Red Hat, IBM, Black Duck, Twistlock, Aqua Security, and CoreOS have joined the initiative.

“Modern [software development] processes are making things harder in a lot of cases —  creating smaller services that ship independently of each other,” said Stephen Elliott, Google product manager, developer platforms.

This process has exacerbated problems such as CIOs being confident that what’s being shipped into production is secure, the most up to date, was built in-house and has the right licenses.

“We were hearing from customers that this was a problem that was getting worse,” he said.

“A common question we get is, ‘How do I figure out if component X is running in production?’ Or ‘How do I figure out if this latest vulnerability affects anything in my pipeline that’s about to hit production?’

“These are not new questions because of containers. It’s just the way things are opening up and getting distributed, it’s just harder to answer.”

Google internally had the same problem, working with a large number of containers and releases, and having a flexible and trustworthy system that could track all the combined components.

Grafeas (“scribe” in Greek) provides a central source of truth for tracking and enforcing policies across software development teams and pipelines.

“If you have a really robust knowledge base about your artifacts, then you can build a really strong governance tool,” Elliott said.

With a second piece, Kritis (“judge” in Greek) organizations can enforce, in real time, governance policies for Kubernetes clusters that draw from the Grafeas metadata.

For e-commerce platform Shopify, which has 6,000 container builds a day and 330,000 images in its primary container registry, keeping track of all the components has been a challenge. The company explained in a blog post that it uses Grafeas to answer questions such as:

• Is this container deployed to production?
• When was the time this container was pulled (downloaded) from our registry?
• What packages are installed in this container?
• Does this container contain any security vulnerabilities?
• Does this container meet our security controls?

“By integrating Grafeas and Kritis into our Kubernetes pipeline, we are now able to automatically store vulnerability and build information about every container image that we create and strictly enforce a built-by-Shopify policy: our Kubernetes clusters only run images signed by our builder,” said Jonathan Pulsifer, senior security engineer at Shopify.

Google is hosting its alpha Grafeas API, but it hopes to build an ecosystem around it. Because it’s not coupled with any particular industry technology or packaging technology, it can run anywhere and store metadata wherever it lives, even in hybrid cloud use cases. It integrates and aggregates metadata from existing tools.

“We expect our collaborators to implement it in their own systems, but they’ll all be able to talk to each other. So with JFrog, we’re working on bi-directional sync, so if they’re using Artifactory and [Jfrog] XRay on-premises or Google products on their private cloud, they can all talk to each other. That’s really important for clients like financial services companies that have hybrid environments,” Elliott said.

He likened it to basic internet plumbing. With a strong central metadata store of the supply chain, it becomes much easier to build a dashboard.

“You don’t have to do a lot of custom engineering to integrate every disparate tool into some kind of dashboard,” he said. “If the API is being plugged into tools for build/test/deploy, the dashboard doesn’t have to query all the different tools, it just queries Grafeas — and only the metadata that you need, so you’re not getting some monolithic report. It can handle simple queries, such as ‘Has the QA team signed off on this image?’ to more rich queries, such as, ‘I’m the CIO and I want to see that status of certain vulnerabilities across all my deployed components.’”

And it opens up possibilities for further automation, according to a JFrog blog post:

“It’s easier to automate processes that add and extract metadata from components created with different technologies if all those components present their metadata in a standard format, and you can extract that metadata using a standard API. But that’s only the beginning. When you start adding your own private metadata using the same standards, you open up new opportunities for automated auditing and governance of software components you are using, whether they are open source components or proprietary components created in-house.”

Releases expected this quarter:

• JFrog’s Xray implementation of Grafeas API
• A Google artifact metadata API implementation of Grafeas, together with Google Container Registry vulnerability scanning
• Bi-directional metadata sync between JFrog Xray and the Google artifact metadata API
• Black Duck integration with Grafeas and the Google artifact metadata API

CoreOS, Google, Red Hat and Twistlock are sponsors of InApps.

Feature art via Shopify.

InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: JFrog.