4 New Year’s Resolutions to Integrate Security into DevOps – InApps 2022

GitLab sponsored this post.

It is widely accepted that cloud native development and continuous cloud deployments cause disruptions in traditional security models. However, potential problems and vulnerabilities these disruptions can pose can far outweigh the opportunities they offer. Success — or failure — hinges on attention paid to a few foundational elements critical for evolving a classic secure development lifecycle (SDLC) and achieving security in DevOps at enterprise scale.

Making a resolution to improve these key areas of your business will lay a solid foundation and set your organization up for lasting DevSecOps transformation.

Based on lessons learned from Axway’s transformation and guidance from recent studies (2019 State of DevOps Report and the Building Security In Maturity Model [BSIMM] Version 10), we’ll describe four foundational resolutions to ensure your security program keeps pace with the velocity and innovation required by today’s DevSecOps lifecycles.

People and Culture

Nothing can be achieved without motivated and enabled team members to execute the heavy lifting required to integrate security tools and processes into the DevOps lifecycle. Attention to organization structure and how developers and operations collaborate with a core security team pays in multiples to achieve scale and velocity for enterprise software security objectives.

Centralized Software Security Team

In a DevSecOps culture, it is critical that all contributors understand their responsibilities and have up-to-date visibility on the security status of each delivery.

The core software security group aligns and collaborates with the individual DevOps teams to make sure security requirements are integrated into the CI/CD pipelines and assure the security of every software release.

Most software security initiatives start with a core software security team and a named security program leader. This core group may be known as “Software Security Group” (SSG) as defined by the BSIMM framework.

As the organization moves to DevOps delivery, the SSG works with development teams to integrate core security tools used for automated DevOps testing, such as Secure Static Code Analysis (SAST), Dynamic Analysis (DAST), third-party Software Component Analysis, Container Security Scanning, etc.

The core security group also advises the business on acceptable levels of risk when difficult decisions need to be made, such as deferring resolution of security finding to a future release.

Resolution 1 — Make Sure Your Core Software Security Group (SSG) Is Aligned with DevOps Teams and if You Don’t Already Have an SSG, Resolve to Establish One Immediately.

“Without an SSG, successfully carrying out security activities across a software portfolio is very unlikely.” — BSIMM

The core software security group (SSG) should be full-time security engineers with experience in software development, threat modeling, penetration testing and secure architecture. This team is generally responsible to define security requirements, policies, develop threat models and perform manual pentests.

Where to find them — BSIMM also advises, if you are building an SSG from scratch, it’s best to start with the developers and operations teams already embedded in the DevOps program and teach them about security practices rather than starting with IT security professionals and attempting to teach them all about software development, CI/CD and SDLC.

A core software security group is critical to establish early wins in a DevSecOps program, however, the SSG alone is usually not sufficient to reach enterprise scale and elite level continuous delivery performance in DevOps.

Security Champions

In large organizations with hundreds of software applications, development projects and microservices, it’s ultimately the individual project teams’ responsibility to ensure security objectives are met on each deployment. The Security Champion is a critical contributor, who performs the integration of security tools and processes into the DevOps cycle.

Security Champions are developers, product owners and other technical contributors assigned to specific DevOps projects and therefore have the most knowledge about the individual product or microservice — i.e., its purpose, architecture, codebase and specific pipeline configurations. Typically, they are full-time developers that dedicate a portion of their time to integrating security tooling into the CI/CD pipeline and collaborating with the software security group on threat modeling, security testing, vulnerability remediation and other security requirements.

Thanks to their closer proximity to the end-product and the orchestration pipelines, Security Champions may be “ahead of” the core security team on the innovation curve, devising and developing new approaches and integrating new third-party security tools into the development pipeline and production-security monitoring. Open collaboration between the core Software Security Group and Security Champions allows new approaches to rise up from grassroots solutions and become enterprise-wide standards and best practices.

“Security becomes a shared responsibility across delivery teams that are empowered to make security improvements. Security teams are able to act in an advisory role, leading to time-saving and security-enhancing capabilities” — 2019 State of DevOps Report

Resolution 2 — Launch or Enhance Your Security Champions program.

This program could take many forms, depending on the size and maturity level of your organization. Many organizations refer to this team as “Security Champions,” but whatever moniker you decide to use, you should identify these individuals, celebrate their contributions and provide the resources they need for success.

• Provide advanced security training based on roles and platforms in-use. Security Champions training should go well beyond the basics of “top-10 code vulnerabilities.” For example, your DevOps Security Champions may need Kubernetes, Serverless and Container specific security training.
• Create an online community and face-to-face meetups for discussing challenges, solutions and best practices. Host structured and unstructured sessions where the core security team and the champions can compare notes on what’s working and what’s not working in their respective DevOps projects.
• Live all-hands events such as a Capture-the-Flag (CTF), using open source CTF platforms like OWASP Security Shepherd, are a great way to build the skill set and camaraderie of the Security Champions.
• Rewards and recognition such as “Champion of the Quarter,” t-shirts and other giveaways are a great way to spread security culture and awareness.

Where to find them: for starters, most security champions will find you. They are the DevOps team members already expressing a deeper concern in security, by raising security risks, committing security updates and may have already completed advanced security training on their own time. Ideally, choose members that have demonstrated ability to influence and drive change in their project team.

Process and Governance

While DevOps teams and Security Engineers are empowered and motivated to execute the DevSecOps transformation, they will not get far without some guidelines and guardrails.

It’s often said, “security is everyone’s responsibility.” To translate this slogan into action requires defined processes and governance so that everyone understands DevOps security requirements and security responsibilities for their role.

Clarify and Align on DevOps Security Requirements

As cloud native development has brought along new technologies and frameworks, it’s important that security requirements evolve to address the latest cloud security risks and top threats for Kubernetes, containers and serverless computing. Development stage requirements now “shift left” to include post-deployment security requirements like monitoring and intrusion detection in the upfront design.

Few things can be more frustrating and demotivating for a DevOps team than to reach the release stage and then have to postpone deployment or rollback due to a security requirement being misunderstood or entirely overlooked.

It’s up to the security program leaders and line-of-business executives to define and agree on where to set the bar and prioritize security into the development lifecycle.

Whatever unique requirements make sense for your company, it’s imperative  DevOps security gates are clearly defined and all teams understand their responsibilities to achieve a passing security sign-off.

Resolution 3 – Clearly Define DevOps Security Passing Criteria and Prioritize These Requirements into the Product Backlog.

There is obviously no one-size-fits-all definition for DevOps security requirements.

The degree of formality and specific procedure you implement depends on too many factors to list, but the critical resolution is to set it down clearly in writing and make sure all DevOps teams buy-in. Top-down or bottom-up approaches can be successful, depending on your organization and culture. Often, engineering-led cultures prefer a bottom-up approach to achieve successful buy-in and ownership.

Provided as an example, the Axway Product Security Group has defined an “Application Security Bar.” This includes clearly defined passing criteria for security gates, such as:

• Threat Modeling.
• Third-party Component Analysis.
• Dynamic Analysis (DAST).
• Static Analysis (SAST).
• Container Security.
• Manual Pentest.
• Infrastructure Hardening.
• Security Monitoring.
• Production Vulnerability Scanning.

As opposed to a one-size-fits-all specification, the Application Security Bar is a framework, applied as applicable and based on unique characteristics of each DevOps project.

Project teams align with an Initial Security Review (ISR) during the planning phase of each new cycle, to highlight specific security risks, determine which controls apply and agree on when they will be in a compliant/passing state.

As a result, all cross-functional stakeholders agree on the criteria and plan to achieve the security requirements for each release, which is validated by a Final Security Review (FSR).

Mature DevSecOps teams have automated security testing in each merge request and scheduled pipeline jobs. This Continuous Security Review (CSR) process allows teams to release multiple times per day.

Automate Security Risk Sign-off

If you insist on the perfect achievement of all requirements (security or otherwise) on every release, you will not be shipping much software. You need to establish a security risk acceptance procedure and instrument the approval and enforcement directly in your DevOps projects.

While the core Security Group and Security Champions have the knowledge and wisdom to determine which security issues are must-fix release-blockers and which can be deferred with a low risk, the ultimate accountability for risk must rest on an authorized business owner.

Requiring security sign-off provides multiple benefits:

• Raise the visibility on security issues and risks to the business and executive level.
• Accountability and monitoring in the process as teams must deliver on the resolution timelines agreed to in the risk sign-off.
• In worst-case scenarios, if there is an actual security incident as a result of deferring an issue, accountability rests on the proper business level and not the individual contributors.

As teams strive to deploy updates several times per day, waiting on the signature of a busy executive can significantly impact the workflow of a global DevOps team.

Resolution 4 — Automate Your Security Sign-off Procedure to Accommodate Accelerated DevOps Delivery Cycles.

A risk acceptance procedure defines who has authority to approve based on impact and severity, types of issues that can be deferred and acceptable timelines for resolution (never allow an unbounded timeline, for example). The approval process should be automated, for example, by executing the approver’s decision in a JIRA ticket and enforcing the fixed timeline directly within the CI/CD pipeline.

Provided as an example, Axway’s Continuous Security Review (CSR) procedure includes an automated risk sign-off process.

• The CSR sign-off defines authority levels to defer a valid security finding based on severity and timeline.
• Decisions are executed and tracked in JIRA and the expiration dates are automated in the build pipelines.
• If the issue is not resolved within the approved timeline the build fails.

This enables a daily deployment while ensuring the DevOps project meets security requirements within acceptable timelines and SLAs.

There is a multitude of security requirements to address in a mature DevSecOps program (see Figure 1, above). Resolve to focus on the elements highlighted in this article and you will be well-positioned to establish the cultural and process foundations needed to achieve elite-level performance.

For more content on how to integrate security in DevOps or to hear Axway discuss this topic in full during their talk, consider registering for GitLab Commit SF January 14.

Feature image via Pixabay.

InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Resolve.