Scaling Security Design Reviews and why the time is now

This post was first published in Edition 26 of the Boring AppSec newsletter

The best AppSec teams invest a significant amount of effort in enabling builders (Developers, DevOps, Architects, Product teams, etc.) to build securely. This usually means two things:

1. Build artifacts that help all developers: Secure-by-default libraries, security standards, security champions program, and developer security training.

2. Detect security defects introduced by developers as early as possible: SAST, DAST, IAST, Manual PenTesting, etc.

While the first category truly enables developers, it is not just-in-time. The latter does integrate with the SDLC but waits for developers to make mistakes before pointing them out.

There’s a third, more critical category: Provide developers contextual feedback on the precise feature they are building before they start writing code.

Hypothesis

Providing your developers with contextual security requirements is a great way to avoid design flaws and reduce obvious security bugs later in the lifecycle. Security Design Reviews (SDR) are a great way to do this. Thanks to Gen AI, SDR is having its Snyk moment (i.e., seamlessly integrating a Security assessment into the developer workflow with minimal manual involvement from Security). AppSec teams must perform automated security design reviews (SDR) on every new feature their developers decide to build.

A 3-phase approach to building an SDR program

Before we dive deep into the “How,” here are a few things to keep in mind:

1. Define your goals. Do you want to “enable” developers or “enforce” these requirements? If it’s the former, your goal is to inform the developer and be done with it. If it’s the latter, you may need to find a way to “block” pipelines. This is much harder in the pre-coding phase.

   1. FWIW - I am not a fan of enforcing requirements this early in the lifecycle. AppSec teams lack the same context that developers do, and we are better off passing on the information and letting them take the call on what needs to be implemented. However, depending on the company’s culture, YMMV (your mileage may vary).

2. Who is doing the heavy lifting to generate the requirements? Developers, AppSec teams, or Security Champions? Is the goal to build a self-serve platform for developers or a platform to help AppSec teams scale the program? The program's user experience (UX) will change depending on the answer to this question.

3. It’s essential to understand how developers and product teams document their plans. Are all new features documented well in Jira? Is there a PRD and TechSpec associated with every new feature? Is everything a Slack thread (or god-forbid, an MS Teams thread? :P), or is all planning happening on a Whiteboard inside a conference room, and there is no trail of anything that happened?

   
   

Read more: A Guide To Current Threat Modeling Practices In SDLC

As with any new initiative, it’s best to break the problem down into smaller phases and learn from each one. Here’s how I’d recommend building the program:

Phase 1: Understand the landscape and build stepping stones

1. Understand where developers and product teams track new features. If you are lucky, there may be a central location where things are tracked (SharePoint, Google Drive, and Jira are popular choices).

2. Decide the right stage to perform the design review. Unlike code, planning documents (such as design documents or PRDs) are not always in a “done” state. They usually undergo drafts and reviews before being considered “final.” You will need to decide when to assess or scan the document. There is no correct answer here. It may be best to pick a stage and run with it. It’s pretty easy to change this, so the cost of inaction may be higher than choosing the wrong time.

3. Risk-rank each new feature. Define a “high-risk” feature and leverage Gen AI to determine if your scanning input fits the bill. This activity can help you understand how many new features need deeper security analysis when running continuously.

4. Set a flag to indicate that the security team will perform manual assessments on High-risk features. For now, you can ignore medium and low-risk features. In other words, the tool's purpose (in this phase) is to highlight high-risk features. While this may seem like a small win, seasoned AppSec folks will know how hard it is to scale this with precision (and without manual inputs from developers).

5. Leverage open-source tools like the Open AI Security team’s SDLCbot to do the heavy lifting on the AI portion. Be sure to modify the prompts to suit your company’s needs.

Phase 2: Auto-generate threats and requirements.

1. Now that you can assess each new feature, figure out what the outcome of the assessment should be. Is it a list of threats using STRIDE? A list of security requirements for developers? Just a security-relevant summary of the document?

2. Once you have the above, generate assessment results that match your requirements. Projects like STRIDEGPT can help you get up and running quickly, but you must significantly modify the prompts and how input is handled to make it work for your company.

   1. In general, single-prompt Gen AI apps are excellent for PoCs but hard to productionize. Use these tools to prove a concept. Only build the full-blown tool if the upside is significant enough to justify the effort needed. More on this later in the post.

3. The input (document, Jira ticket, etc.) will likely not contain sufficient information to produce the perfect output. Ensure the tool is built so that you can publish open questions or request more information from the developers.

4. Make it easy to send feedback to developers and product teams.

   1. Pet peeve: Don’t make developers and product teams visit your custom portal to understand requirements. They dislike it and will likely find ways to avoid consuming it. Instead, publish them where they typically read requirements.

Phase 3: Build a feedback loop

1. Enable back-and-forth communication between the tool and the builder. Results should change if the document changes.

2. Make it easy for developers to answer the open questions and automatically update results based on the answers.

A few thoughts before we end:

1. Why hasn’t this been done before? The input to SDR is unstructured. Traditional security tools need structured data to run rules against (SAST→code, DAST→traffic, and so on). All this changes with Gen AI. LLMs can extract context from unstructured data such as documents and architecture diagrams. A lot more needs to be done to scale SDR, but the underlying technical challenge is now a solved problem.

2. PoC v/s Production: Building a compelling PoC will be simple (especially with tools like Cursor being widely available), but building a full-blown solution will require human and tech investment. Your best bet may be to create a proof of concept (PoC), show it to your stakeholders, and gauge whether a full-blown product will indeed be beneficial for your company. If so, commit sufficient resources and develop a comprehensive solution. If you have an ML team in your organization, involving them now may be a good idea (you don’t need them for the PoC).

3. Build vs. Buy: Over the last nine months, I have spoken to hundreds of individuals in AppSec teams, startups, and large Security companies, who all agree that we can utilize Gen AI to scale threat modeling. There is no doubt that we will have multiple companies providing this offering. Having said that, if you have a security engineering team with reasonable dev + LLM chops, this problem can be solved internally.

Conclusion

Scaling threat modeling is an idea whose time has come. Implementing automated Security Design Reviews to provide developers with security requirements is a quick win. Beyond the apparent benefits for developers, these techniques can also help organizations meet compliance obligations, especially in industries such as healthcare and fintech.