Skip to main content
// ple

// blog / prompt patterns

The security-review prompt: why "find all vulnerabilities" fails

The broad scan produces confident noise. The narrow, threat-modeled slice produces findings. The difference is entirely in the prompt.

Why the broad scan fails

"Review this codebase for security vulnerabilities" fails for the same reason it would fail as an instruction to a human: no attacker, no asset, no boundary — so the model optimizes for lookingthorough. You get a page of theoretical XSS in code that never renders HTML, and nothing about the deserializer that actually parses user input. Practitioners who have used LLMs to find real, disclosed vulnerabilities tend to converge on the same shape: minimal standing scaffolding, maximal targeted exploration — think first, then read one narrow slice with full attention. It's also an attention-budget argument— a whole codebase in context guarantees the vulnerable line gets a fraction of the model's attention.

The two-prompt pattern

Split the job the way a human reviewer would. First prompt: build the threat model. One sentence naming who attacks, through which surface, to gain what — then assets, trust boundaries, and the three most plausible vulnerability classes for thiscomponent. Ours is public: threat-model-first.

Second prompt: audit one boundary. Take a single trust boundary from the model and the narrow code slice behind it, and check only that: input validation, injection paths, authorization, fail-open behavior. Ours: trust-boundary-scan. Two rules in it do most of the work:

the two load-bearing rules
1. "Report 'no finding' per checklist item you cleared."
   → coverage becomes explicit; the model can't skip a check silently.

2. "Never invent findings for completeness."
   → kills the noise that makes broad scans unusable.

The prompt improves with every audit

The quiet advantage of the split: each audit teaches you which vulnerability classes and boundary types your stack actually produces, and that knowledge belongs in the prompt— a new checklist line, a sharper severity rule. That's an argument for keeping security prompts versioned: v4 of your boundary scan encodes lessons v1 didn't know, the diff shows exactly what your team learned, and a finding can cite the prompt version that produced it.

One honest caveat: an LLM review complements — never replaces — SAST tooling and human review for anything that matters. Its edge is the middle ground: sharper than grep, cheaper than a pentest, and available at 2am before you merge.