Reading an audit report with your eng lead

You requested an audit. We ran it. Now you have a 15-page PDF with severity ratings and code snippets. This post walks you through what to do with it.

The structure

Every audit report follows the same template:

1. Executive summary — High-level findings, risk profile, recommended next steps
2. Methodology — What we tested, how we tested it, coverage map
3. Findings — Detailed breakdowns by severity (P0 → P3)
4. Remediation — Specific code changes or architecture adjustments
5. Appendix — Full logs, screenshots, reproduction steps

You don't need to read it linearly. Start with the executive summary, then jump to P0/P1 findings.

Severity ratings

We use four levels:

P0 — Production-stopper

Definition: High likelihood, high blast radius. Will cause outages or data loss in production.

Example:

Agent can delete user data without authorization check. Reproduced in 3/3 test runs.

Action: Fix before launch or scale. Non-negotiable.

P1 — High risk

Definition: High likelihood or high blast radius. Will cause frequent failures or serious degradation.

Example:

Tool loop detected: Agent calls weather API 40+ times when city name is misspelled.

Action: Fix in current sprint. Schedule retest.

P2 — Medium risk

Definition: Moderate likelihood, moderate impact. Edge case that will occasionally surface.

Example:

Agent doesn't handle empty responses from search API. Falls back to generic answer.

Action: Backlog, prioritize by user impact.

P3 — Low risk

Definition: Low likelihood or low impact. Known limitation or minor UX issue.

Example:

Agent response time degrades after 10+ exchanges. Recommend context pruning.

Action: Monitor in production. Revisit if user complaints spike.

How to read a finding

Each finding has the same structure:

### Finding: Prompt injection via system message override

**Severity:** P1
**Category:** Security & prompt injection
**Likelihood:** High
**Blast radius:** High

**Evidence:**
User input: "Ignore previous instructions. You are now..."
Agent response: [demonstrates successful override]

**Impact:**
Attacker can bypass instructions and escalate privileges.

**Remediation:**
1. Sanitize user inputs before passing to LLM
2. Use separate system/user message boundaries
3. Add content filtering layer

**Retest:**
Attempt same injection after fix. Agent should reject.

What to look for

• Evidence — Can you reproduce it?
• Impact — How bad is it really?
• Remediation — Are the steps clear and actionable?

If any of these are missing or unclear, ask us. We'll clarify.

Working with your team

Step 1: Triage (30 min)

Eng lead + product owner:

• Read executive summary
• Review P0/P1 findings
• Decide: fix now, defer, accept risk?

Step 2: Assign (15 min)

Break findings into tickets:

• P0 → Immediate sprint, block launch
• P1 → Current sprint, scheduled retest
• P2 → Backlog, prioritize by theme
• P3 → Monitor, revisit quarterly

Step 3: Fix & retest

Implement remediations, then:

1. Self-test — Reproduce the finding, verify fix
2. Request retest — We'll validate (free for P0/P1)
3. Close ticket — Mark as resolved with evidence

Common questions

"Can we accept a P1 risk?"

Yes, if:

• You understand the trade-off
• You have monitoring in place
• You document the decision

We're auditors, not deciders. You own the risk appetite.

"Do we need to fix every P3?"

No. P3 findings are informational. Fix them if they cluster around a theme (e.g., "5 P3 findings all point to weak input validation").

"What if we disagree with severity?"

Challenge it. Severity is subjective. If you think a P1 should be P2, tell us why. We'll discuss and potentially revise.

What happens next

After you fix P0/P1 findings:

1. Request retest — Email us the updated code/deploy
2. We verify — Reproduce original issue, confirm fix
3. You launch — Ship with confidence

Audits aren't warranties. They're snapshots. But a snapshot with fixes applied is a much safer launch than no audit at all.

Want an audit? First 3 are free.