ClearPenTest
Audit operations

The evidence worth generating before anyone asks for it

5 min read

Audits fail on evidence far more often than on controls. The difference between a smooth audit and a scramble is whether records are produced as a byproduct of work or assembled by a person afterwards.

Most first audits do not fail because a control was missing. They produce exceptions because a control ran and left nothing behind.

An auditor picks a date and asks what happened. A control with no record did not happen, as far as the audit is concerned, and no amount of describing the process afterwards changes that.

The sampling model

This is the mechanic worth internalizing. The auditor does not review everything. They select a sample, often dates or items of their choosing, and test those.

That has a consequence people miss: evidence has to be complete across the period, not merely present. A quarterly access review with three of four quarters documented does not pass three quarters of the time. It produces an exception, because the sample may land on the missing one.

What gets sampled

Access. Who was granted what, when, approved by whom. Who was removed, and how quickly after leaving. The periodic review, per period, with the outcome recorded rather than the fact that it occurred.

Change. How code reached production. A pull request with a review and an approval is excellent evidence because it exists at the moment the work happens. A deployment with no linked change record is the common gap.

Vulnerabilities. What was found, how it was triaged, what was fixed and when. This is where a penetration test report contributes, alongside the scanning cadence and the triage records. The report alone evidences an event; the process is what is being assessed.

Incidents. Any incident, its timeline, and the follow-up. A period with no incidents is fine and should be recorded as such, since "we had none" is a claim and a register showing none is evidence.

Vendors. Which subprocessors, what review was performed, when it was last refreshed.

Monitoring. That alerts fire, reach a person, and get triaged. The common failure here is logging without alerting: the activity is recorded, nobody is told, and the auditor's question about how you would know has no good answer.

Byproduct beats project

The distinction that determines whether renewals are easy:

Byproduct evidence exists because the work happened. A merged pull request with an approval. A ticket with a state history. An automated offboarding run with a log. Nobody assembled it, and it cannot have gaps in the months the auditor selects.

Assembled evidence is produced by a person before fieldwork, often reconstructed from memory and Slack. It has gaps precisely where attention lapsed, and it costs a person several weeks each cycle.

Moving from the second to the first is the single highest-value thing to do during a first audit, because it is what makes the second one routine.

The four that are usually missing

Periodic reviews with outcomes. Not "an access review was performed" but who was reviewed, what changed as a result, and who signed off.

Offboarding timeliness. Access removed, and when relative to the departure. Auditors check the interval, not just the removal.

Alert triage. The alert, who looked at it, what they concluded. Alerts that fire into a channel nobody acknowledges are a finding waiting to happen.

Risk decisions. A risk accepted rather than remediated needs a record: who accepted it, the rationale, any compensating control, and when it will be revisited. Written down, it is defensible. Undocumented, it is indistinguishable from an oversight.

A test worth running

Pick three controls you claim. Pick a date two months ago that you did not choose deliberately. Try to produce evidence that each ran on or around it.

If that takes more than a few minutes per control, the gap is evidence generation rather than security, and the fix is plumbing rather than policy. Better to find that now than during fieldwork, when the same discovery costs weeks.

Questions people ask

Why do SOC 2 audits produce exceptions when the controls exist?

Because auditors sample: they pick dates and ask what happened. A control that ran but left no record cannot be evidenced, and evidence that is incomplete across the period fails whenever the sample lands on a gap.

What is the most commonly missing audit evidence?

Periodic reviews recorded with outcomes rather than just occurrence, offboarding timeliness relative to departure dates, alert triage records, and documented risk acceptance decisions with named owners and review dates.

START WITH A CLEAR SCOPE

Get a scoped price without a discovery call

Scope an assessment