ClearPenTest
Audit operations

Why auditors treat a scan and a penetration test differently

5 min read

A vulnerability scan proves a tool ran. A penetration test proves a control works. That distinction is why an auditor assessing CC4.1 accepts one more readily than the other, and why PCI DSS numbers them as separate requirements.

The two get conflated constantly, sometimes by vendors selling one as the other. For an audit the difference is not a matter of degree, it is a difference in what the artifact can establish.

What each one proves

A vulnerability scan compares software versions and configurations against a database of known vulnerabilities. It proves a tool ran against a target on a date and reported what it recognized.

A penetration test involves a person attempting to exploit weaknesses and establishing what can actually be reached. It proves something about whether a control functions.

That is the whole distinction, and it maps directly onto what auditors are assessing.

Why CC4.1 cares

SOC 2's CC4.1 asks for evaluations to ascertain whether the components of internal control are present and functioning. Read the verb: ascertain whether functioning.

A scan does not ascertain whether a control functions. It ascertains that a scanner recognized no known-vulnerable versions. Those are different claims, and the gap between them is the entire class of vulnerabilities that exist only in code you wrote.

The clearest example is cross-tenant access. A scanner cannot find it, because it has no concept of what a tenant means in your product. It is also, consistently, the most serious finding in multi-tenant software and the one enterprise buyers ask about by name.

Where PCI DSS is explicit

PCI DSS removes the ambiguity by numbering them separately. Requirement 11.3 covers vulnerability scanning, including quarterly external scans by an Approved Scanning Vendor. Requirement 11.4 covers penetration testing, internal and external, at least every 12 months and after significant change, following a documented methodology, with exploitable vulnerabilities corrected and testing repeated to verify.

Passing scans does not satisfy 11.4. This is not an interpretation; they are separate requirements with separate sub-requirements.

What scanning is genuinely good for

Frequency. A penetration test is an annual event and a scan runs weekly or continuously, which means scanning covers the eleven months between tests.

It is also the right tool for breadth: patch level across a large estate, dependency vulnerabilities in a build pipeline, configuration drift. None of that needs a person, and paying a person to do it is a waste of an engagement.

The sensible programme runs both: continuous scanning for known-vulnerability coverage, periodic testing for the logic that scanners structurally cannot evaluate.

The commercial trap

The thing to watch for is an automated scan sold as a penetration test, usually described as continuous or AI-powered penetration testing.

The tell is what appears in the report. If nothing in it required understanding how your product works, no one understood how your product works. A missing security header and an outdated library are findings a tool produced. A role that can reach another tenant's records is a finding a person produced.

Automation in testing is fine and increasingly standard. Reconnaissance, enumeration, and evidence capture should be automated, because that frees the tester's time for the judgment work. The question is not whether tools were used. It is whether anything in the engagement required judgment, and whether the report shows it.

For an auditor conversation

If your auditor has said a scan is sufficient, that is their call to make and the criteria give them room. Two things are worth confirming anyway. Whether your system description commits you to penetration testing, in which case the commitment is testable regardless. And whether the enterprise buyers reading the report will ask, because increasingly they do, and the report is written for them more than for the auditor.

Questions people ask

Does a vulnerability scan satisfy SOC 2?

It can, with a cooperative auditor, particularly for a Type I. It is weaker evidence for CC4.1, which asks whether controls are present and functioning, because a scan establishes that a tool recognized no known-vulnerable versions rather than that a control works.

Does PCI DSS accept scanning instead of penetration testing?

No. Requirement 11.3 covers vulnerability scanning and Requirement 11.4 covers penetration testing as separate requirements with separate sub-requirements. Passing scans does not satisfy 11.4.

Is automated penetration testing real?

Automation is a legitimate and standard part of testing: reconnaissance, enumeration and evidence capture should be automated. A product that automates the judgment as well is a scanner, and the test is whether anything in its report required understanding how your specific product works.

START WITH A CLEAR SCOPE

Get a scoped price without a discovery call

Scope an assessment