Preparing for a penetration test: 7 decisions that make the report worthwhile
Objective, scope, black box or grey box, environment, follow-up over time: lessons from our
client engagements to get the most out of a penetration test.
By Arnaud Courty, PASSI Lead Auditor
6 min read
Monday morning, 9 a.m. The audit team is ready to start a five-day penetration test. At
9:15, the test accounts don't exist yet. At 2 p.m., the web application firewall blocks our
IP addresses. The test actually begins on Tuesday at noon.
A day and a half lost out of five, and it's nobody's fault: the test simply hadn't been
prepared.
Our observation, engagement after engagement, is clear: much of a penetration
test's value is decided before day one. Here are the seven decisions that make the
difference, illustrated by recent engagements we carried out for our clients.
1. Define an objective, not just a scope
"Test our application" is not an objective. The first question we always ask is:
why are you doing this test?
Each objective steers the engagement differently: a customer's contractual requirement,
certification (ISO 27001, HDS, Ségur framework), validation before going live, assessment
after an incident. Formalizing it from the start avoids delivering an excellent report that
answers the wrong question.
2. Define the scope precisely
The scope must be defined unambiguously: URLs, IP addresses, APIs, mobile apps,
environments, features included and excluded.
Two reflexes to adopt:
list every instance, including partner variants, white-label versions and subdomains;
identify third-party components (identity provider, payment solution, hosting provider), which cannot be tested without their owner's consent.
3. Choose between black, grey and white box
Approach
Information provided
What it simulates
Black box
None, apart from the target
An external attacker with no access
Grey box
User accounts, documentation
A malicious customer, a compromised account
White box
Source code, architecture, administrator access
An in-depth review
For a SaaS application, grey box testing is almost always essential: the most critical
vulnerabilities sit behind authentication.
4. Prepare test accounts and data
This is the most common cause of lost days. You need:
several accounts per role (user, manager, administrator);
at least two separate customer organizations;
realistic test data, without real personal data;
an access check the day before the start.
The second point is essential for any multi-tenant solution. In the engagement for the
vendor pursuing a certification, the API served many businesses and public bodies from the
same infrastructure. The central question was therefore tenant isolation:
can a user from one customer, by tampering with a request, access another customer's data?
This question can only be tested with at least two accounts belonging to two different
organizations.
5. Choose the environment and anticipate impacts
Test in production or in pre-production? Our recent engagements illustrate both cases.
In every case, plan for a recent backup, allowlisting of the auditors' IP addresses unless
the goal is to test filtering, and informing your hosting provider and operations teams.
6. Know when a penetration test is not enough
Some situations call for more than a standard penetration test.
7. Plan for follow-up over time
A penetration test is a snapshot. The system, on the other hand, keeps changing.
At a minimum, plan from the start:
a debriefing meeting with the technical teams;
remediation time in the development schedule;
a retest to verify that the fixes are effective, often expected by customers and certification auditors.
The checklist to keep at hand
Test objective formalized (certification, customer, go-live, incident)
Precise scope, with every instance and variant listed
Third-party components identified and excluded or authorized
Approach chosen (black, grey or white box)
Previous test report shared with the auditors
Test accounts checked, across several roles and several organizations
Environment chosen and representative, backup taken
IP addresses authorized, hosting provider and teams informed