This is the only purchase in application security where you are not buying software. You are buying a few weeks of specific humans’ attention, and a document.
That changes everything about how you buy it. There is no free tier to try over a weekend. There is no trial that tells you whether it works, because the output is a report and you cannot evaluate a report you have not commissioned. The thing that determines quality is who is assigned to your test, and you generally cannot choose them and often are not told who they are. Two engagements from the same vendor, six months apart, can differ more than two engagements from different vendors, because the variable is the person.
The buying process reflects that. The vendor asks for scope, you have a call, you get a proposal with a number of testing days and a date. The date is usually further out than you wanted, because good testers are booked. Then you wait, the test happens, and a document arrives that either lands on an engineer’s desk and produces fixes, or lands in a compliance folder and produces nothing.
Which of those two outcomes you get is mostly decided before the test starts, in the scoping conversation and in the contract terms. Specifically: what is in scope and whether that matches where your risk actually is, whether the tester gets working credentials for every role in your application, whether retesting the fixes is included, and what the report looks like.
The deadline driving the purchase is usually an enterprise customer’s security questionnaire or a SOC 2 audit, and that deadline creates a specific failure. A test bought to satisfy a requirement gets scoped to satisfy the requirement, which means scoped small, scoped cheap, and scoped away from the messy parts. You get a clean report and learn nothing. That report is genuinely useful for the sales cycle and it is not security work.
Key takeaways
- You are buying a person’s time, so consistency between engagements is lower than for any software product. Ask who is testing, what their background is, and whether you can have the same team next time.
- Credentialed testing against every user role finds the flaws that matter in SaaS. Unauthenticated testing of a login page finds very little, and authorisation between tenants is where real breaches live.
- Retest inclusion is the contract term with the largest effect on value. Without it, the report is a snapshot and you pay again to prove you fixed anything.
- PTaaS shortens time to start and gives you findings as they are discovered rather than in a document weeks later. It does not automatically mean deeper testing, and the report your auditor needs still has to come out of it.
PTaaS and the traditional engagement
The category has split into two delivery models, and the difference is real but narrower than the marketing suggests.
A traditional engagement is a firm, a statement of work, a scheduled block of tester-days, and a report delivered at the end. Communication during the test is typically a shared channel and a status call. The relationship is with the firm, and continuity comes from asking for the same people again.
Platform-delivered testing, usually sold as PTaaS, wraps the same human testing in software. You scope through a web application, the engagement starts sooner because the platform matches you to available testers, findings appear in the platform as they are discovered rather than at the end, and there is an integration that pushes findings into your issue tracker. Retests are often requested through the same interface.
What genuinely improves with the platform model:
Time to start. Weeks rather than months, because the platform is managing tester availability across many engagements rather than one firm’s calendar.
Findings arrive during the test. This is the most underrated practical benefit. A critical finding on day two can be fixed on day three and retested on day four, instead of appearing in a document three weeks later when the relevant engineer has moved to something else.
Workflow integration. Findings as tickets with the fields your engineers need, rather than a PDF someone re-types into Jira.
A durable record. Historical engagements, findings, fixes and retests in one place, which is exactly what an auditor asks for in the following year and exactly what nobody can find when it lives in email.
What does not automatically improve:
Depth. The platform does not make the tester better. Depth comes from the tester’s skill, the time allocated, and how good your scoping was.
Consistency. Platform-matched testers rotate. If you want continuity, you have to ask for it explicitly and it is not always available.
Business logic coverage. The hardest and most valuable findings in SaaS come from understanding what the application is for. That requires the tester to spend time learning your domain, and no delivery model creates that time by itself.
A caution about the word “continuous”. Some products described as continuous pentesting are automated scanning with periodic human review. Automated scanning is worthwhile and it is a different thing, covered in DAST tools. Ask directly how many human tester-hours the engagement includes, because that number is the product.
Scoping: where the value is won or lost
More engagements are wasted here than anywhere else, and the fix costs nothing.
Give the tester credentials for every role. Admin, standard user, read-only, and at minimum two separate tenants or organisations. In a multi-tenant SaaS application, the highest-value question is whether a user in tenant A can reach data in tenant B, and a tester without two tenants literally cannot ask it. This is the single most common scoping mistake and it removes the most valuable class of finding from the engagement.
Include the API, explicitly. Modern applications put most of their logic behind APIs, and the web frontend is a client that happens to be first. A test scoped to “the web application” may or may not cover the endpoints the mobile app uses, and the endpoints nobody has a client for at all are the ones with weak authorisation. Say API, and hand over the specification.
Say what you are worried about. The engagement is a few weeks long and the application is bigger than that. If you know that the billing flow is held together by trust, or that the permissions model was retrofitted, say so. Testers are not graded on finding things you already suspected, and pointing them at the weak areas gets you more than letting them rediscover the map.
Do not scope out the ugly parts. The legacy admin panel nobody wants to talk about is where the finding is. Excluding it produces a cleaner report and a less safe product.
Agree rate limits and defensive controls in advance. If your WAF blocks the tester on day one, you have paid for a test of your WAF. Usually the right answer is to allowlist the tester at the edge so the application itself is tested, and separately confirm the edge controls work. Decide deliberately rather than discovering it mid-engagement.
Provide documentation and a walkthrough. An hour with an engineer explaining the domain model at the start of the test converts directly into better findings. Vendors who ask for this are showing you something about how they work.
Agree the rules for destructive testing and production data. Whether tests run against production or a production-like environment, what happens if the tester finds a way to delete data, and who is on call if something breaks.
Needs first-hand data: Before scoping, list every user role, every tenant boundary, every API surface and every integration in your product, and mark which ones have ever been tested by anyone. The untested list is your scoping priority. Most teams discover that the areas they assumed were covered by previous engagements were explicitly excluded from them, and the only way to know is to read the old statements of work.
What makes a report usable
The report has two audiences with different requirements, and a good one serves both without pretending they are the same reader.
Your engineers need to reproduce and fix. That means, per finding: the exact request or steps, the account and role used, the observed response, the affected endpoints and parameters enumerated rather than sampled, and a fix recommendation specific to your stack rather than a paragraph of generic advice. A finding that says “the application is vulnerable to insecure direct object references” without naming which objects is not actionable, and you will pay an engineer to rediscover what you already paid a tester to find.
Your auditor and your customers need assurance. That means a report with the firm’s name and credentials, the scope stated precisely, the dates, the methodology, the testers’ qualifications, a summary an executive can read, and a severity rating with a stated basis. For customer-facing use you also want a version you can share without disclosing unfixed vulnerabilities, which most firms provide as a letter or an attestation.
Things that separate a good report from a bad one:
Severity that reflects your context, not a generic score. CVSS gives a base score describing a vulnerability in the abstract. A good report explains reachability and impact in your application: who can exploit this, what they get, and what else it chains with. A report that is a list of CVSS numbers has outsourced the thinking back to you.
False positives are marked or absent. Findings copied out of a scanner without verification are the clearest signal that you paid for tooling and not testing. Automated output belongs in an appendix, labelled.
Business logic findings are present. If every finding is something a scanner could have found, that is your answer about the depth of the engagement. Missing authorisation checks, workflow steps that can be skipped, race conditions in state transitions, quota and billing bypasses: these are the findings that justify a human.
The absence of findings is explained. A report that says nothing was found should say what was tried. Otherwise you cannot distinguish a secure application from a shallow test, and those look identical from the outside.
Remediation guidance names your stack. Advice that names your framework, your library and your configuration gets fixed. Advice that quotes a generic checklist does not.
Needs first-hand data: Ask every vendor for a redacted sample report before signing, and score the samples against a fixed list: reproduction steps present, full enumeration of affected endpoints, stack-specific fix guidance, business logic findings present, scanner output separated, severity justified with context. Compare the samples side by side. This is the closest thing to a product trial that exists in this category, and the difference between vendors on that list is usually obvious.
Retests and turnaround
Two contract terms decide whether the engagement produces fixed software or a document.
Retest inclusion. After you fix the findings, someone has to verify the fixes actually work. This is not optional: a meaningful share of remediations are incomplete, and an incomplete fix for a critical finding is as bad as no fix, with the added problem that everyone believes it is done. Ask precisely: are retests included, how many, for how long after the engagement, do they cover all findings or only high severity, and does the retest produce an updated report you can hand to a customer. Vendors differ substantially here and the difference does not appear in the headline description.
Turnaround. Three distinct intervals matter and they are often conflated. Time from signing to test start, which is availability. Time from test start to first findings, which is where the platform model wins. Time from test end to final report, which can be surprisingly long and is what your audit deadline actually depends on. Ask for all three separately and get them in the agreement.
Cadence. An annual test finds what exists in the product on one week of the year, and most SaaS teams deploy continuously. The realistic pattern for a product under active development is an annual full engagement for the report your customers need, plus targeted testing when a major feature ships, plus automated scanning continuously. Nobody sells that as one package, and assembling it is the correct answer anyway.
Boutique firms: a category, not a vendor
A large share of the best testing in this market comes from small specialist firms of a few practitioners, and they are worth understanding as a category even though you cannot compare them on a table.
What they offer: direct access to senior people rather than whoever the platform matched you with, deep specialisation in a particular area such as cloud architecture, cryptography or a specific technology stack, continuity because the same person tests your product every year and remembers it, and flexibility in scope because the person you are talking to is the person doing the work.
What they cost you in convenience: no platform, so findings arrive in a document or a shared channel and your issue tracker integration is copy and paste. Availability is limited and good ones are booked out. Procurement is harder because your enterprise customer may not recognise the name. There is no marketplace to fall back on if the relationship ends. And quality varies more across the category than within any of the named platforms, because there is no aggregate reputation mechanism.
How to find a good one: ask your investors, your customers’ security teams, and other founders in your space, because this market runs on referral rather than search. Look at published research, conference talks and disclosed advisories, since a firm whose people publish is demonstrating capability in the only public way available. Then ask for a redacted sample report and score it the same way you would score a platform’s.
When a boutique is the right answer: when your product has an unusual technology or threat model that generic testing will handle badly, when you want the same tester year after year, or when you want a specific named person whose work you have read. When it is the wrong answer: when the driver is an auditor or an enterprise customer who wants a recognisable name on the letter, and when you need the engagement to start next week.
Cobalt

Cobalt built the platform-delivered model and remains the clearest example of it: scoping through a web application, a vetted tester community matched to your engagement, findings appearing in the platform during the test, integrations into issue trackers, and retests requested through the same interface. It is aimed squarely at SaaS companies buying their first or fifth test with a compliance deadline attached.
Pros
- Fast time from scoping to test start, which is the constraint that matters when a deadline is driving the purchase
- Findings appear during the engagement rather than at the end, so fixes can begin on day two rather than after the report
- Issue tracker integrations mean findings arrive as tickets with reproduction steps rather than as a PDF someone re-types
- Report formats designed for auditors and customer security reviews, which is the actual deliverable most buyers need
Cons
- Tester assignment rotates between engagements, so continuity and accumulated product knowledge need to be requested rather than assumed
- Depth is a function of the days you buy, and the default scoping conversation tends toward a standard size rather than what your application needs
- Platform convenience can substitute for scoping rigour, since a fast process discourages the slow conversation that produces good scope
- Less suited to unusual technology or threat models where a specialist would do better
Best for: SaaS companies who need a credible report on a deadline with findings integrated into their existing workflow.
Pricing: Sold in engagement units scoped by testing days and asset complexity, with subscription arrangements for recurring testing.
HackerOne

HackerOne is best known for running bug bounty programmes at scale, and it also sells structured, time-boxed penetration tests delivered by vetted researchers through the same platform. That combination is the point: an ongoing bounty programme provides continuous coverage from many perspectives, while a scoped pentest provides the report and the assurance a bounty programme cannot.
Pros
- One platform for both scoped testing and an ongoing bounty programme, which covers the continuous and point-in-time needs together
- Very large researcher community, so unusual technology stacks and niche expertise can be found rather than substituted
- Mature triage and disclosure workflow, which is the part of running a bounty programme most teams underestimate
- Strong recognition in enterprise procurement, which matters when the report has to satisfy someone else’s vendor review
Cons
- Bounty programmes generate submission volume that your team has to handle, and triage is real ongoing work even with vendor help
- Scoped pentest depth depends on the researchers matched to the engagement, with the same consistency caveat as any platform
- The bounty model rewards findings that are easy to demonstrate, which skews away from subtle business logic issues
- Running a public programme has reputational and operational consequences that need a decision at a level above engineering
Best for: Companies wanting continuous researcher coverage alongside scoped tests, with the maturity to triage incoming reports.
Pricing: Platform subscription for programme management plus per-engagement pricing for scoped tests, with bounty awards paid separately per valid finding.
Bugcrowd

Bugcrowd occupies similar ground with crowdsourced security across bounty programmes, vulnerability disclosure programmes and scoped penetration tests, differentiated mainly by how it matches researchers to programmes based on skill profiles and by the managed triage it layers in front of your team. For organisations whose main worry about crowdsourced testing is submission noise, that triage layer is the answer to it.
Pros
- Managed triage filters and validates submissions before they reach your engineers, which is the main operational cost of crowdsourced testing
- Researcher matching by skill and technology profile, so the people testing your product have relevant background
- Offers disclosure programmes as well as paid bounties, which is the low-cost first step for a team not ready to pay per finding
- Scoped pentests and crowdsourced coverage under one contract and one workflow
Cons
- Crowdsourced coverage is broad and shallow by nature, concentrating on what is quick to find and demonstrate
- Programme quality depends heavily on scope definition and reward structure, both of which take iteration to get right
- Managed triage adds cost and adds a layer between your engineers and the researcher who found the issue
- As with every crowdsourced model, coverage is unpredictable: you cannot direct attention to the area you are most worried about
Best for: Teams wanting crowdsourced coverage without absorbing raw submission volume into their engineering process.
Pricing: Subscription for programme management and triage, plus per-engagement pricing for scoped tests and per-finding awards for bounties.
Synack
Synack runs a vetted, private researcher network combined with its own reconnaissance tooling, positioned toward government and regulated enterprise buyers. Researchers are screened and background checked, testing traffic runs through a controlled channel that produces an auditable record of what was done, and results are delivered through a platform with continuous coverage rather than a single time-boxed window.
Pros
- Vetted and background-checked researcher network, which is the answer to the objection that blocks crowdsourced testing in regulated environments
- Controlled testing channel produces an auditable record of testing activity, which auditors and regulators specifically ask for
- Continuous coverage model rather than a single window, so newly shipped changes get attention through the year
- Strong fit for government and highly regulated buyers where provenance of the tester matters as much as the finding
Cons
- Enterprise commercial model and procurement cycle, which does not suit a small company with a deadline
- The controlled channel adds constraints on how testing is performed, which can limit certain techniques
- Continuous coverage still needs a point-in-time report for some compliance uses, so check what deliverable you actually receive
- Less flexible for unusual scopes than a direct relationship with a specialist firm
Best for: Regulated and government buyers who need vetted testers and an auditable record of testing activity.
Pricing: Enterprise subscription covering continuous coverage across defined assets, with engagement-based options.
NetSPI
NetSPI is a traditional consultancy with a platform layered on top: experienced in-house consultants delivering deep engagements across application, network, cloud and red team work, with findings delivered through a portal that supports triage, retesting and trending across years of engagements. It sits at the deeper and more expensive end of this list, and the depth is the reason to be there.
Pros
- In-house consultants rather than a matched marketplace, which gives more consistent depth and easier continuity across years
- Breadth across application, network, cloud and adversary simulation, so one relationship covers several kinds of test
- Delivery platform provides trending across engagements, which turns a series of tests into a picture of whether you are improving
- Depth suited to complex environments where a standard application test would miss the interesting parts
Cons
- Consultancy pricing and scheduling, so engagements are planned well ahead rather than started next week
- Oriented toward larger organisations, which makes it a poor fit for a small SaaS company buying its first test
- Broad service catalogue means the scoping conversation is longer and needs someone on your side who knows what to ask for
- Depth is worth paying for only if you will act on it, and a team that cannot fix findings quickly gets less from a deeper test
Best for: Larger organisations with complex environments who want consistent, deep testing across several disciplines.
Pricing: Consultancy engagements scoped by consultant days and asset complexity, with subscription arrangements for recurring programmes.
Intigriti
Intigriti is a European crowdsourced security platform offering bug bounty programmes, vulnerability disclosure and scoped penetration testing, with European data handling and a researcher community concentrated in the region. For companies whose procurement and data protection requirements point at a European vendor, that is a material simplification rather than a preference.
Pros
- European base and data handling, which removes a class of procurement and data protection objections before they start
- Active researcher community with strong regional participation, giving coverage from a different pool than the larger platforms
- Offers disclosure programmes alongside paid bounties and scoped tests, so a programme can start small and grow
- Responsive platform and support relative to its size, with less process between you and a decision
Cons
- Smaller researcher community than the largest platforms, so highly specialised expertise may be harder to source
- Less name recognition in some enterprise procurement processes outside Europe, which matters when the report is for a customer
- Crowdsourced coverage carries the same predictability limits as any bounty model
- Scoped pentest delivery is a newer part of the offering than the bounty platform itself
Best for: European companies who want crowdsourced security and scoped testing from a vendor inside their regulatory boundary.
Pricing: Platform subscription for programme management plus per-engagement pricing for scoped tests, with bounty awards per valid finding.
How to choose
Answer the first question honestly, because it changes everything downstream.
What is this test for? If the answer is an auditor or a customer questionnaire, then the deliverable is the report and the name on it, the timeline is fixed, and a platform vendor with a recognisable name and a fast start is the efficient choice. If the answer is that you genuinely want to know what is wrong with your product, the scoping conversation matters more than the vendor and you should be considering a specialist. Most teams need both, and the mistake is buying one while believing you bought the other.
When do you need the report? Work backwards through all three intervals: signing to start, start to findings, end to final report. Deadline-driven buyers routinely discover the last one too late.
Is your product unusual? An unusual stack, a cryptographic protocol, a hardware component, an uncommon architecture: generic testing handles these badly. This is where a boutique with the specific expertise is worth the procurement friction.
Can you fix what comes back? A deep engagement against a team with no capacity to remediate produces a longer list of things you know are broken. If remediation capacity is the constraint, buy a smaller test, fix everything, and buy a bigger one next year. Findings that sit unfixed are worse than not knowing, because now they are documented.
Do you want continuous coverage too? Bounty programmes give you many eyes over time and cannot give you a scoped report. Scoped tests give you the report and a fixed window. They are complements, and the bounty side has an ongoing triage cost that needs an owner before you start it.
| Option | Model | Start speed | Continuity | Picks itself when |
|---|---|---|---|---|
| Cobalt | Platform-delivered pentest | Fast | Request it explicitly | SaaS with a compliance deadline and a workflow to integrate |
| HackerOne | Bounty plus scoped tests | Fast for bounty | Community rotates | You want continuous coverage and the report |
| Bugcrowd | Crowdsourced with managed triage | Fast | Community rotates | Crowdsourced coverage without the submission noise |
| Synack | Vetted private network | Enterprise pace | Continuous model | Regulated or government, tester provenance matters |
| NetSPI | Consultancy plus platform | Planned ahead | Strong | Complex environment, depth across disciplines |
| Intigriti | Crowdsourced, Europe-based | Fast | Community rotates | European data handling is a requirement |
| Boutique firms (a category, not a vendor) | Direct consultancy | Depends on availability | Strongest | Unusual stack, or you want a specific named tester |
Whatever you buy, three things matter more than the logo. Give the tester credentials for every role and at least two tenants. Get retests in the contract. Ask for a redacted sample report before you sign and actually read it.
Frequently asked questions
How often should we run a penetration test?
Once a year is the compliance answer and it is not a security answer for a product that deploys continuously. The pattern that works is an annual scoped engagement producing the report your customers and auditors need, targeted testing when you ship something structurally new such as a permissions model change or a new tenant isolation boundary, and automated scanning running continuously in between. The annual test tells you where you stood on one week; the rest is what keeps the answer current.
Is PTaaS better than a traditional engagement?
It is better at logistics and neutral on depth. You start sooner, you see findings during the test rather than weeks later, findings land in your issue tracker, and retests are easier to request. What it does not change is that the quality of the test is the quality of the tester, and platforms rotate testers between engagements. If your product is conventional and your constraint is time, the platform model is clearly better. If your product is unusual or you value the same person knowing it year after year, a firm or a specialist is worth the slower start.
What should be in scope for a SaaS penetration test?
The authenticated application with credentials for every role, at least two separate tenants so cross-tenant authorisation can actually be tested, the API including endpoints with no user-facing client, the authentication and session handling, and any admin or internal tooling that touches customer data. Hand over the API specification and an hour with an engineer explaining the domain. Resist the urge to exclude the legacy component you are embarrassed about, since that is where the finding is, and decide in advance whether edge controls are allowlisted so the application itself gets tested rather than your WAF.
Will a pentest satisfy SOC 2?
It contributes and it is not the whole requirement. Auditors want evidence of a control operating over the audit period, which means the test, the findings, the remediation, and evidence that fixes were verified. A report with open critical findings and no remediation record is worse than useless in an audit, because it documents that you knew. The workflow that produces acceptable evidence continuously is covered in SOC 2 compliance automation platforms, and the findings themselves belong in the same queue as everything else, which is what vulnerability management platforms exist for.
Should we run a bug bounty instead?
They answer different questions. A bounty gives you many people looking at your production system over a long period, with no guarantee of coverage anywhere in particular, and it produces a stream of submissions your team must triage. A scoped test gives you directed attention on the areas you chose, within a window, and produces a report you can hand to an auditor. Most companies should have a vulnerability disclosure policy from early on, because people will find things regardless and you want them to have somewhere to send them, and should add a paid bounty only when there is an owner for triage and a team that fixes things quickly enough that researchers stay interested.
What do we do with the findings?
Put them in the same queue as findings from every other source, with owners and dates, rather than in a separate document. The specific discipline that matters: fix and retest the criticals before the engagement’s retest window closes, since that window is the free verification you paid for. Then look for patterns rather than instances. Three separate authorisation findings are one broken permissions model, and fixing the model is cheaper than fixing three symptoms and waiting for the fourth. The code-level classes that keep recurring are what SAST tools and API security testing are meant to catch before the next test does.
Related reading
- Best application security tools — where a penetration test fits among the controls that run continuously.
- Best DAST tools — automated testing against a running application, between engagements.
- Best SAST tools — catching the recurring code-level classes before a tester charges you to find them.
- Best API security testing tools — the authorisation flaws that dominate real SaaS findings.
- Best vulnerability management platforms — one queue for findings from every source, with owners and dates.
- Best SOC 2 compliance automation platforms — the evidence trail an auditor wants around the report.