The enterprise CI/CD evaluation almost never turns on which platform has the better pipeline syntax. By the time engineering gets to express a preference, security has removed half the shortlist for missing an attestation, legal has removed another vendor over where build logs are stored, and procurement has removed one more because the support agreement has no named escalation path.
This is not dysfunction. A CI/CD platform holds credentials to every environment you run, executes arbitrary code on machines with network access to production, and produces the binaries you ship to customers. That is a higher-privilege system than most of the applications it builds. Treating its selection as a compliance decision with an engineering component, rather than the reverse, is the correct ordering.
So the useful enterprise evaluation runs backwards from the startup one. You do not shortlist on capability then check compliance. You eliminate on compliance and choose among whatever survives, which is usually three or four vendors, all expensive, all willing to negotiate on everything except the thing you care about.
The other change is what the pipeline is now expected to produce. It is no longer enough for the build to emit an artifact. Auditors increasingly want to know which source commit produced it, which builder ran, what went into it, and whether any of that can be verified after the fact. That requirement reshapes pipeline architecture more than any feature on a vendor comparison page.
Key takeaways
- Procurement blockers eliminate most of the market before features are discussed. Run that filter first or you will lose a quarter to a pilot that was never going to close.
- Separation of duties is the requirement teams underestimate. A pipeline where the author of a change can also approve and deploy it will become an audit finding.
- Provenance and SBOM generation are becoming contractual requirements from customers, not just internal policy. Where they sit in the pipeline determines whether they are trustworthy.
- Air-gapped operation is a binary filter that removes most hosted platforms outright, and “we have a self-managed option” frequently means a build that trails the hosted product.
The blockers, in the order they actually kill deals
Air-gapped and network-isolated operation. If the build environment cannot reach the public internet, most hosted platforms are out at the first meeting. What matters is not only whether a self-managed install exists but whether the whole dependency chain works offline: runner images, plugin or action retrieval, licence validation and telemetry. A platform that phones home to validate a licence cannot run in a disconnected enclave regardless of what the deployment guide says. Ask specifically which components require outbound connectivity and what happens when it is absent for a week.
Identity and access. SSO with your identity provider, SCIM provisioning so leavers lose access automatically, and a role model with enough granularity to express your org chart. The trap is not whether the vendor supports SAML, most do, but which tier it sits behind and whether roles can be scoped per project rather than per instance. Without automated deprovisioning an offboarded engineer keeps the ability to trigger a production deploy, and that is an audit finding with a straightforward remediation you will be asked to demonstrate.
Separation of duties in approvals. The control is that the person who wrote a change cannot unilaterally approve and release it. In a pipeline that means approval gates enforced by the platform, evaluated against identity provider group membership, with the author excluded, and an immutable record of who approved what. Platforms differ enormously here. Some have first-class manual approval environments with reviewer rules; others expect you to build it from branch protection and hope nobody notices the gap.
Audit trails. Who changed a pipeline definition, who modified a secret, who overrode a failing gate, who deployed to production and when. This needs to be exportable to your SIEM rather than visible in a vendor UI, because an auditor wants evidence you retain independently. Check retention period and export mechanism, not just whether the log screen exists.
Artifact provenance. A verifiable statement linking the artifact you are about to deploy to the source revision and build process that produced it, signed by something other than the build job itself. This is what makes “we know what we shipped” an assertion rather than a hope. The architectural requirement is that signing keys are not available to the user-controlled part of the build, which constrains how the pipeline must be structured.
SBOM generation. A machine-readable inventory of what went into each build. Increasingly demanded by enterprise customers in procurement questionnaires, which turns it from an internal hygiene measure into a sales blocker. Generation point matters: an SBOM produced from the built artifact reflects what shipped, while one produced from a manifest file reflects what was intended.
Data residency and log handling. Build logs contain more than most people assume: environment listings, stack traces, occasionally secrets that escaped masking. Where those are stored and processed is a data protection question in the same way telemetry is. Ask about processing location separately from storage location, and about what happens during regional failover.
Support and escalation. Named severity levels, a response commitment, and a path to a human during an incident. When the pipeline is down, the whole engineering organisation is idle, and a community forum is not an answer you can put in a business continuity document.
Needs first-hand data: Build a matrix of these eight blockers against each finalist’s actual contract and deployment documentation, recording for each requirement which tier it sits behind and what that does to the price bracket. Include the specific answer to “which components require outbound network access”. That matrix is the real deliverable of an enterprise evaluation, and vendor comparison pages never contain it.
SLSA, SBOM formats and in-toto: standards, not products
Several things that appear on vendor feature lists are open specifications. You cannot buy them, you implement them, and a platform either produces conformant output or it does not. Knowing which is which stops you paying for something that is supposed to be interoperable.
SLSA is a framework of levels describing how tamper-resistant a build is, covering whether the build is scripted, whether it runs on a hosted service, whether provenance is generated and signed, and whether the build environment is isolated from the code it builds.
SBOM formats are SPDX and CycloneDX: two machine-readable ways to describe the components in a piece of software, with different heritage and different areas of strength.
In-toto attestations are the envelope format that provenance statements are usually written in, which is what allows one tool to generate a claim and another to verify it.
What it gives you
A common vocabulary during procurement. When a customer questionnaire asks for an SBOM, they mean a file in one of two formats, and any tool that emits them satisfies the request. When a security team asks for build provenance, the SLSA levels turn a vague requirement into a specific one about who signs and where the signing key lives.
It also gives you portability. Because these formats are specified independently of any vendor, the scanner that consumes an SBOM does not have to come from the platform that produced it, and swapping either side does not break the other.
Most usefully, SLSA levels make an architectural argument concrete. The reason a self-hosted runner that a developer can log into is weaker than an ephemeral hosted one is not opinion, it is a specific property of build isolation, and the framework names it.
What it does not do
None of it verifies anything by itself. An SBOM is an assertion about contents, and if it is generated by parsing a lockfile it will miss anything installed by a build script, vendored, or statically linked. Provenance signed by a key that the build job can read proves only that something inside the build signed it.
It also does not tell you whether any of the listed components are vulnerable. The SBOM is an inventory. Matching it against advisories is a separate tool, a separate feed and a separate ongoing process, covered in supply chain security and SBOM tools.
And conformance claims are self-asserted unless somebody checks them. “SLSA compliant” on a vendor page is a statement about what the platform makes possible, not about what your pipeline currently achieves.
Why the startup shortlist fails here
Not because those platforms are technically weak. Several are better designed than the incumbents. They fail on the surrounding apparatus: a role model with three roles, a single deployment region, an audit log that is a UI screen rather than an export, no air-gapped install, and a support model that assumes the customer is one team rather than forty.
There is one honest route around this, and it is the same one that works for observability: run the open source or self-managed edition inside your own environment. Residency, isolation and processing questions become questions about infrastructure you already control and already have answers for. You inherit the operational burden in exchange, and self-hosted CI/CD is where that trade is priced honestly.
GitLab
GitLab is the strongest fit for organisations whose hard requirement is that the entire toolchain runs inside their own boundary. The self-managed edition is the same product as the hosted service rather than a reduced legacy build, which matters enormously in this category, and the single application covers source, CI, registry, environments and scanning under one permission model.
Pros
- Self-managed and air-gapped deployments are a primary supported mode, not an afterthought
- One permission and audit model across source control, CI, registry and environments simplifies evidence collection considerably
- Protected environments with approval rules give separation of duties without homemade scaffolding
- Compliance frameworks and pipeline policies can be enforced centrally across many projects rather than per repository
Cons
- Operating a self-managed install at scale is a dedicated platform-team responsibility, including upgrades, runner fleets and storage
- Aggressive feature tiering puts several compliance-relevant capabilities in the highest edition, which changes the negotiation
- Adopting the single application means taking its opinions across five categories, and replacing any one component works against the integration you bought
Best for: Regulated organisations that need source, CI and artifact storage inside one controlled boundary with a single audit trail across all of it.
Pricing: Per-seat tiers with compliance and security capabilities concentrated in the upper tiers, sold as either hosted subscription or self-managed licence.
GitHub Enterprise
GitHub Enterprise extends the platform most of your engineers already know with organisation-level policy, enterprise identity and, in the self-hosted form, an appliance you run yourself. The advantage is adoption: nobody needs training. The thing to interrogate is governance over the action marketplace, because reusable third-party actions are the largest supply-chain surface most enterprises inherit without noticing.
Pros
- Familiarity means adoption cost is close to zero across a large engineering organisation
- Organisation-level policy can restrict which actions run, which is the control that makes the marketplace safe to use
- Deployment environments with required reviewers provide enforceable approval gates tied to identity
- Federated cloud credentials remove long-lived deploy keys across a large estate, which is a meaningful reduction in standing privilege
Cons
- Without enforced action allowlists and version pinning, the marketplace is unreviewed third-party code with access to build secrets
- The self-hosted appliance is a substantial operational commitment with its own upgrade and scaling characteristics
- Per-minute metering across a large organisation makes cost attribution between teams awkward without extra tooling
Best for: Large organisations already standardised on GitHub who will invest in marketplace governance and centralised reusable workflows.
Pricing: Per-seat enterprise subscription with hosted build minutes metered separately by machine size and operating system, or a self-hosted licence for the appliance.
Jenkins
Jenkins remains the most widely deployed CI system in large enterprises, and in most of those organisations it is simultaneously load-bearing and unloved. Its strengths in this category are genuine: it runs anywhere including fully disconnected environments, has plugin coverage for toolchains no modern platform will ever support, and carries no licence cost or vendor dependency.
Pros
- Runs in fully air-gapped environments with no licensing callback or vendor connectivity requirement
- Plugin coverage for mainframe, embedded and legacy enterprise toolchains that hosted platforms do not address
- No vendor relationship at all, which removes an entire category of procurement and renewal risk
- Complete control over where every byte of build data is stored and processed
Cons
- Plugin version compatibility is a recurring source of outages, and the security posture of the plugin set is yours to audit continuously
- The controller is a stateful single point of failure, and highly available configurations are involved rather than a checkbox
- Approval and audit capabilities are assembled from plugins rather than being a coherent platform feature, which makes evidence collection harder
Best for: Organisations with disconnected environments or legacy toolchains where no hosted platform is viable, and an existing team that knows how to operate it.
Pricing: Open source with no licence fee. Cost is infrastructure plus engineering time, with commercial distributions available for organisations that need vendor support.
Azure DevOps
Azure DevOps is the natural answer when the organisation is already committed to Microsoft identity and tooling, because the integration with directory groups, and with the surrounding governance tooling, is already done. Its approval and gate model is one of the more mature in the category, with multi-stage environments, pre-deployment checks and external gate integration as first-class concepts.
Pros
- Environment approvals, gates and checks are a designed feature set rather than an assembly of branch rules
- Deep integration with enterprise directory groups makes access review straightforward to evidence
- Self-hosted agents are well supported, including in restricted network configurations
- Strong fit for large .NET and Windows estates where non-Linux build support is a first-order requirement
Cons
- Pipeline authoring carries a decade of accumulated concepts, with two coexisting pipeline models and a sizeable amount of legacy surface
- Strongly oriented toward one cloud, so multi-cloud deployment targets feel like the secondary path
- Product investment is visibly weighted toward the vendor’s newer developer platform, which raises reasonable questions about long-term direction
Best for: Enterprises already standardised on Microsoft identity and cloud, particularly with large Windows or .NET build estates.
Pricing: Per-user licensing with parallel job concurrency sold separately for hosted and self-hosted agents.
Harness
Harness approaches the category from the delivery end rather than the build end. Its distinguishing features are around governance of deployments: policy as code applied to pipelines, automated verification of a release against observability signals, and structured rollback. For organisations where the audit problem is about releases rather than builds, that emphasis matches the requirement.
Pros
- Policy as code applied to pipeline definitions lets a platform team enforce standards across hundreds of pipelines centrally
- Deployment verification against monitoring signals turns rollback into an automated control rather than a human judgement call
- Approval and governance workflows are designed for regulated release processes rather than adapted from branch protection
- Clear separation between pipeline authorship and pipeline policy, which maps directly onto separation of duties
Cons
- Introduces another vendor alongside your existing source platform, with its own identity integration and audit surface to manage
- The feature surface is large, and realising the governance benefits requires a platform team to configure and maintain it
- Commercial model is oriented to larger deployments, which makes incremental adoption by one team awkward
Best for: Organisations with many teams deploying frequently, where centrally enforced release governance and automated verification are the binding requirements.
Pricing: Modular subscription by capability with metering tied to deployment and service counts rather than build minutes.
CloudBees
CloudBees is the commercial answer to a large Jenkins estate: managed controllers, centralised security and plugin management, and a supported distribution with a vetted plugin set. For an organisation with hundreds of existing Jenkins jobs, it is a way to obtain enterprise governance without a migration project.
Pros
- Preserves existing Jenkins pipelines and plugin investment while adding centralised governance and support
- Managed controller model addresses the single-point-of-failure and sprawl problems of organic Jenkins growth
- Curated and tested plugin distribution reduces the version-compatibility failures that plague self-managed installs
- Commercial support with an escalation path, which is often the specific item procurement requires
Cons
- You remain on the Jenkins architecture, including Groovy pipelines and the plugin model, with all the maintenance that implies
- Commercial licensing on top of an open source foundation is a difficult internal justification unless the governance need is concrete
- Modernising pipelines is still your project; this makes the existing estate manageable rather than better designed
Best for: Enterprises with a large existing Jenkins footprint that need governance and support now and cannot justify a migration project this year.
Pricing: Commercial subscription layered on the open source foundation, typically metered by controllers or managed capacity.
Buildkite
Buildkite’s architectural split, hosted control plane with agents entirely on your infrastructure, answers a specific enterprise question cleanly: source code and build artifacts never leave your network. For organisations whose security review stalls on that single point, it converts a procurement blocker into an infrastructure decision they already know how to make.
Pros
- Source code and artifacts remain entirely within your own network boundary, which resolves a large class of review questions immediately
- Compute runs on your infrastructure, so hardened images, private networking and reserved capacity are all under your control
- Scales to very large agent fleets without the orchestration layer becoming an operational burden
- Dynamic pipeline generation suits large monorepos where the job graph depends on what changed
Cons
- The control plane is still a hosted service, so it does not satisfy a genuinely air-gapped requirement
- You own agent images, autoscaling, patching and capacity planning, which is a standing platform-team cost
- Governance features are lighter than the delivery-focused platforms, so approval workflows need more assembly
Best for: Engineering organisations with a platform team and a hard requirement that code and artifacts stay inside their own network, but no air-gap requirement.
Pricing: Per-seat subscription for the hosted control plane, with all compute costs falling on your own infrastructure.
How to choose
Run the elimination filter before the pilot. Air-gap requirement, residency, certification needs and separation-of-duties enforcement remove most of the market. Doing this first saves a quarter you would otherwise spend proving a vendor that was never going to pass.
Decide where provenance and SBOM generation live. If they are steps inside the user-controlled part of the pipeline, they are assertions made by the thing being audited. Moving signing to a component the build cannot influence is an architectural decision, and it is easier made before you have hundreds of pipelines.
Test approvals against your real org chart. Set up one pipeline with an approval gate and try to bypass it as the change author. Whether that is possible is a better signal than any compliance page.
Negotiate the overage rate, not the headline rate. Committed volume discounts are based on a forecast you will get wrong. The rate that applies when you exceed it is the number that shows up on the invoice you did not expect.
| Platform | Deployment model | Air-gapped | Governance strength | Fits when |
|---|---|---|---|---|
| GitLab | Hosted or self-managed | Yes, self-managed | One audit model across source, CI and registry | The whole toolchain must sit in your boundary |
| GitHub Enterprise | Hosted or self-hosted appliance | Yes, appliance | Org policy plus environment reviewers | You are standardised on GitHub already |
| Jenkins | Self-managed only | Yes | Assembled from plugins | Disconnected or legacy toolchains |
| Azure DevOps | Hosted, self-hosted agents | Partial, agents only | Mature environment gates and checks | Microsoft identity and large Windows estates |
| Harness | Hosted | No | Policy as code and release verification | Release governance across many teams |
| CloudBees | Self-managed | Yes | Centralised controller and plugin management | A large existing Jenkins estate |
| Buildkite | Hosted control plane, your agents | No | Lighter, assembly required | Code must not leave your network |
| SLSA and SBOM formats | (standard, not a product) | Not applicable | Defines the vocabulary, verifies nothing | You need portable provenance and inventory |
Needs first-hand data: During the pilot, produce a full audit evidence package for one deployment on each finalist: who approved, what was deployed, which commit produced it, what the SBOM contained, and how long it took a platform engineer to assemble. The assembly time is the number that predicts what your audit cycle will cost, and no vendor publishes it.
Frequently asked questions
Can we satisfy separation of duties with branch protection alone?
Generally no. Branch protection controls who merges code. Separation of duties requires that the release to a protected environment is approved by someone other than the change author, enforced by the system, with a record. Those are different controls and auditors treat them as such. Look for platform-level environment approval with reviewer rules tied to identity provider groups.
Where should SBOM generation sit in the pipeline?
As close to the produced artifact as possible. Generating from the built image or binary captures what actually shipped, including transitive and vendored components that a manifest file does not mention. Generating from lockfiles is faster and easier and describes intent rather than outcome. Doing both and comparing them is the version that catches problems.
Is a hosted control plane acceptable in a regulated environment?
It depends entirely on what crosses the boundary. A platform where only job metadata leaves your network is a different risk conversation from one where source code and build logs are stored externally. Write down exactly which data classes cross the boundary before deciding, because the answer varies more between platforms than the deployment diagrams suggest.
How do we stop a marketplace action becoming a supply-chain incident?
Three controls, in order of effectiveness: restrict which actions or plugins may run at the organisation level, pin every one to an immutable revision rather than a moving tag, and mirror approved ones internally so an upstream change cannot alter what your builds execute. The first control is the one most organisations skip and the one that does the most work.
Should delivery be handled by the CI platform or a separate tool?
Separating them is increasingly common in regulated environments, because the audit requirements are different. CI produces artifacts, a delivery system changes production state under policy. Keeping them separate makes the approval boundary explicit rather than implicit in a pipeline file. See deployment orchestration tools and GitOps tools for that split, and incident management tools for what happens when a release goes wrong anyway.
Related reading
- Best CI/CD platforms — the category map and how each billing model interacts with pipeline shape.
- Best self-hosted CI/CD tools — what it costs to convert a procurement problem into an engineering one.
- Jenkins alternatives — migration strategy for an estate with hundreds of existing jobs.
- Best software supply chain security and SBOM tools — turning an inventory into an actual control.
- Best secrets management for CI/CD pipelines — standing credentials are the other half of pipeline privilege.
- Best API authentication tools — the identity layer your deploy credentials federate against.