The number is a claim, not a check
A version string like 1.4.0 is a promise about a declared public API. SemVer assigns that meaning only after someone declares what the public API is — and the specification says so in the imperative: “Software using Semantic Versioning MUST declare a public API. This API could be declared in the code itself or exist strictly in documentation” (Semantic Versioning 2.0.0).
That sentence is the whole problem in miniature. CI can verify the string. It cannot verify the declaration unless the declaration is an artifact the pipeline can read. Everything below follows from that gap.
What SemVer actually specifies
The rules are short and mostly about classification, not about tooling. The minor version “MUST be incremented if new, backward compatible functionality is introduced to the public API” (semver.org). The major version must be incremented for incompatible changes. The patch version is for backward-compatible bug fixes. Once a version is released, “the contents of that version MUST NOT be modified. Any modifications MUST be released as a new version” (semver.org).
Two consequences matter for CI design. First, the spec is a contract about a declared surface, not about the diff. A one-line change that alters a return type is breaking; a thousand-line internal refactor that preserves the surface is a patch. A retrieved publishing guide states the rule plainly: “classify the bump by impact on the public API, never by how much code changed” (JavaScript Package Publishing). Second, the spec anticipates mislabels. Its FAQ says that if a backward-incompatible change ships as a minor, the fix is “release a new minor version that corrects the problem and restores backward compatibility,” and that modifying the released version remains unacceptable (semver.org).
So the spec already tells you what to do after the incident. It does not tell you how to prevent it, because prevention depends on what you can observe before release.
What CI can verify cheaply
There is a real list here, and it is worth being precise about it because teams often conflate these checks with compatibility itself.
- Version-string grammar. The SemVer BNF and the published regular expressions reject malformed versions. This is a syntax check, not a semantic one.
- That a bump occurred. Comparing the manifest version against the last tag catches the “forgot to bump” failure.
- That the bump matches a declared intent. Changesets requires each pull request to add “a small markdown file declaring the bump level and a human summary,” and at release time the tool “aggregates pending changesets, applies the highest level” (JavaScript Package Publishing). semantic-release instead infers the version from Conventional Commits, where “
fix: yields a patch, feat: a minor, and a feat!: or BREAKING CHANGE: footer a major” (same source).
- That the package resolves under the consumer’s range operator. Caret is the npm default, “and the reason a minor bump reaches every
^ consumer automatically — and why a breaking change mislabeled as minor is so destructive” (same source). A resolution test tells you who will receive the release; it does not tell you whether they will survive it.
None of these checks answers the question the version number is supposed to answer. They verify that a classification artifact exists and is internally consistent. They do not verify that the classification is true.
What CI cannot verify from the number alone
Compatibility is a property of the consumer-visible surface, and the surface is larger than the schema. Three categories routinely escape schema-only checks.
Type-level changes. Tightening a parameter type, narrowing a return type, or making an optional field required can break TypeScript consumers even when runtime behavior is unchanged, because their build now fails to compile against the new types. The publishing guide treats the type surface as part of the public API for exactly this reason. A schema diff that only inspects JSON shapes will miss it.
Changed defaults and stricter validation. GitHub’s REST API versioning documentation lists “Adding a new validation rule to an existing parameter” and “Making a previously optional parameter required” among breaking changes that require a new API version (GitHub Docs). Both are additive-looking in a diff and behavior-changing in production.
Error contracts and status semantics. A stability policy from one API provider lists “removing or renaming a path, method, request field, response field or error code” and “changing a field’s type or meaning” as breaking for a stable surface (Olivares API stability). Error codes are part of the contract because consumers branch on them. A test suite that only asserts happy-path response bodies will not notice.
HTTP semantics add a further layer that schema checks cannot reach. RFC 9110 defines the meaning of methods, status codes, and conditional requests; RFC 7232 (obsoleted by RFC 9110) defines validators and preconditions such as If-Match and If-None-Match for avoiding lost updates (RFC 9110; RFC 7232). Whether a server still honors a precondition correctly after a minor release is a runtime property. A contract test can assert it for the cases you wrote; it cannot prove it for the cases you did not.
gRPC makes the same point from a different direction. The gRPC status code guide notes that DEADLINE_EXCEEDED “may be returned even if the operation has completed successfully” for state-changing operations, and that UNAVAILABLE is “most likely a transient condition” but “it is not always safe to retry non-idempotent operations” (gRPC status codes). A minor release that changes retry behavior or deadline handling can break consumers without changing a single field in the proto. Schema compatibility is necessary and nowhere near sufficient.
The heuristic: bump by blast radius, not by diff size
Ask what a consumer can observe — types, defaults, validation rules, error codes, auth requirements, retry and deadline behavior — and let that answer choose the digit. The diff size is irrelevant. The publishing guide’s example is the cleanest statement of it: a one-line fix that alters a return type is a breaking change; a large internal refactor that preserves the surface is a patch.
This heuristic is not a CI check. It is a decision rule that produces an artifact CI can check. The artifact is the bump-intent file or the commit footer. The check is that the artifact exists and matches the version. The judgment stays with a human or a tool that was told what to look for.
What a minimal pipeline can actually assert
A defensible pipeline does four things, and is honest about the fifth it cannot do.
- Declare the public API in a machine-readable place. An OpenAPI document, a proto package, or a typed interface file. SemVer requires the declaration; CI requires it to be parseable.
- Require a bump-intent artifact on every change to a publishable package. A changeset file or a Conventional Commit footer. Fail the pull request if the artifact is missing. This is the check that catches the “forgot to classify” failure, not the “classified wrong” failure.
- Run consumer-visible contract tests against the previous release’s expectations. Not just the new schema against the new code, but the old consumer’s assertions against the new build. This is where type-level changes, changed defaults, and error-code changes surface.
- Verify resolution under the range operators your consumers actually use. A caret-range resolution test tells you the blast radius of a mislabel before it ships.
The fifth thing — proving backward compatibility for every consumer — is not available. The spec does not provide it, and no retrieved source claims a tool does. What you can do is make the unverified part small and visible.
Deprecation is a contract too, and it has the same gap
SemVer’s FAQ says that when you deprecate part of your public API, you should update documentation and “issue a new minor release with the deprecation in place,” with at least one minor release containing the deprecation before removal in a new major (semver.org). That is a process requirement, not a runtime signal.
The runtime signal is a separate artifact. RFC 8594 defines the Sunset header, and RFC 9745 defines Deprecation; a Link header with rel="deprecation" points at the migration guide (API Contract Testing guide). The same guide recommends returning 410 Gone after the sunset date rather than 404, because 410 tells the caller the resource existed and was intentionally removed. GitHub’s documentation makes the same choice for unsupported API versions: “If you specify an API version that is no longer supported, you will receive a 410 Gone response” (GitHub Docs).
CI can verify that the headers are present on deprecated routes. It cannot verify that consumers read them. That is a documentation and client-SDK problem, and it is the same shape as the version-number problem: the signal exists, the interpretation is unverified.
When the mislabel ships, treat it as an incident
The SemVer-correct response is a corrective release, not a silent re-tag. The spec is explicit that released contents must not be modified and that the fix is a new minor that restores compatibility (semver.org). The engineering-correct response is a postmortem that answers one question: what did the classifier see that made it choose minor?
That question is answerable only if the classification was recorded. A changeset file records the bump level and a human summary. A Conventional Commit footer records the intent. An ADR or RFC that says “we treat type-level narrowing as a patch because our consumers are JavaScript-only” records the assumption that turned out to be wrong. Without one of these artifacts, the postmortem degenerates into “someone mislabeled it,” which is not a finding.
The decision record is the cheapest thing in this article and the one most often skipped. It costs a paragraph. It converts a future incident review from archaeology into reading.
What I would do differently
I would stop treating the version number as the compatibility check and start treating it as the label on a check that has to exist somewhere else. Concretely: declare the public API in a file the pipeline parses; require a bump-intent artifact on every publishable change; run the previous release’s consumer assertions against the new build; and write down the classification assumptions in the same pull request. The number then means what SemVer says it means, because something other than the number was verified.
The uncomfortable part is that the verification is always partial. The heuristic — bump by blast radius, not by diff size — is a decision rule, not a proof. It narrows the gap between what the API promises and what the consumer experiences. It does not close it, and any pipeline that claims otherwise is checking the label instead of the thing.
FAQ
Does SemVer require a machine-readable API declaration?
No. The spec says the public API “could be declared in the code itself or exist strictly in documentation” (semver.org). A machine-readable declaration is a CI convenience, not a SemVer requirement.
Can a schema diff catch every breaking change?
No. Type-level changes, changed defaults, stricter validation, and error-code changes can break consumers without changing the JSON shape. GitHub’s versioning documentation lists “Adding a new validation rule to an existing parameter” as breaking (GitHub Docs), and that change is often invisible to a structural diff.
Is a changeset file or a Conventional Commit footer enough?
It is enough to make the classification auditable. It is not enough to make the classification correct. The artifact records a judgment; it does not verify the judgment.
What should a consumer do when a minor release breaks them?
Pin to the last working version, report the break with the version numbers and the consumer-visible change, and expect a corrective release rather than a re-tag. The SemVer FAQ’s prescribed fix is a new minor that restores compatibility (semver.org).
Does gRPC have the same problem?
Yes, with different surface area. Proto field compatibility is checkable; deadline and retry semantics are not, and the gRPC status code guide notes that DEADLINE_EXCEEDED can occur even when the operation completed successfully (gRPC status codes). A minor release that changes retry guidance can break consumers without touching the schema.