Your CI Pipeline Is an Attack Surface. Most Teams Just Don't Treat It Like One.
Quick question: when's the last time your security review looked at your package-lock.json diff, or the commit hash your GitHub Action is pinned to, as closely as it looked at your actual application code?
For most teams, the honest answer is never. And that's exactly the gap that's been exploited, repeatedly, in some of the most unsettling security incidents of the last few years — not through a clever exploit in your code, but through something upstream of it entirely.
Here's the thing about supply chain attacks on software: they don't care how good your input validation is. If the backdoor is compiled into the binary before your code ever runs, your OWASP Top 10 checklist was never going to catch it.
A Two-Year Con Job, Hidden in Plain Sight
In early 2024, a Microsoft engineer named Andres Freund noticed something small and strange: SSH logins on a test system were taking slightly longer than they should. Not broken — just off. A performance regression nobody else would have thought twice about.
That tiny anomaly unraveled one of the most sophisticated supply chain attacks ever caught. For roughly two years, someone using the identity "Jia Tan" had been quietly building trust as a co-maintainer of xz-utils — a compression library buried deep inside most Linux distributions, linked into OpenSSH through a downstream patch. Legitimate commits. Reasonable code review contributions. Patient, boring, credible work — exactly the kind that earns you commit access to something critical.
Then the backdoor went in. Partially hidden in binary test files that never showed up in a normal source diff. It modified how the library interacted with SSH authentication. It was one release away from landing in major distributions used by a meaningful fraction of the internet's servers.
Nobody exploited a code flaw. They exploited the social contract of open source maintenance — the reasonable assumption that an active maintainer with a good commit history is a maintainer you can trust.
Sit With That for a Second
If your dependency review process is "does it have an active maintainer and does CI pass," xz-utils should worry you. Both of those were true, right up until they very much weren't.
This Isn't a One-Off. It's a Pattern.
Rewind to 2018. event-stream was a wildly popular npm package — tens of millions of weekly downloads. Its original maintainer was overworked and under-resourced, like most maintainers of critical open-source infrastructure. So when someone offered to help maintain it, they said yes, and handed over publish access.
A later version quietly added a new dependency — flatmap-stream — containing code that specifically targeted a cryptocurrency wallet application several layers down the dependency tree. Not event-stream itself. A transitive dependency nobody explicitly chose or reviewed, riding in on a patch version bump of a package everyone already trusted.
Two completely different incidents. Same underlying failure mode: trust in a maintainer, extended to code nobody actually looked at.
The Attack That Didn't Even Need a Vulnerability
Then there's the one that isn't really a story about malware at all — it's a story about a default setting.
In 2021, researcher Alex Birsan showed that if your build system checks a public package registry for a name that also exists as a private, internal package — and many do, especially if the public version has a higher version number — the public package wins. He registered public packages under the same names as internal packages at more than 30 major tech companies. His (benign, proof-of-concept) code executed inside their networks. Not because of a bug. Because of how package resolution defaults to work.
That's the one that should really bother you, because it means you can do everything "right" — no sketchy maintainers, no typos, no obviously malicious code — and still get owned by a naming collision and a resolution order you never audited.
Six Places Something Can Go Wrong Before Your Code Even Runs
Most security training is aimed at runtime — injection, auth, the usual suspects. Supply chain risk lives earlier, at trust boundaries most review processes never touch:
- Source — your repo, branch protections, who can merge
- Dependencies — every direct and transitive package you pull in
- Build environment — the runner, its image, anything cached and reused
- Build definition — your CI/CD config itself is code, and it's rarely reviewed like code
- Artifact registry — where built things live and how they're fetched
- Deployment — what verifies the artifact being deployed is the one you think it is
A backdoor at any one of these defeats a perfectly secure application. Worth asking yourself right now: which of these six do you actually have visibility into?
What Actually Closes the Gap
None of this means throwing out your workflow. It means a handful of specific, concrete habits:
Pin third-party CI actions to a commit hash, not a version tag.
Weaker: uses: some-org/some-action@v2
Stronger: uses: some-org/some-action@a1b2c3d...A mutable tag means the action's maintainer — or whoever compromises their account — can change behavior underneath every pipeline using it, silently.
Scope your internal package names explicitly, and don't let your build tool fall back to public registries for anything internally namespaced. This alone would have stopped the dependency confusion attack cold.
Actually review the diff on dependency version bumps for anything critical — not just the changelog. event-streamlooked completely normal from the outside. The changelog wouldn't have saved you; the diff would have.
Generate SBOMs for what you ship. It won't stop an attack, but when the next xz-utils drops, it's the difference between "we know exactly which services are affected" and a week of frantic manual auditing.
Use short-lived, narrowly-scoped CI credentials. A build job that can build code shouldn't automatically also hold production deploy credentials it doesn't need.
Look into provenance and reproducible builds if you're not already. Tools like Sigstore/cosign have made artifact signing genuinely practical without the usual key-management pain. The SLSA framework gives you a maturity ladder if "start somewhere" is the question.
The Uncomfortable Truth
xz-utils wasn't caught by a security tool. It was caught by one engineer noticing that SSH felt a little slow. That's not a repeatable defense — it's luck, and we got lucky.
The actual defense is treating your build pipeline, your dependency tree, and your CI configuration with the same suspicion you already apply to user input. Not because open source maintainers are untrustworthy — most are doing unpaid, thankless work holding up software the entire internet depends on. But because trust, once it's the only control you have, is exactly what a patient attacker is optimizing to exploit.
Check your action pins today. It takes five minutes, and it's the kind of five minutes that occasionally matters a great deal more than you'd expect.
Member discussion