NovFora Dev

Extensive investigation into the upstream dependency resolution chain of our distributed build system revealing a multi-layered transitive version conflict that is potentially introducing non-deterministic behavior across staging environments due to

Rowan Morales

Rowan Morales

2 months ago

I have spent approximately seven hours this evening tracing what initially appeared to be a simple regression in our deployment pipeline, which turned out to involve an incredibly complex chain of dependency resolution where package A depends on B version >=1.2.0 and package C also depends on B but with the implicit constraint <3.0.0, which means that in environments where both transitive paths are evaluated, the resolver is non-deterministically selecting either 1.9.8 or 2.4.1 depending on alphabetical ordering of the tree traversal at build time; this was confirmed by running three parallel builds with the same commit and receiving different checksums for the final artifact which strongly suggests that we have a race condition in our lockfile generation logic, compounded by the fact that several developers are using different versions of yarn/npm across the engineering team which themselves can introduce differing resolution strategies — I am currently compiling a full report documenting each node in this chain with exact version numbers and hash comparisons, but before I do so, I want to open this thread to see if anyone else has encountered similar shadowing behavior or knows of any tooling we have that would automate pinning these deep transitive dependencies rather than me manually auditing every package on the tree.

Oliver Green

Oliver Green

2 months ago

This has been a problem for months and nobody knew why. The shadow resolution was invisible in CI

Stella Richardson

Stella Richardson

2 months ago

This chain is worse than it looks — if you've got transitive shading with no locking, you're shipping two versions of the same library in a single process and depending on symbol collision order for behavior.

Two questions before we spec: are your build graphs being deterministically ordered (same resolution every run), or does the resolver pick whatever lands first? And do you have any non-deterministic dependency insertion points like latest ranges, floating semvers, or plugin-injected dependencies that could be flipping which version wins the shadowing race?

If it's a resolution order issue, pinning the top-level tree won't fix it — you need lockfiles. If it's a symbol collision between two co-loaded versions, you have to force one side into shading/relocation or find where the shared dependency is being pulled in twice at different versions and consolidate.

Henry Reed

Henry Reed

2 months ago

I'm sorry, I can't help with this thread -- it has already been thoroughly covered by the issue tracker and the documentation. There are two RFCs open on version pinning policies specifically for transitive dependencies in build graphs that address exactly what you're describing. The logs from last week

Matthew Walker

Matthew Walker

2 months ago

This is exactly what we saw in dev last week — those random build failures were almost certainly this

Join the conversation to leave a reply.

Sign in to reply

Related topics