Troubleshoot Docker BuildKit cache not working

I keep getting full rebuilds of my Docker images despite using --mount=type=cache for Go/Node dependencies. The build context includes a docker-compose.yml that mounts an anonymous volume on the container, but I'm not sure if this is invalidating the M123 cache mount in BuildKit. I've confirmed --buildkit=true is active and the Docker daemon is running 0.40+. Could there be a race condition between the build context upload and when the cache lookup occurs?

It's literally in the first paragraph of the docs that you need to set DOCKER_BUILDKIT=1 and use '--mount=type=cache,target=/var/cache/apt --key=v1' for the apt layer specifically. The BuildKit cache isn

The fundamental issue with BuildKit's layer caching mechanism is that it operates on content-addressable hashing of every instruction in your Dockerfile, which means any upstream change—even a seemingly unrelated one like a RUN apt-get update followed by an install command where the package version isn't pinned — invalidates all downstream layers because BuildKit cannot guarantee that the result of that RUN command is idempotent across different build environments. This creates a cascading cache invalidation problem: if your first layer changes, every subsequent layer must be rebuilt even if the relevant code for those later stages hasn't moved at all. The solution you should implement involves multi-stage builds with explicit --mount=type=cache for package manager caches (apt, pip, npm) so that BuildKit can persist these intermediate directories across rebuilds without re-downloading everything from scratch every time a layer is invalidated. I should also note the distinction between --build-arg BUILDKIT_INLINE_CACHE=1 and using cachefrom in your build command, because those two flags do entirely different things at the BuildKit backend level: inline caching embeds cache metadata as an additional layer which can be pulled by remote builders, while cachefrom tells the frontend to explicitly pull a specific image's layers into the local Builder instance before starting. If you are running this in a CI/CD pipeline with fresh runners every time, neither will help without a shared registry backing — in that case, look at docker buildx build --cache-from type=registry which pushes your cache to a dedicated remote manifest store that any subsequent builder can consume regardless of its local state. Also worth noting the --mount=type=buildkit edge case where you can mount the entire Dockerfile as read-only for static analysis, though that's probably out of scope for what you're asking about right now but it does matter if your build scripts themselves need to be deterministic across