GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||
GeezerVison is showing github.com — Keep the Docker image's toolchain layers stable across releases, and slim them by lelia · Pull Request #439 · SocketDev/socket-python-cli Conversation
|
| Upgrade | Layers reused | New compressed bytes |
|---|---|---|
| 2.11.5 → 2.11.6 | 8 of 27 | 1,025 MB of 1,046 |
| 2.11.10 → 2.11.11 | 8 of 27 | 1,028 MB of 1,049 |
The base image digest was the same across 2.11.10 → 2.11.11, so the churn didn't come from upstream. The release ARGs are now declared just before the CLI install. tests/unit/test_dockerfile.py lists the ARGs allowed above the toolchain layers (GO_VERSION, JAVA_VERSION, DOTNET_VERSION, TARGETARCH, PIP_NO_CACHE_DIR) and fails on any other, so a new per-release ARG is caught as well. It fails against main's Dockerfile. Dockerfile is added to the Unit Tests push paths, since the suite now reads it.
Moving the ARGs only helps if a release can read the previous release's cache, and it can't today. release.yml caches with type=gha. Actions caches are scoped to the ref that wrote them, and each release builds on its own tag: the repository's BuildKit cache entries all sit under refs/tags/v2.11.x, a full set per release, none ever read by the next one. That is also why the Actions cache is pinned at its 10 GB limit. The release build now uses a registry cache at socketdev/cli:buildcache (mode=max), which any tag can read.
I checked the mechanism with a local registry and a multi-platform push. With the registry cache, two builds with different CLI_VERSION values produced identical layer digests for a RUN step that writes a timestamp, so the step was reused rather than re-run. An inline cache (cache-to: type=inline with cache-from pointing at the published image) did not hit after a multi-platform push, which is why this uses a dedicated cache tag. The cache tag is pushed to the same Docker Hub repository and isn't meant to be pulled.
e2e-image.yml)e2e-test.yml runs reachability on the runner, so nothing checked that the image's toolchains actually work for Coana. The new workflow builds the image once from the commit (reading the release cache, never writing it) and runs --reach inside it against the existing npm fixture and a new Rust fixture (smallvec 1.6.0, which has a known advisory). Each fixture's result is recorded, so one failure doesn't hide the other. It runs when the Dockerfile, the reachability launcher, or the e2e fixtures change, on push to main, and on demand. It isn't a required check.
Both e2e workflows now share one policy for the backend's intermittent empty results:
tests/e2e/run-with-retries.sh is the retry loop: up to three attempts while a probe fails, with retries surfaced as warnings. e2e-test.yml now calls it instead of its inline loop.tests/e2e/reach_facts.py is the probe, and the one definition of the inconclusive backend result (a manifest uploaded, components orphaned, nothing to analyze). It replaces reach-facts-probe.sh, and validate-reachability.sh calls it instead of its own greps.The image check asserts only what the image controls: the run completes, and Coana reaches a reachable or unreachable verdict for at least one vulnerability, which needs the ecosystem's toolchain to work. It doesn't assert which verdict.
.github/dependabot.yml now ignores major updates to the node image. Dependabot doesn't know Node's release schedule, so it proposes odd-numbered majors, which never become LTS, and it can't check the reachability engine's minimum Node version. The floating node:24-alpine3.24 tag already picks up Node 24 minor and patch releases on every build. A Node major becomes a deliberate change, checked with e2e-image.yml.
minimal profile (rustc, cargo, rust-std) instead of default. Drops about 900 MB of offline HTML docs, plus rustfmt and clippy. Reachability only invokes cargo.npm cache clean --force after the global install, and ARG PIP_NO_CACHE_DIR=1 so no pip install leaves a cache. As an ARG it applies only at build time, and its value never changes, so it doesn't affect layer caching.apk update is gone; apk add --no-cache doesn't need it.strace is no longer installed (it was referenced nowhere). The changelog notes apk add strace for anyone who relied on it..dockerignore: lists what the build context includes (package source, build metadata, entrypoint, and dist/ for Dockerfile.preview's wheel) instead of what it excludes, so new top-level directories stay out by default.CI's dockerfile-smoke job builds this Dockerfile natively on amd64 and arm64 and runs its tool and TLS checks. The measurements below are from local linux/arm64 builds.
socketdev/cli:2.11.11 vs 3,446 MB for this branch (uncompressed, arm64). /root/.rustup went from 1,693 to 753 MB, /root/.npm from 20 to 1 MB, /app from 11 to 2 MB.CLI_VERSION=2.11.10, then rebuilt with CLI_VERSION=2.11.11. Steps 2–17, everything through the locked dependency install, were CACHED. Only the CLI install and the small layers after it rebuilt. A real release also changes pyproject.toml/uv.lock, which adds the locked-dependency layer (about 21 MB). So an upgrade is roughly 25 MB uncompressed instead of about 1 GB compressed.socketcli, cargo/rustc, go, java, dotnet, node, ruby/bundle, pyenv, uv and socket all run. cargo generate-lockfile, cargo metadata and cargo build work on a crate with a crates.io dependency.Rust reachability in the image is covered by e2e-image.yml on this PR. A throwaway run with node:26-alpine3.24 (#444, closed) also passed both fixtures, for when Node 26 is adopted.
Once the registry cache is in use, the old type=gha BuildKit entries (about 10 GB) can be deleted from the repository's Actions caches.
Bumps to 2.12.0, a minor version because the image drops binaries (strace, rustfmt, clippy) and changes how it is built and published. #438 also targets 2.12.0; whichever merges second rides that version, and needs a small merge in the changelog and in release.yml's build step.
Fixes CE-430
🤖 Generated with Claude Code
Note
Medium Risk
Changes the default Docker toolchain (minimal Rust, no strace) and layer layout that all CLI container users depend on; Rust reachability in the image was not fully re-verified in this PR.
Overview
Release 2.11.12 reshapes how the published socketdev/cli image is built so version bumps reuse cached toolchain layers instead of forcing a near-full re-download, and trims roughly a gigabyte of unused payload.
Layer caching: CLI_VERSION and the other release-specific build args move to just before the CLI install step. BuildKit invalidates every subsequent RUN when those args change; having them at the top previously busted cache for Go, Java, Rust, pyenv, locked deps, etc. on every release. A new unit test in test_dockerfile.py asserts release ARGs stay below toolchain installs, and Dockerfile is included in the CI paths filter on push.
Smaller image: Rust is installed with rustup’s minimal profile (no offline docs, rustfmt, or clippy); strace is dropped from the base packages; npm and pip caches are cleared or use --no-cache-dir; redundant apk update is removed. A new .dockerignore keeps tests, docs, and similar out of the build context while still allowing preview wheels from dist/.
Reviewed by Cursor Bugbot for commit 29a60c9. Configure here.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…slim them CLI_VERSION was declared at the top of the Dockerfile. Every RUN after an ARG is cache-keyed on its value, and CLI_VERSION changes on every release, so each release rebuilt every toolchain layer even when nothing in them changed. Between 2.11.10 and 2.11.11, 1,028 MB of the 1,049 MB compressed amd64 image was new layers, so every upgrade was close to a full pull. The release ARGs are now declared just before the CLI install, and a unit test keeps them there. The image also drops payload nothing runs: - rustup's minimal profile replaces the default one, which added offline HTML docs, rustfmt and clippy. Reachability runs only cargo. - npm and pip caches are no longer left in the image. - strace is no longer installed. - A .dockerignore keeps tests and other repo files out of the build context. dist/ stays in, since Dockerfile.preview copies the preview wheel from it.
|
bugbot run |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 29a60c9. Configure here.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…e the image in CI The release build cached layers with type=gha. Actions caches are scoped to the ref that wrote them, and each release builds on its own tag, so no release could read another's cache: every release rebuilt every layer under a new digest, and each wrote about 2.6 GB of cache nobody read. A registry cache at socketdev/cli:buildcache is readable from any tag, and reusing its layers keeps their digests stable across releases. e2e-image.yml builds the image from the commit, reusing that cache read-only, and runs --reach inside it against the npm fixture and a new Rust fixture. It follows e2e-test.yml's retry policy for the backend's intermittent empty results. Validation asserts what the image controls: the run completes and Coana reaches a reachable or unreachable verdict for at least one vulnerability, which needs the ecosystem's toolchain to work. It does not assert which verdict, since that depends on Coana and the backend.
There was an error while loading. Please reload this page.
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
[agent] Simplification pass (quality only, not a bug hunt). Five suggestions and one comment-wording note, all inline. Looked fine: the Dockerfile parser in the test, the validator's own logic, the minimal Rust profile with cargo --version, npm cache clean, dropping apk update, adding Dockerfile to the unit-test push paths, and the .dockerignore entries (none are dead).
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
| if args.probe: | ||
| return 0 if alert_states(args.fixture) else 1 |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
[agent] --probe repeats tests/e2e/reach-facts-probe.sh. This script could be the single implementation, with reach-facts-probe.sh reduced to uv run python tests/e2e/<script>.py <fixture> --probe. The script would then want a name that isn't specific to the image.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Done in ede86cf. tests/e2e/reach_facts.py (renamed, so the name isn't image-specific) is the single implementation of the probe. reach-facts-probe.sh is gone: e2e-test.yml's retry-probe and validate-reachability.sh call uv run python tests/e2e/reach_facts.py <fixture> --probe directly, and the image job runs the same script with the image's Python.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
The reachability engine sets a minimum Node version, and Dependabot doesn't know Node's release schedule, so it would propose odd-numbered majors, which never get LTS. The floating node tag already picks up minor and patch releases.
… the Dockerfile guards - tests/e2e/run-with-retries.sh is the one retry loop for the backend's intermittent empty results, used by e2e-test.yml and e2e-image.yml. - tests/e2e/reach_facts.py is the one probe and the one definition of the inconclusive backend result, replacing reach-facts-probe.sh and the image-only validator. validate-reachability.sh calls it. - e2e-image.yml builds the image once and runs every fixture against it, recording each fixture's result so one failure doesn't hide another. - The Dockerfile test lists the ARGs allowed above the toolchain layers and fails on any other, so a new per-release ARG is caught too. - PIP_NO_CACHE_DIR as a build ARG replaces the per-call --no-cache-dir flags. - .dockerignore lists what the build context includes instead of what it excludes.
…ayers # Conflicts: # CHANGELOG.md # pyproject.toml # socketsecurity/__init__.py # uv.lock
Set PIP_NO_CACHE_DIR=1 as an ARG at the top of the image build, the same way #439 does, so every pip call during the build skips the cache. This covers the CLI install retries and replaces the per-call flags.
* Disable pip's cache in the Docker image's CLI install retries The v2.11.14 release published to PyPI but failed to push the Docker image. The first pip install inside the build saw a stale PyPI index without 2.11.14, and pip cached that response, so all ten retries reused it and never saw the release. Pass --no-cache-dir to the CLI and SDK installs so each retry fetches the index fresh. The release workflow's host-side verify step already does this. Bump to 2.11.15. * Disable pip's cache through a build-time ARG Set PIP_NO_CACHE_DIR=1 as an ARG at the top of the image build, the same way #439 does, so every pip call during the build skips the cache. This covers the CLI install retries and replaces the per-call flags.
965638cc
Deployed
At least 1 approving review is required to merge this pull request.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.