GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Keep the Docker image's toolchain layers stable across releases, and slim them by lelia · Pull Request #439 · SocketDev/socket-python-cli

Conversation

@lelia

@lelia lelia commented Nov 4, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Every release currently rebuilds every layer of the Docker image, so upgrading socketdev/cli downloads nearly the whole image. This PR fixes that and removes about 1.1 GB of payload nothing runs.

Root cause of the per-release re-pull

ARG CLI_VERSION (with the other release ARGs) was declared at the top of the Dockerfile. BuildKit cache-keys every RUN after an ARG on that ARG's value, and CLI_VERSION changes on every release, so every toolchain layer missed the cache on every release build even when nothing in it changed. From the published amd64 manifests:

Upgrade Layers reused New compressed bytes
2.11.5 → 2.11.6 8 of 27 1,025 MB of 1,046
2.11.10 → 2.11.11 8 of 27 1,028 MB of 1,049

The base image digest was the same across 2.11.10 → 2.11.11, so the churn didn't come from upstream. The release ARGs are now declared just before the CLI install. tests/unit/test_dockerfile.py lists the ARGs allowed above the toolchain layers (GO_VERSION, JAVA_VERSION, DOTNET_VERSION, TARGETARCH, PIP_NO_CACHE_DIR) and fails on any other, so a new per-release ARG is caught as well. It fails against main's Dockerfile. Dockerfile is added to the Unit Tests push paths, since the suite now reads it.

Release build cache

Moving the ARGs only helps if a release can read the previous release's cache, and it can't today. release.yml caches with type=gha. Actions caches are scoped to the ref that wrote them, and each release builds on its own tag: the repository's BuildKit cache entries all sit under refs/tags/v2.11.x, a full set per release, none ever read by the next one. That is also why the Actions cache is pinned at its 10 GB limit. The release build now uses a registry cache at socketdev/cli:buildcache (mode=max), which any tag can read.

I checked the mechanism with a local registry and a multi-platform push. With the registry cache, two builds with different CLI_VERSION values produced identical layer digests for a RUN step that writes a timestamp, so the step was reused rather than re-run. An inline cache (cache-to: type=inline with cache-from pointing at the published image) did not hit after a multi-platform push, which is why this uses a dedicated cache tag. The cache tag is pushed to the same Docker Hub repository and isn't meant to be pulled.

Reachability inside the image (e2e-image.yml)

e2e-test.yml runs reachability on the runner, so nothing checked that the image's toolchains actually work for Coana. The new workflow builds the image once from the commit (reading the release cache, never writing it) and runs --reach inside it against the existing npm fixture and a new Rust fixture (smallvec 1.6.0, which has a known advisory). Each fixture's result is recorded, so one failure doesn't hide the other. It runs when the Dockerfile, the reachability launcher, or the e2e fixtures change, on push to main, and on demand. It isn't a required check.

Both e2e workflows now share one policy for the backend's intermittent empty results:

  • tests/e2e/run-with-retries.sh is the retry loop: up to three attempts while a probe fails, with retries surfaced as warnings. e2e-test.yml now calls it instead of its inline loop.
  • tests/e2e/reach_facts.py is the probe, and the one definition of the inconclusive backend result (a manifest uploaded, components orphaned, nothing to analyze). It replaces reach-facts-probe.sh, and validate-reachability.sh calls it instead of its own greps.

The image check asserts only what the image controls: the run completes, and Coana reaches a reachable or unreachable verdict for at least one vulnerability, which needs the ecosystem's toolchain to work. It doesn't assert which verdict.

Dependabot and Node majors

.github/dependabot.yml now ignores major updates to the node image. Dependabot doesn't know Node's release schedule, so it proposes odd-numbered majors, which never become LTS, and it can't check the reachability engine's minimum Node version. The floating node:24-alpine3.24 tag already picks up Node 24 minor and patch releases on every build. A Node major becomes a deliberate change, checked with e2e-image.yml.

Payload removed

  • Rust: rustup's minimal profile (rustc, cargo, rust-std) instead of default. Drops about 900 MB of offline HTML docs, plus rustfmt and clippy. Reachability only invokes cargo.
  • Caches: npm cache clean --force after the global install, and ARG PIP_NO_CACHE_DIR=1 so no pip install leaves a cache. As an ARG it applies only at build time, and its value never changes, so it doesn't affect layer caching.
  • apk update is gone; apk add --no-cache doesn't need it.
  • strace is no longer installed (it was referenced nowhere). The changelog notes apk add strace for anyone who relied on it.
  • .dockerignore: lists what the build context includes (package source, build metadata, entrypoint, and dist/ for Dockerfile.preview's wheel) instead of what it excludes, so new top-level directories stay out by default.

Verification

CI's dockerfile-smoke job builds this Dockerfile natively on amd64 and arm64 and runs its tool and TLS checks. The measurements below are from local linux/arm64 builds.

  • Size: 4,579 MB for socketdev/cli:2.11.11 vs 3,446 MB for this branch (uncompressed, arm64). /root/.rustup went from 1,693 to 753 MB, /root/.npm from 20 to 1 MB, /app from 11 to 2 MB.
  • Layer reuse: built with CLI_VERSION=2.11.10, then rebuilt with CLI_VERSION=2.11.11. Steps 2–17, everything through the locked dependency install, were CACHED. Only the CLI install and the small layers after it rebuilt. A real release also changes pyproject.toml/uv.lock, which adds the locked-dependency layer (about 21 MB). So an upgrade is roughly 25 MB uncompressed instead of about 1 GB compressed.
  • Toolchains: socketcli, cargo/rustc, go, java, dotnet, node, ruby/bundle, pyenv, uv and socket all run. cargo generate-lockfile, cargo metadata and cargo build work on a crate with a crates.io dependency.
  • Tests: 734 unit tests pass.

Rust reachability in the image is covered by e2e-image.yml on this PR. A throwaway run with node:26-alpine3.24 (#444, closed) also passed both fixtures, for when Node 26 is adopted.

Once the registry cache is in use, the old type=gha BuildKit entries (about 10 GB) can be deleted from the repository's Actions caches.

Bumps to 2.12.0, a minor version because the image drops binaries (strace, rustfmt, clippy) and changes how it is built and published. #438 also targets 2.12.0; whichever merges second rides that version, and needs a small merge in the changelog and in release.yml's build step.

Fixes CE-430

🤖 Generated with Claude Code


Note

Medium Risk
Changes the default Docker toolchain (minimal Rust, no strace) and layer layout that all CLI container users depend on; Rust reachability in the image was not fully re-verified in this PR.

Overview
Release 2.11.12 reshapes how the published socketdev/cli image is built so version bumps reuse cached toolchain layers instead of forcing a near-full re-download, and trims roughly a gigabyte of unused payload.

Layer caching: CLI_VERSION and the other release-specific build args move to just before the CLI install step. BuildKit invalidates every subsequent RUN when those args change; having them at the top previously busted cache for Go, Java, Rust, pyenv, locked deps, etc. on every release. A new unit test in test_dockerfile.py asserts release ARGs stay below toolchain installs, and Dockerfile is included in the CI paths filter on push.

Smaller image: Rust is installed with rustup’s minimal profile (no offline docs, rustfmt, or clippy); strace is dropped from the base packages; npm and pip caches are cleared or use --no-cache-dir; redundant apk update is removed. A new .dockerignore keeps tests, docs, and similar out of the build context while still allowing preview wheels from dist/.

Reviewed by Cursor Bugbot for commit 29a60c9. Configure here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@lelia
Keep the Docker image's toolchain layers stable across releases, and …
29a60c9
…slim them

CLI_VERSION was declared at the top of the Dockerfile. Every RUN after an ARG
is cache-keyed on its value, and CLI_VERSION changes on every release, so each
release rebuilt every toolchain layer even when nothing in them changed. Between
2.11.10 and 2.11.11, 1,028 MB of the 1,049 MB compressed amd64 image was new
layers, so every upgrade was close to a full pull. The release ARGs are now
declared just before the CLI install, and a unit test keeps them there.

The image also drops payload nothing runs:
- rustup's minimal profile replaces the default one, which added offline HTML
  docs, rustfmt and clippy. Reachability runs only cargo.
- npm and pip caches are no longer left in the image.
- strace is no longer installed.
- A .dockerignore keeps tests and other repo files out of the build context.
  dist/ stays in, since Dockerfile.preview copies the preview wheel from it.
@lelia
lelia requested a review from a team as a code owner November 4, 2026 18:25
@lelia
lelia temporarily deployed to socket-firewall November 4, 2026 18:25 — with GitHub Actions Inactive
This was referenced Nov 4, 2026
Attach an SBOM to the Docker image, retag stable, and document the image #438
Open
Install the pinned reachability engine in the Docker image #345
Closed
@lelia

lelia commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

bugbot run

cursor[bot] reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

cursor[bot]
cursor Bot reviewed Nov 4, 2026
View reviewed changes

@cursor cursor Bot left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 29a60c9. Configure here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@lelia
Point the Dockerfile test at the Dockerfile's own explanation
0cf93cd
@lelia
lelia temporarily deployed to socket-firewall November 6, 2026 05:18 — with GitHub Actions Inactive
@lelia
Share the release build cache across tags, and run reachability insid…
8dc4c6b
…e the image in CI

The release build cached layers with type=gha. Actions caches are scoped
to the ref that wrote them, and each release builds on its own tag, so no
release could read another's cache: every release rebuilt every layer
under a new digest, and each wrote about 2.6 GB of cache nobody read. A
registry cache at socketdev/cli:buildcache is readable from any tag, and
reusing its layers keeps their digests stable across releases.

e2e-image.yml builds the image from the commit, reusing that cache
read-only, and runs --reach inside it against the npm fixture and a new
Rust fixture. It follows e2e-test.yml's retry policy for the backend's
intermittent empty results. Validation asserts what the image controls:
the run completes and Coana reaches a reachable or unreachable verdict
for at least one vulnerability, which needs the ecosystem's toolchain to
work. It does not assert which verdict, since that depends on Coana and
the backend.
@lelia
lelia temporarily deployed to socket-firewall November 6, 2026 05:31 — with GitHub Actions Inactive
@socket-security

socket-security Bot commented Nov 6, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedgithub/​docker/​setup-buildx-action@​f87e5991a6d7451dcb8d9637bfbc97413f497069100100100100100

View full report

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

BarrensZeppelin
Oskar Haarklou Veileborg (BarrensZeppelin) reviewed Nov 6, 2026
View reviewed changes

@BarrensZeppelin Oskar Haarklou Veileborg (BarrensZeppelin) left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[agent] Simplification pass (quality only, not a bug hunt). Five suggestions and one comment-wording note, all inline. Looked fine: the Dockerfile parser in the test, the validator's own logic, the minimal Rust profile with cargo --version, npm cache clean, dropping apk update, adding Dockerfile to the unit-test push paths, and the .dockerignore entries (none are dead).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread .github/workflows/e2e-image.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread .github/workflows/e2e-image.yml Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread tests/e2e/validate-image-reachability.py Outdated
Comment on lines +82 to +83
if args.probe:
return 0 if alert_states(args.fixture) else 1

@BarrensZeppelin Oskar Haarklou Veileborg (BarrensZeppelin) Nov 6, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[agent] --probe repeats tests/e2e/reach-facts-probe.sh. This script could be the single implementation, with reach-facts-probe.sh reduced to uv run python tests/e2e/<script>.py <fixture> --probe. The script would then want a name that isn't specific to the image.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@lelia lelia Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ede86cf. tests/e2e/reach_facts.py (renamed, so the name isn't image-specific) is the single implementation of the probe. reach-facts-probe.sh is gone: e2e-test.yml's retry-probe and validate-reachability.sh call uv run python tests/e2e/reach_facts.py <fixture> --probe directly, and the image job runs the same script with the image's Python.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread tests/e2e/validate-image-reachability.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Dockerfile Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread tests/unit/test_dockerfile.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread .dockerignore Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Dockerfile Resolved

Uh oh!

There was an error while loading. Please reload this page.

lelia added 4 commits November 6, 2026 17:07
@lelia
Leave Node major versions to a deliberate bump, not Dependabot
d379f7e
The reachability engine sets a minimum Node version, and Dependabot doesn't
know Node's release schedule, so it would propose odd-numbered majors, which
never get LTS. The floating node tag already picks up minor and patch releases.
@lelia
Share the e2e retry policy, build the image once per run, and tighten…
ede86cf
… the Dockerfile guards

- tests/e2e/run-with-retries.sh is the one retry loop for the backend's
  intermittent empty results, used by e2e-test.yml and e2e-image.yml.
- tests/e2e/reach_facts.py is the one probe and the one definition of the
  inconclusive backend result, replacing reach-facts-probe.sh and the
  image-only validator. validate-reachability.sh calls it.
- e2e-image.yml builds the image once and runs every fixture against it,
  recording each fixture's result so one failure doesn't hide another.
- The Dockerfile test lists the ARGs allowed above the toolchain layers and
  fails on any other, so a new per-release ARG is caught too.
- PIP_NO_CACHE_DIR as a build ARG replaces the per-call --no-cache-dir flags.
- .dockerignore lists what the build context includes instead of what it
  excludes.
@lelia
Merge remote-tracking branch 'origin/main' into lelia/ce-430-docker-l…
4e9e170
…ayers

# Conflicts:
#	CHANGELOG.md
#	pyproject.toml
#	socketsecurity/__init__.py
#	uv.lock
@lelia
Merge main and move the Docker image changes to 2.12.0
965638c
@lelia
lelia deployed to socket-firewall November 6, 2026 23:18 — with GitHub Actions Active
BarrensZeppelin
Oskar Haarklou Veileborg (BarrensZeppelin) approved these changes Nov 9, 2026
View reviewed changes
Martin Torp (mtorp) added a commit that referenced this pull request Nov 12, 2026
@mtorp
Disable pip's cache through a build-time ARG
5780789
Set PIP_NO_CACHE_DIR=1 as an ARG at the top of the image build, the same
way #439 does, so every pip call during the build skips the cache. This
covers the CLI install retries and replaces the per-call flags.
@mtorp Martin Torp (mtorp) mentioned this pull request Nov 12, 2026
Disable pip's cache during the Docker image build #446
Merged
Martin Torp (mtorp) added a commit that referenced this pull request Nov 12, 2026
@mtorp
Disable pip's cache during the Docker image build (#446)
e35fc59
* Disable pip's cache in the Docker image's CLI install retries

The v2.11.14 release published to PyPI but failed to push the Docker
image. The first pip install inside the build saw a stale PyPI index
without 2.11.14, and pip cached that response, so all ten retries
reused it and never saw the release.

Pass --no-cache-dir to the CLI and SDK installs so each retry fetches
the index fresh. The release workflow's host-side verify step already
does this. Bump to 2.11.15.

* Disable pip's cache through a build-time ARG

Set PIP_NO_CACHE_DIR=1 as an ARG at the top of the image build, the same
way #439 does, so every pip call during the build skips the cache. This
covers the CLI install retries and replaces the per-call flags.

This branch was successfully deployed

1 active deployment
socket-firewall — 965638cc Deployed Nov 6, 2026 by lelia via python-sfw-smoke-enterprise #571
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cursor cursor[bot] cursor[bot] left review comments
@BarrensZeppelin Oskar Haarklou Veileborg (BarrensZeppelin) BarrensZeppelin approved these changes

At least 1 approving review is required to merge this pull request.

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@lelia @BarrensZeppelin
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.