GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GitHub - fenollp/supergreen: A way for cargo to build crates locally or remotely and share and fine-grained--cache them via a Docker registry.

BranchesTags
Open more actions menu

Latest commit

 

History

1,098 Commits
1,098 Commits

Folders and files

NameName
Last commit message
Last commit date
.github
.github
 
 
cargo-green
cargo-green
 
 
hack
hack
 
 
recipes
recipes
 
 
.gitignore
.gitignore
 
 
Cargo.lock
Cargo.lock
 
 
Cargo.toml
Cargo.toml
 
 
LICENSE.md
LICENSE.md
 
 
README.md
README.md
 
 
TODO.md
TODO.md
 
 
docker-bake.json
docker-bake.json
 
 
green.toml
green.toml
 
 
release-plz.toml
release-plz.toml
 
 
rustfmt.toml
rustfmt.toml
 
 
taplo.toml
taplo.toml
 
 

Repository files navigation

supergreen ~ Faster Rust builds!

cargo-green: a cached & remote-ready Rust projects builder.

cargo-green is

  • a cargo plugin that sets a $RUSTC_WRAPPER then calls cargo.
  • a RUSTC_WRAPPER that builds Dockerfiles
  • by forwarding rustc calls to BuildKit builders

A rusty crab character named Ferris, featuring a unique hairstyle resembling 'Ruby Road', a vibrant and textured hairdo often seen in flamboyant red

  • Installation
  • Usage
  • How it works
  • Remote execution
  • Caching
  • Configuration
    • $CARGOGREEN_LOG_PATH
    • $CARGOGREEN_LOG
    • $CARGOGREEN_LOG_STYLE
    • $CARGOGREEN_RUNNER
    • $BUILDX_BUILDER
    • $CARGOGREEN_BUILDER_IMAGE
    • $CARGOGREEN_BUILDKIT_TLSDIR
    • $CARGOGREEN_SYNTAX_IMAGE
    • $CARGOGREEN_REGISTRY_MIRRORS
    • $CARGOGREEN_CACHE_IMAGES
    • $CARGOGREEN_CACHE_FROM_IMAGES
    • $CARGOGREEN_CACHE_TO_IMAGES
    • $CARGOGREEN_FINAL_PATH
    • $CARGOGREEN_BASE_IMAGE
    • $CARGOGREEN_SET_ENVS
    • $CARGOGREEN_WITH_NETWORK
    • $CARGOGREEN_COMPONENTS
    • $CARGOGREEN_ADD_APT
    • $CARGOGREEN_ADD_APK
    • $CARGOGREEN_EXPERIMENT
  • Alternatives
  • Origins

Installation

cargo install cargo-green
cargo install --locked --force --git https://github.com/fenollp/supergreen.git cargo-green

# Make sure $CARGO_HOME/bin is in your $PATH
which cargo-green

When building locally, ensure either a docker or podman client is installed.

Minimum requirements:

  • Ubuntu 22.04
  • buildkit 0.12.0
  • github.com/docker/buildx v0.11.2
  • rust 1.73

Usage

# Usage:
  cargo green supergreen setup                                   Create required symlinks
  cargo green supergreen env [ENV ...]                           Show used values
  cargo green supergreen doc [ENV ...]                           Documentation of said values
  cargo green supergreen show-rust-base                          Show base stage in use
  cargo green fetch                                              Pulls images and crates
  cargo green supergreen sync                                    Pulls everything, for offline usage
  cargo green supergreen push                                    Push cache image (all tags)
  cargo green supergreen builder [ { recreate | rm } --clean ]   Manage local/remote builder
  [BUILDX_BUILDER=NAME] cargo green supergreen builder create    Create a builder
  cargo green supergreen doctor                                  Diagnose setup, runners, dirs and builder
  cargo green supergreen -h | --help
  cargo green supergreen -V | --version
  cargo green ...any cargo subcommand...

# Try:
  cargo clean # Start from a clean slate
  cargo green build
  cargo green supergreen env CARGOGREEN_BASE_IMAGE 2>/dev/null
  cargo green supergreen help
  { cargo green supergreen setup 2>/dev/null || true; } | sudo /bin/sh -xe
  cargo green supergreen doctor

# Suggestion:
  alias cargo='cargo green'
  # Now try, within your project:
  cargo fetch
  cargo test

How it works

This encapsulates the compilation of Rust programs using Docker's build engine. In other words: this cargo plugin registers a rustc wrapper that calls to BuildKit to build crates and the execution of their build scripts.

These builds use OCI images pinned to the local Rust installation version, eventually producing a single (huge) Containerfile that can then be shared around and built with eg. docker build -o=. https://host.my/tiny/binary.Dockerfile.

Builds reproducibility or hermeticity is guaranteed via:

  • Every image is locked by its digest (@sha256:..)
  • Target directory paths are renamed /target/..
  • crates.io sources paths are renamed to $CARGO_HOME/registry/src/index.crates.io/..
    • additionally, this index.crates.io path is created locally.
  • $CARGO_HOME has to get linked to the one used inside the image (/usr/local/cargo): sudo ln -s ~/.cargo /usr/local/cargo
  • git dependencies are pinned to their commit hash
  • Produced files timestamps' are rewritten to some fixed epoch (SOURCE_DATE_EPOCH)

Switching from cargo build to cargo green build requires a cargo clean because of the additional work cargo-green does. Running cargo build after cargo green build works normally though: the same files are produced.

Remote execution

Say you have a bigger machine in your ~/.ssh/config called extra-oomph:

DOCKER_HOST=ssh://extra-oomph cargo green test

This will compile the tests on the remote machine and run then locally. TODO: also run tests remotely, like cross does: https://github.com/cross-rs/cross/blob/49cd054de9b832dfc11a4895c72b0aef533b5c6a/README.md#supported-targets

For more knobs to tune, see also:

  • $BUILDX_BUILDER
  • $CARGOGREEN_BUILDER_IMAGE

Caching

Share your build cache with your team and CI, never feel cold starts!

[package.metadata.green]
cache-images = [ "docker-image://my.org/team/my-project", "docker-image://ghcr.io/me/fork" ]
cache-from-images = [ "docker-image://some.org/global/cache" ]

Configuration

Tune the behavior of cargo-green either through environment variables or via the package's green metadata section.

[package]
name = "my-crate"
# ...

[package.metadata.green]
registry-mirrors = [ "mirror.gcr.io", "public.ecr.aws/docker" ] # Default values

Environment variables that are prefixed with $CARGOGREEN_ override TOML settings.

export CARGOGREEN_REGISTRY_MIRRORS=mirror.gcr.io
cargo green build

$CARGOGREEN_LOG_PATH

Path to a text file to write logs.

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_LOG_PATH="/tmp/my-logs.txt"
# This needs to be set: nothing is logged by default
export CARGOGREEN_LOG="info"

$CARGOGREEN_LOG

Filter logs. Equivalent to $RUST_LOG (and doesn't conflict with cargo's).

By default, writes logs under /tmp.

See https://docs.rs/env_logger/#enabling-logging

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_LOG="trace,cargo_green::build=info"

$CARGOGREEN_LOG_STYLE

Style logs. Equivalent to $RUST_LOG_STYLE.

See https://docs.rs/env_logger/#disabling-colors

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_LOG_STYLE="never"

$CARGOGREEN_RUNNER

Pick which executor to use: "docker" (default), "podman" or "none".

The runner gets forwarded these environment variables:

  • $BUILDKIT_COLORS
  • $BUILDKIT_HOST
  • $BUILDKIT_PROGRESS
  • $BUILDKIT_TTY_LOG_LINES
  • $BUILDX_BAKE_GIT_AUTH_HEADER
  • $BUILDX_BAKE_GIT_AUTH_TOKEN
  • $BUILDX_BAKE_GIT_SSH
  • $BUILDX_BUILDER
  • $DOCKER_BUILDKIT
  • $BUILDX_CONFIG
  • $BUILDX_CPU_PROFILE
  • $BUILDX_EXPERIMENTAL
  • $BUILDX_GIT_CHECK_DIRTY
  • $BUILDX_GIT_INFO
  • $BUILDX_GIT_LABELS
  • $BUILDX_MEM_PROFILE
  • $BUILDX_METADATA_PROVENANCE
  • $BUILDX_METADATA_WARNINGS
  • $BUILDX_NO_DEFAULT_ATTESTATIONS
  • $BUILDX_NO_DEFAULT_LOAD
  • $DOCKER_API_VERSION
  • $DOCKER_CERT_PATH
  • $DOCKER_CONFIG
  • $DOCKER_CONTENT_TRUST
  • $DOCKER_CONTENT_TRUST_SERVER
  • $DOCKER_CONTEXT
  • $DOCKER_DEFAULT_PLATFORM
  • $DOCKER_HIDE_LEGACY_COMMANDS
  • $DOCKER_HOST
  • $DOCKER_TLS
  • $DOCKER_TLS_VERIFY
  • $EXPERIMENTAL_BUILDKIT_SOURCE_POLICY
  • $HTTP_PROXY
  • $HTTPS_PROXY
  • $NO_PROXY

When runner is set to none the above runner-specific environment variables are ineffective and they are ignored. Except for those picking where builds run, which then make for an error rather than a silently local build: $BUILDKIT_HOST, $BUILDX_BUILDER, $CONTAINER_CONNECTION, $CONTAINER_HOST, $DOCKER_CONTEXT and $DOCKER_HOST.

With runner none, rustc and build scripts run directly on the host, and the results cache is only read: what previous runs with a runner stored gets reused, nothing new is stored.

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_RUNNER="docker"

$BUILDX_BUILDER

Sets which BuildKit builder to use, through $BUILDX_BUILDER.

See https://docs.docker.com/build/building/variables/#buildx_builder

A builder is what actually runs BuildKit, and so what runs each crate's compilation. cargo green either manages one for you, or uses the one you pick:

$BUILDX_BUILDER Builder used Managed by cargo green
unset, or "supergreen" the managed builder (named after where it runs, see below) yes
"" (empty) none picked: docker build's, ie. the daemon's own default builder (or, after docker buildx install, buildx's current one, see docker buildx use) no
any other name that existing builder, as is (an error if it does not exist) no

The managed builder

cargo green takes care of this builder so that you don't have to:

  • it is created when missing, with the docker-container driver running $CARGOGREEN_BUILDER_IMAGE (which defaults to the latest moby/buildkit image)
  • it is re-created when it does not run $CARGOGREEN_BUILDER_IMAGE, when it runs a BuildKit older than the latest release cargo green knows of, or when it points at another endpoint than its daemon's (e.g. not at the connection pool, see below). Re-creating first tries to keep the builder's state (its build cache), with docker buildx rm --keep-state.

Where it runs depends on which daemon is configured, checked in this order:

  1. $BUILDKIT_HOST (e.g. ssh://host, tcp://host:1234, unix:///run/buildkit/buildkitd.sock): builds go straight to that BuildKit daemon, through a remote driver builder named "supergreen-buildkit-<address>". No container gets created there, so $CARGOGREEN_BUILDER_IMAGE cannot apply, and re-creating that builder does not upgrade the daemon.
  2. $DOCKER_HOST (e.g. ssh://host, tcp://host:2376, unix:///run/user/1000/docker.sock): the builder's container runs on that docker daemon, and is named after its address (e.g. "supergreen-<host>"). The default unix:///var/run/docker.sock counts as local (the builder is then just "supergreen").
  3. A docker context, from $DOCKER_CONTEXT or docker context use, whose daemon is on an ssh:// or tcp:// host: this is the same as setting $DOCKER_HOST to that address (along with the context's TLS settings).
  4. Otherwise: the builder runs on the local docker daemon and is named "supergreen".

Each daemon thus gets a builder of its own, and switching between daemons does not re-create builders back and forth.

Remote daemons and the connection pool

Builds talking to an ssh:// or tcp:// daemon open many connections: at least one per crate. With the docker runner, cargo green pools these so the whole build only pays for a few handshakes:

  • ssh:// uses your ssh command and so your ~/.ssh/config (aliases, ProxyJump, IdentityFile, ...). Use an ssh-agent (or a key without a passphrase): a few connections get opened, and each would prompt otherwise.
  • $DOCKER_HOST=tcp://… reads $DOCKER_TLS_VERIFY and $DOCKER_CERT_PATH the way docker does ($DOCKER_TLS alone, ie. without verification, is unsupported). See protecting the daemon socket with TLS. $BUILDKIT_HOST=tcp://… reads $CARGOGREEN_BUILDKIT_TLSDIR.
  • The pool is a local socket that only exists while cargo green runs. The managed builder points at it, so outside of cargo green, docker buildx ls shows that builder as erroring: this is expected.
  • Without a remote daemon (no $DOCKER_HOST, $BUILDKIT_HOST nor remote docker context), there is no pool: builds are local.

Remote podman ($CONTAINER_HOST, $CONTAINER_CONNECTION) is not supported yet: with the podman runner, builds run locally.

Picking your own builder

Set $BUILDX_BUILDER to the name of a builder you created to have builds run there. cargo green then never creates, re-creates nor upgrades it:

  • when it does not run $CARGOGREEN_BUILDER_IMAGE, that is an error
  • when its BuildKit is outdated, a warning shows how to re-create it yourself

BUILDX_BUILDER=mine cargo green supergreen builder create creates a builder set up like the managed one, with a data dir (so a build cache) of its own. Without $BUILDX_BUILDER, this creates the managed builder.

For instance, to build on a remote docker daemon with a builder of your own:

# once: create the builder on that daemon, and start it
docker buildx create --name mine --driver docker-container --bootstrap ssh://me@host

# then, each time:
DOCKER_HOST=ssh://me@host BUILDX_BUILDER=mine cargo green build
  • Without --driver, docker buildx create … ssh://host makes a remote driver builder: it expects a bare BuildKit daemon on host, and builds then fail with context deadline exceeded when there is none.
  • Without --bootstrap, a builder only gets its container (and its BuildKit version) once started: until then, cargo green cannot inspect it and its BuildKit shows as outdated. docker buildx inspect --bootstrap mine starts it.
  • Nothing gets (re-)created on the remote daemon for you, as noted above.

Builds with your builder go through the connection pool (see above) when all of these hold:

  • $DOCKER_HOST (or a docker context) points at an ssh:// or tcp:// daemon, with the docker runner
  • your builder was created on that same daemon: the address it was created with (or the docker context it was created on) names the same user, host, port and socket path. These are compared by name, not resolved: ssh://host and ssh://me@host are told apart, and so are tcp://localhost:2375 and tcp://127.0.0.1:2375, even when both reach the same daemon.
  • it uses the docker-container (or docker) driver

Otherwise your builder still works, but without pooling: each build opens its own connections. A warning says so when your builder is not on $DOCKER_HOST's daemon, which includes a builder created with another name for that daemon. Better point $DOCKER_HOST at your builder's daemon anyway: cargo green also uses it to inspect your builder.

How this works: buildx dials a builder at the address recorded when it was created, ignoring $DOCKER_HOST. So cargo green gives docker calls a private $BUILDX_CONFIG: a copy of buildx's state directory ($BUILDX_CONFIG, else $DOCKER_CONFIG/buildx) made of links to the real one, except for builders on the pooled daemon, whose copies point at the pool's socket. Your builders' definitions and their containers stay untouched (the pool reaches the same daemon), and that copy is removed when cargo green exits.

With $BUILDX_BUILDER="", the same applies to buildx's current builder, when docker buildx install made docker build use it.

$BUILDKIT_HOST only ever selects the managed builder: buildx does not read $BUILDKIT_HOST, so setting it along with a non-empty $BUILDX_BUILDER (even "supergreen") is an error.

Other unsupported combinations

These are errors:

  • $CARGOGREEN_BUILDER_IMAGE with $BUILDX_BUILDER="" (no builder to run that image)
  • both $BUILDKIT_HOST and $DOCKER_HOST (a remote builder does not use $DOCKER_HOST)

See also

  • $DOCKER_HOST, $DOCKER_CONTEXT, $DOCKER_CONFIG, $DOCKER_TLS, $DOCKER_TLS_VERIFY, $DOCKER_CERT_PATH: https://docs.docker.com/reference/cli/docker/#environment-variables
  • Docker over TLS ($DOCKER_TLS_VERIFY, $DOCKER_CERT_PATH): https://docs.docker.com/engine/security/protect-access/#use-tls-https-to-protect-the-docker-daemon-socket
  • Docker contexts: https://docs.docker.com/engine/manage-resources/contexts/
  • $BUILDKIT_HOST: https://docs.docker.com/build/building/variables/#buildkit_host
  • $CONTAINER_HOST, $CONTAINER_CONNECTION: https://docs.podman.io/en/latest/markdown/podman.1.html#environment-variables

$CARGOGREEN_BUILDER_IMAGE

Sets which BuildKit builder version to use.

See https://docs.docker.com/build/builders/

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_BUILDER_IMAGE="docker-image://docker.io/moby/buildkit:latest"

$CARGOGREEN_BUILDKIT_TLSDIR

TLS settings to reach a $BUILDKIT_HOST=tcp://… daemon: a directory with ca.pem (and maybe cert.pem & key.pem), or ca.crt (and maybe tls.crt & tls.key). Read like buildctl --tlsdir.

The daemon's name to verify is the host in $BUILDKIT_HOST. Setting this without $BUILDKIT_HOST=tcp://… is an error.

See https://docs.docker.com/build/builders/drivers/remote/

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_BUILDKIT_TLSDIR="$HOME/.config/buildkit/certs"

$CARGOGREEN_SYNTAX_IMAGE

Sets which BuildKit frontend syntax to use.

See https://docs.docker.com/build/buildkit/frontend/#stable-channel

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_SYNTAX_IMAGE="docker-image://docker.io/docker/dockerfile:1"

$CARGOGREEN_REGISTRY_MIRRORS

Registry mirrors for Docker Hub (docker.io). Defaults to GCP & AWS mirrors.

See https://docs.docker.com/build/buildkit/configure/#registry-mirror

Namely hosts with maybe a port and a path:

  • dockerhub.timeweb.cloud
  • dockerhub1.beget.com
  • localhost:5000
  • mirror.gcr.io
  • public.ecr.aws/docker
registry-mirrors = [ "mirror.gcr.io", "public.ecr.aws/docker" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_REGISTRY_MIRRORS="mirror.gcr.io,public.ecr.aws/docker"

$CARGOGREEN_CACHE_IMAGES

Both read and write cached data to and from image registries

Exactly a combination of [Cache::from_images] and [Cache::to_images].

See

  • type=registry at https://docs.docker.com/build/cache/backends/
  • and https://docs.docker.com/build/cache/backends/registry/
cache-images = [ "docker-image://my.org/team/my-project", "docker-image://some.org/global/cache" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_CACHE_IMAGES="docker-image://my.org/team/my-project,docker-image://some.org/global/cache"

$CARGOGREEN_CACHE_FROM_IMAGES

Read cached data from image registries

See also [Cache::images] and [Cache::to_images].

cache-from-images = [ "docker-image://my.org/team/my-project-in-ci", "docker-image://some.org/global/cache" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_CACHE_FROM_IMAGES="docker-image://my.org/team/my-project-in-ci,docker-image://some.org/global/cache"

$CARGOGREEN_CACHE_TO_IMAGES

Write cached data to image registries

Note that errors caused by failed cache exports are not ignored.

See also [Cache::images] and [Cache::from_images].

cache-to-images = [ "docker-image://my.org/team/my-fork" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_CACHE_TO_IMAGES="docker-image://my.org/team/my-fork"

$CARGOGREEN_FINAL_PATH

Write final containerfile to given path. Its directory must exist and the path be writable: this is checked on startup.

Helps e.g. create a containerfile of e.g. a binary to use for best caching of dependencies.

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_FINAL_PATH="$PWD/[email protected]"

$CARGOGREEN_BASE_IMAGE

Sets the base as an image URL, with scheme: docker-image://.

On top of this, rustup installs the toolchain. This toolchain is picked by your local Rust installation or through cargo +toolchain ...

If needing additional envs to be passed to rustc or build script, set them in the base image.

See also:

  • components
  • also-run
  • additional-build-arguments
base-image = "docker-image://docker.io/library/debian:trixie-slim"

For remote builds, make sure this image is accessible non-locally.

export CARGOGREEN_BASE_IMAGE=docker-image://my_ubuntu_with_libs_and_envs
DOCKER_HOST=ssh://my-remote-builder docker buildx build -t my_ubuntu_with_libs_and_envs - <<EOF
FROM ubuntu:latest@sha256:c4a8d5503dfb2a3eb8ab5f807da5bc69a85730fb49b5cfca2330194ebcc41c7b
RUN set -eux && apt update && apt install -y libpq-dev libssl3
ENV KEY=value
EOF

This environment variable takes precedence over any Cargo.toml settings:

export CARGOGREEN_BASE_IMAGE="docker-image://docker.io/library/debian:trixie-slim"

$CARGOGREEN_SET_ENVS

Pass environment variables through to build runner.

May be useful if a build script exported some vars that a package then reads. See also:

  • packages

See about $GIT_AUTH_TOKEN: https://docs.docker.com/build/building/secrets/#git-authentication-for-remote-contexts

NOTE: this doesn't (yet) accumulate dependencies' set-envs values! Meaning only the top-level crate's setting is used, for all crates/dependencies.

WARNING: values are written as-is into the generated Containerfiles, so they end up wherever these go: $CARGOGREEN_FINAL_PATH, BuildKit's build history & cache, and any cache image pushed to a registry. They are also part of cache keys: a changing value means rebuilds. Do not pass secrets this way. (Logs only show the first few characters of values not known to be harmless.)

set-envs = [ "GIT_AUTH_TOKEN", "TYPENUM_BUILD_CONSTS", "TYPENUM_BUILD_OP" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_SET_ENVS="GIT_AUTH_TOKEN,TYPENUM_BUILD_CONSTS,TYPENUM_BUILD_OP"

$CARGOGREEN_WITH_NETWORK

Controls runner's --network none (default) | default | host setting.

Set this to "default" if e.g. your additional stages to the base image call curl or wget or install any packages.

If adding system packages via add, this gets automatically set to "default".

It turns out --network is part of BuildKit's cache key, so an initial online build won't cache-hit on later offline builds.

Set to none when in $CARGO_NET_OFFLINE mode. See

  • https://doc.rust-lang.org/cargo/reference/config.html#netoffline
  • rust-lang/rustup#4289
[package.metadata.green]
with-network = "none"
base-image = "docker-image://docker.io/library/ubuntu@sha256:c4a8d5503dfb2a3eb8ab5f807da5bc69a85730fb49b5cfca2330194ebcc41c7b"

This environment variable takes precedence over any Cargo.toml settings:

export CARGOGREEN_WITH_NETWORK="none"

$CARGOGREEN_COMPONENTS

Tells rustup to add the given components to the base image.

See https://rust-lang.github.io/rustup/concepts/components.html

[package.metadata.green]
components = [ "rust-src", "llvm-tools-preview" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_COMPONENTS="rust-src,llvm-tools-preview"

$CARGOGREEN_ADD_APT

Adds OS packages to the base image with apt install.

Supported syntax:

  • libssl-dev
  • libssl-dev(>=3.5)
  • libssl-dev(=3.5.5-1~deb13u2) is the encouraged usage for better build cache hits

See also:

  • add.apk
  • base-image
[package.metadata.green]
add.apt = [ "libpq-dev", "pkg-config" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_ADD_APT="libpq-dev,pkg-config"

# Inspect the resulting base stage with:
cargo green supergreen show-rust-base 2>/dev/null

$CARGOGREEN_ADD_APK

Adds OS packages to the base image with apk add.

See also:

  • add.apt
  • base-image
add.apk = [ "libpq-dev", "pkgconf" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_ADD_APK="libpq-dev,pkg-conf"

# Inspect the resulting base stage with:
cargo green supergreen show-rust-base 2>/dev/null

$CARGOGREEN_EXPERIMENT

A comma-separated list of names of features to activate.

A name that does not match exactly is an error.

  • buildscriptsources:

    • Mounts build scripts' dependencies' sources so build scripts that read a dependency's bundled files at execution time may find them (eg. https://lib.rs/crates/protoc-bin-vendored ships protoc binaries).
  • cachebuildkit:

    • Write BuildKit build cache to disk
    • Reuse it from disk
  • finalpathcomments:

    • Write final containerfile on every rustc call.
    • Contains internal debugging structs: as commented TOML
    • Helps e.g. debug builds failing too early.
  • finalpathnonprimary:

    • Write final containerfile on every rustc call.
    • Perfect format to share the file.
    • Helps e.g. debug builds failing too early.
  • incremental:

    • Also wrap incremental compilation.
    • See https://doc.rust-lang.org/cargo/reference/config.html#buildincremental
  • repro:

    • Try and test for builds hermeticity and reproducibility.
    • See https://docs.docker.com/reference/cli/docker/buildx/build/#no-cache-filter

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_EXPERIMENT="finalpathnonprimary,repro"

Alternatives

In no particular order:

  • ipetkov's crane: A Nix library for building cargo projects. Never build twice thanks to incremental artifact caching.
    • crane.dev
    • => Very complete! Relies on nix tools and language.
  • Mozilla's sccache: sccache - Shared Compilation Cache
    • Cargo Book ~ Build cache ~ Shared cache
    • sccache's Known Caveats
    • => Relies on everyone having the same paths and doesn't cache all crate types.
  • garentyler's distrustc: A Rust-compatible distcc implementation
    • distcc's manpage
  • LukeMathWalker's cargo-chef: A cargo-subcommand to speed up Rust Docker builds using Docker layer caching.
    • 5x Faster Rust Docker Builds with cargo-chef
    • => Relies on everyone having the same paths + cache isn't crate-granular.
  • Bazel's rules_rust: Rules Rust
    • Building a Rust workspace with Bazel
    • Rust rules for Bazel
    • => Replaces cargo with bazel.
  • sgeisler's cargo-remote: cargo subcommand to compile rust projects remotely
    • Building on a remote server
    • => Uses rsync and ssh; seems unmaintained.
  • Swatinem's rust-cache: A GitHub Action that implements smart caching for rust/cargo projects
    • => A GitHub Action.
  • cross-rs's cargo-cross: “Zero setup” cross compilation and “cross testing” of Rust crates
    • Look at all these (compilation + testing) Supported targets!
    • Remote building and caching with Data Volumes
    • => seldomly maintained but a lifesaver for cross compilation. See also this article on what cargo-green does (perfect layering):
  • Better support of Docker layer caching in Cargo
    • Exploring the problem of faster Cargo Docker builds
    • Another reddit discussion

Origins

  • PoC originally written in Bash: https://github.com/fenollp/buildxargs/blob/buildx/tryin.sh
  • Initial blog post https://fenollp.github.io/faster-rust-builds-docker_host

About

A way for cargo to build crates locally or remotely and share and fine-grained--cache them via a Docker registry.

cargo.green

Topics

buildkitbuildxcargocargo-plugincargo-subcommanddocker-builderdocker-hostshacktoberfestrustc

Resources

Readme
AGPL-3.0 license
Activity

Stars

22 stars

Watchers

3 watching

Forks

2 forks
Report repository

Releases

Used by

Contributors

Languages

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.