GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — verity volumes: attach verified read-only data disks to CVMs by h4x3rotab · Pull Request #752 · Dstack-TEE/dstack

Conversation

@h4x3rotab

@h4x3rotab h4x3rotab commented Jul 3, 2026 •
edited by kvinwang
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

What this does

Adds generic read-only verity data volumes to CVMs. An operator builds or wraps a filesystem image, places it in the VMM's configured volumes_dir, and attaches it by name at deploy time. The guest discovers the disk, verifies it against the root measured in app-compose.json, and mounts it read-only at the requested absolute path.

How it fits together

  • dstack verity --dir <path> packs a directory into a reproducible squashfs volume.
  • dstack verity --fs-image <path> wraps an existing filesystem image.
  • Each disk uses a generic DSTACK_VOLUME envelope in partition 1, filesystem data in partition 2, and the dm-verity superblock/hash tree in partition 3.
  • The VMM resolves bare volume names under cvm.volumes_dir and attaches the raw disks read-only.
  • The guest dstack-volume helper scans /sys/class/block, matches the full root, opens the disk with veritysetup using the measured root, and mounts it before the application starts.
  • Attestation binds verity_root and target through app-compose.json and therefore the measured app_id.

Example:

dstack verity --dir ./models -o models.img
dstack deploy -c docker-compose.yaml \
  --volume models.img:<verity-root>:/run/models

See docs/verity-volumes.md.

Why Docker image seeding is not included

An earlier version injected pre-extracted layers directly into Docker's private overlay2 state. That couples the feature to Docker internals (imagedb, layerdb, overlay2, and repository metadata) and creates unsafe lifecycle edge cases: bind mounts disappear on reboot while Docker metadata persists, pulled images may share layers with verity-seeded images, and removing or changing a volume can leave existing images or containers with dangling layer references.

Safely reconciling that state would require ownership tracking and Docker-specific graph cleanup, while a custom containerd snapshotter would be a substantially larger and less mature integration. This PR therefore keeps the trust and lifecycle boundary simple: it only exposes verified, read-only data filesystems and does not modify Docker image state.

Validation

  • cargo clippy -p dstack-volume --all-targets -- -D warnings
  • cargo clippy -p dstack-cli --all-targets -- -D warnings
  • cargo test -p dstack-volume -p dstack-cli -p dstack-types
  • cargo test -p dstack-vmm --no-fail-fast
  • End-to-end local volume build with dstack verity --dir

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@h4x3rotab h4x3rotab changed the title verity volumes: reproducible dm-verity image/data pre-seeding for CVMs verity volumes: pre-load docker images and data into CVMs Jul 4, 2026
@h4x3rotab
h4x3rotab force-pushed the verity-volumes branch 12 times, most recently from b0a463a to 109c584 Compare July 4, 2026 23:16
github-advanced-security[bot]
github-advanced-security Bot found potential problems Jul 7, 2026
View reviewed changes
Comment thread dstack/vmm/src/app/qemu.rs Fixed

Uh oh!

There was an error while loading. Please reload this page.

Comment thread dstack/vmm/src/app/qemu.rs Fixed

Uh oh!

There was an error while loading. Please reload this page.

Comment thread dstack/vmm/src/app/qemu.rs Fixed

Uh oh!

There was an error while loading. Please reload this page.

@kvinwang
kvinwang force-pushed the verity-volumes branch from dfbc7a5 to 76b324f Compare July 13, 2026 04:13
github-advanced-security[bot]
github-advanced-security Bot found potential problems Jul 13, 2026
View reviewed changes
Comment thread dstack/vmm/src/app/qemu.rs Fixed

Uh oh!

There was an error while loading. Please reload this page.

Comment thread dstack/vmm/src/app/qemu.rs Fixed

Uh oh!

There was an error while loading. Please reload this page.

Comment thread dstack/vmm/src/app/qemu.rs Fixed

Uh oh!

There was an error while loading. Please reload this page.

@kvinwang
kvinwang force-pushed the verity-volumes branch from 0f07949 to 9b974c4 Compare July 16, 2026 09:02
@kvinwang kvinwang changed the title verity volumes: pre-load docker images and data into CVMs verity volumes: attach verified read-only data disks to CVMs Jul 21, 2026
h4x3rotab and others added 12 commits July 21, 2026 02:19
@h4x3rotab @claude @kvinwang
vmm: attach verity volumes to CVMs
8238e03
Add read-only verity volumes -- extra virtio-blk disks a CVM can mount
instead of pulling and unpacking their contents. A volume is declared in
app-compose.json as `{ verity_root, target }`, and attached at deploy time
with `--volume <name>`: the vmm looks the name up under cvm.volumes_dir and
attaches it read-only. Because verity_root is part of the measured compose,
the guest can check the bytes it's handed against it.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@h4x3rotab @claude @kvinwang
guest: seed verity volumes before dockerd
7eea80c
Just before dockerd starts, dstack-prepare.sh runs the seeding helper. For
each volume declared in the compose it finds the matching disk by opening it
with veritysetup against the measured verity_root, then either seeds docker's
overlay2 store (target "docker") so the images are already present, or mounts
the volume at a path (a data volume). It's fail-safe throughout: a volume
that's missing or doesn't verify is skipped, and its images just pull.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@h4x3rotab @claude @kvinwang
dstack verity: build reproducible verity volumes
684efb9
`dstack verity <image>...` builds a squashfs + dm-verity volume that
pre-extracts docker images (or, with `--dir`, a plain directory) so a CVM can
start without pulling or unpacking them. It fetches images itself through
oci-client -- no docker daemon -- and lays out the overlay2 store
deterministically (each layer's directory id is its chain-id, with a fixed
timestamp and salt). The same inputs always produce the same verity_root, so
anyone can recompute it from the pinned image digests and confirm what's in
the volume without trusting the builder. `dstack deploy --volume` attaches
the result.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@h4x3rotab @claude @kvinwang
docs: add verity volumes design
e083b39
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@kvinwang
verity: use partitioned volume images
c0a1a89
@kvinwang
vmm: resolve volume paths with path-absolutize
9ec5e96
@kvinwang
docs: relocate verity volume guide for monorepo layout
1089356
@kvinwang
guest: package verity helper in OS image
9c3bca7
@kvinwang
refactor(vmm): align verity volumes with split qemu modules
07a9c00
@kvinwang
feat(verity): discover self-describing guest volumes
1496b26
@kvinwang
refactor(verity): share volume format with guest
a5a5412
@kvinwang
refactor(verity): propagate conversion errors
e18f6e5
kvinwang added 6 commits July 21, 2026 02:20
@kvinwang
refactor(verity): simplify volume envelope
302ccb7
@kvinwang
refactor(verity): use command helpers in builder
dc938c8
@kvinwang
refactor(volumes): merge builder and guest tooling
c61ff43
@kvinwang
refactor(volume): use singular crate name
39ae0b8
@kvinwang
refactor(volume): drop docker image seeding
34c1d6b
@kvinwang
fix(volume): fail immediately on activation errors
77ac69d
@kvinwang
kvinwang force-pushed the verity-volumes branch from b2fbd4c to 77ac69d Compare July 21, 2026 09:24
kvinwang added 9 commits July 21, 2026 03:28
@kvinwang
fix(verity): measure attached volume device count
b22f575
@kvinwang
refactor(volume): configure attachments in app compose
aa47b6b
@kvinwang
refactor(volume): parse commands with clap derive
38087de
@kvinwang
refactor(volume): remove single-volume mount command
895e600
@kvinwang
fix(volume): address CLI and verity review findings
b657585
@kvinwang
fix(volume): preserve compose compatibility and enforce invariants
5ffb0f3
@kvinwang
fix(volume): share duplicate roots across mount targets
517eec9
@kvinwang
Merge remote-tracking branch 'origin/master' into codex/pr752-dstack-…
2edd4ef
…volume

# Conflicts:
#	dstack/crates/dstack-cli-core/src/compose.rs
#	dstack/crates/dstack-cli/src/main.rs
@kvinwang
Merge remote-tracking branch 'origin/master' into codex/pr752-dstack-…
f3c62a3
…volume

# Conflicts:
#	dstack/vmm/src/app/qemu.rs
@kvinwang
kvinwang merged commit 3768c5a into master Jul 22, 2026
15 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@kvinwang
kvinwang deleted the verity-volumes branch August 7, 2026 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@github-advanced-security github-advanced-security[bot] github-advanced-security[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@h4x3rotab @github-advanced-security @kvinwang
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.