GeezerVison

Draft of a containerd change to apply and write layer archives for a
user namespace remapped storage, so that snapshots can keep the
remapped host IDs while layers keep the container IDs.

Add archive.WithApplyIDMapping, which maps the owners of the entries
to the host IDs before they are created, and makes the root user of
the mappings own the root directory, which archives do not change, and
the parent directories created without attributes to inherit.
Add archive.WithWriteDiffIDMapping and
archive.WithChangeWriterIDMapping, which map the owners of the entries
back to the container IDs, keep whiteouts owned by ID 0, and convert
version 3 file capabilities, which store the host ID of the user
namespace root, to version 2.
Apply and WriteDiff refuse the mapping with custom apply and diff
functions, such as the Windows layer ones.
Add IDMap.ToContainer to the internal userns package.

This patches the vendored containerd until the change lands upstream.

Signed-off-by: Paweł Gronowski <[email protected]>

Draft of a containerd change to pass user namespace mappings to the
diff appliers and comparers, through the existing option plumbing of
unpacking and diffing.

Add diff.WithApplyIDMapping and diff.WithIDMapping, with the mappings
in ApplyConfig and Config.
The file system applier and the walking differ pass them to the
archive package.
The diff service proxy refuses them with ErrNotImplemented, as the
diff service API has no fields for them yet, and an older daemon would
ignore them.

This patches the vendored containerd until the change lands upstream.

Signed-off-by: Paweł Gronowski <[email protected]>

With userns-remap, the containerd image store unpacked layers with
their container IDs, and chowned the whole rootfs to the remapped IDs
when creating a container.
With overlayfs, the chown copied every file of the image up into the
upper directory of each container, and committing a container copied
its whole rootfs again to map the IDs back.
The image snapshots also kept files owned by host root, including
setuid binaries, protected only by the permissions of the snapshotter
directories.

Store layers with the remapped IDs instead, as the graph drivers do.
With userns-remap, the daemon's containerd client uses a diff service
which runs containerd's file system applier and walking differ in the
daemon process with the ID mapping options, as the containerd diff
service API does not take them.
Applying a layer maps the owners of its files to the host IDs, and
creating a layer maps them back to the container IDs.
All unpacks and diffs of the client, such as pulls, loads, commits
and classic builds, go through it.
Containers use the image snapshots as they are, and commits diff the
container snapshot directly.
Archives never change the root directory of a snapshot, so the applier
and the snapshots prepared without a parent make it owned by the
remapped root.

The daemon disables the containerd image store with userns-remap, so
no existing setups depend on the previous layout.

Signed-off-by: Paweł Gronowski <[email protected]>

…rary)

With a snapshotter with an ID mapping, BuildKit computed some layers
without the worker's differ, so they kept the remapped host IDs of the
snapshots: with the overlay differ, and for layers compressed with
zstd, estargz or nydus, or with a custom gzip level.

Patch the vendored BuildKit to map the owners of the tar entries back
to the container IDs of the snapshotter's ID mapping in both paths,
before compressing them and computing the diffID.

This is a temporary patch to validate the BuildKit change, which has
to land upstream instead.

Signed-off-by: Paweł Gronowski <[email protected]>

With userns-remap and the containerd image store, BuildKit failed to
run build steps with errors like:

    error mounting ".../buildkit/executor/resolv.conf" to rootfs at "/etc/resolv.conf": ... permission denied

The containerd worker applied and created layers with the containerd
diff service, and its snapshotter had no ID mapping, so the rootfs of
build steps kept the container IDs, which the remapped root cannot
access.

Give the containerd worker the remapped diff service and snapshotter
of the image service, and the ID mapping, so that its snapshots have
the remapped IDs like the image snapshots they share, and file
operations, mounts and the layers it computes itself map the IDs.

Signed-off-by: Paweł Gronowski <[email protected]>

With userns-remap, the daemon disabled the containerd image store, and
refused to start when it was explicitly enabled, because BuildKit
failed to run build steps with it.

The containerd image store now stores layers with the remapped IDs,
and BuildKit builds and exports layers with them, so remove the
restriction.
With userns-remap, the daemon picks the image store as without it: a
remapped root with graph driver data keeps using the graph driver, and
a new one uses the containerd image store.
Migrating to the containerd image store is still refused.

Remove the workaround of test daemons which set the overlay2 storage
driver with userns-remap, and run the tests skipped with userns-remap
because of the restriction.

Signed-off-by: Paweł Gronowski <[email protected]>

The userns mode only ran with graph drivers, as userns-remap disabled
the containerd image store.
Run the integration tests in the userns mode with the containerd image
store too, and the integration-cli suites which pull and push images,
as the containerd image store remaps image layers when unpacking and
creating them.

Signed-off-by: Paweł Gronowski <[email protected]>