Provide environment information
- Self-hosted Trigger.dev v4.7.7 (webapp, supervisor, Helm chart 4.7.7)
- CLI
trigger.dev 4.7.7, runtime: "node-26"
- Webapp configured with custom base images (feat(webapp,cli): let self-hosted instances require deploy base images #5005):
DEPLOY_BASE_IMAGES=node-26=cgr.dev/chainguard/node@sha256:…
DEPLOY_BUILD_BASE_IMAGES=node-26=cgr.dev/chainguard/node:latest-dev@sha256:…
- No
image.instructions / aptGet extensions in trigger.config.ts
Describe the bug
Since 4.7.6, every trigger deploy fails building the task image when the webapp supplies custom base images. 4.7.4 and 4.7.5 deploy the same project fine.
#14 [base 2/3] RUN mkdir -p /usr/local/share/ca-certificates/emsign-block && printf '%s\n' …
#14 0.229 mkdir: can't create directory '/usr/local/share/': Permission denied
ERROR: process "/bin/sh -c mkdir -p /usr/local/share/ca-certificates/emsign-block && …" did not complete successfully: exit code: 1
generateContainerfile inserts distrustCompromisedRoots into the base stage and into the build stage when a build base is set (buildImage.ts#L748-L762, used at L844, L868 and L971). The step runs before the template's USER node and assumes the triggerdotdev/node Debian base:
| The step needs |
A typical custom base (e.g. cgr.dev/chainguard/node) |
| to run as root |
defaults to uid 65532 (node) |
/usr/local/share/ca-certificates/ |
no /usr/local/share |
/etc/ca-certificates.conf to sed |
absent |
update-ca-certificates |
absent (the image ships only the finished bundle) |
The step has no opt-out, so a self-hosted install that requires custom bases cannot deploy at all on 4.7.6 or 4.7.7. main has the same code.
Reproduction repo
None needed. The failing instruction reproduces on its own:
FROM cgr.dev/chainguard/node:latest
RUN mkdir -p /usr/local/share/ca-certificates/emsign-block
mkdir: can't create directory '/usr/local/share/': Permission denied
To reproduce
- Self-host v4.7.6+ with
DEPLOY_BASE_IMAGES / DEPLOY_BUILD_BASE_IMAGES pointing at a non-root image without Debian's ca-certificates tooling, e.g. cgr.dev/chainguard/node and :latest-dev.
- Run
trigger deploy with CLI 4.7.6 or 4.7.7.
- The image build fails at
[base 2/3] (and [build 2/…]) as above.
Suggested fix
Skip the step for stages built on a server-supplied base, and leave the base image's trust store to whoever supplies it. We run this as a local yarn patch:
+const distrustUnlessServerBase = (serverBase) => (serverBase ? "" : distrustCompromisedRoots);
…
-${distrustCompromisedRoots}`;
+${distrustUnlessServerBase(options.image?.buildBase)}`;
…
-${distrustCompromisedRoots}
+${distrustUnlessServerBase(options.image?.base)}
Other options, in case you'd rather keep the step on custom bases:
- An explicit opt-out, e.g. a webapp env var or an
image option.
- Running the step as
USER root and skipping it when /etc/ca-certificates.conf or update-ca-certificates is missing. This still leaves distroless or apk-based bases unprotected, so it seems weaker than documenting that a custom base owns its trust store.
Happy to open a PR for whichever you prefer.
Additional information
- Custom bases aren't necessarily exposed by skipping the step. On
cgr.dev/chainguard/node (Node 26.12.1, ca-certificates-bundle 20260909-r3), neither tls.rootCertificates nor the system store contains emSign ECC Root CA - G3, ECC Root CA - C3 or Root CA - C1.
- A runtime safeguard that works on any base is
tls.setDefaultCACertificates(getCACertificates('default').filter(…)), dropping those three roots at worker start. We run that in our worker init alongside the patch.
- Introduced by
dac3b033eb and 1fb4c7608e (shipped in 4.7.6), which postdate custom base images in feat(webapp,cli): let self-hosted instances require deploy base images #5005.