GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — Forward the AWS web identity variables to cargo by xiasongh · Pull Request #7102 · feldera/feldera Conversation
|
| // Preserve the AWS variables that name an identity without carrying one: | ||
| // AWS_PROFILE for sccache, and the web identity pair for a cargo | ||
| // credential provider authenticating to a private registry. All other | ||
| // AWS_* vars are withheld to prevent leaking credentials to malicious | ||
| // build scripts. | ||
| for key in ["AWS_PROFILE", "AWS_ROLE_ARN", "AWS_WEB_IDENTITY_TOKEN_FILE"] { | ||
| if let Some(value) = std::env::var_os(key) { | ||
| command.env(key, value); | ||
| } | ||
| } |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The commit message argues these two are safe because "the token they point at is readable by a build script with or without them". That is only half of it: AssumeRoleWithWebIdentity needs both the token and the RoleArn, and the ARN is not in the JWT (the SA→role mapping lives in the IAM trust policy). So before this change a build script had a readable token it could not spend; after it, it has everything needed to mint credentials for the pod's role.
That matters here because the build scripts are not all ours — user-supplied UDF dependencies are spliced into the pipeline crate's Cargo.toml (udf_toml, ~L1475) and their build.rs runs in this process tree. That is exactly the leak the old comment was guarding against.
Not necessarily a blocker, but please either:
has_unstable_feature) so existing IRSA deployments do not silently change security posture just because they run on EKS, ordocs/get-started/enterprise/parallel-compilation.md that the compiler pod's role must be scoped to exactly what the registry/sccache needs, because arbitrary UDF build scripts can assume it.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| // credential provider authenticating to a private registry. All other | ||
| // AWS_* vars are withheld to prevent leaking credentials to malicious | ||
| // build scripts. | ||
| for key in ["AWS_PROFILE", "AWS_ROLE_ARN", "AWS_WEB_IDENTITY_TOKEN_FILE"] { |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Is this pair sufficient on its own? The EKS pod identity webhook normally injects AWS_STS_REGIONAL_ENDPOINTS=regional and, when configured, AWS_REGION/AWS_DEFAULT_REGION alongside these two, and most SDKs need a region to build the STS client for AssumeRoleWithWebIdentity (boto3 raises NoRegionError; the Rust SDK falls back to IMDS, which is often hop-limited in EKS). AWS_ROLE_SESSION_NAME is also injected when the SA is annotated. None of these carry a credential either, so they seem to fit the same rule.
Two related gaps worth a sentence in the PR description:
AWS_CONTAINER_CREDENTIALS_FULL_URI + AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE instead, and still will not work;RUSTC_WRAPPER) sees these variables at all, so it incidentally fixes the IRSA path that docs/get-started/enterprise/parallel-compilation.md:86 already promises for sccache. Worth calling out — and worth checking whether sccache's S3 backend needs a region beyond SCCACHE_REGION.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| for key in ["AWS_PROFILE", "AWS_ROLE_ARN", "AWS_WEB_IDENTITY_TOKEN_FILE"] { | ||
| if let Some(value) = std::env::var_os(key) { | ||
| command.env(key, value); | ||
| } |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The allowlist is the security boundary of this function, and nothing in the file pins it down. Consider lifting the selection out of call_compiler (which is untestable because it spawns cargo) into a small pure helper that takes a lookup, e.g.
const INHERITED_AWS_VARS: [&str; 3] = ["AWS_PROFILE", "AWS_ROLE_ARN", "AWS_WEB_IDENTITY_TOKEN_FILE"]; fn inherited_aws_vars( lookup: impl Fn(&str) -> Option<OsString>, ) -> Vec<(&'static str, OsString)> { ... }
Then the existing mod tests can assert: each allowlisted var is forwarded when set, absent vars are skipped, and AWS_SECRET_ACCESS_KEY/AWS_SESSION_TOKEN are not forwarded — the last one is the regression test that keeps the next person from widening this to AWS_*. Taking a lookup closure rather than reading the real process env also keeps the test off std::env::set_var, which is process-global (flaky under the parallel test harness, and unsafe in edition 2024).
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Verified locally: Two
Test cases the PR does not cover, in rough priority order:
Main substantive concern is in the inline comment on the allowlist: user-supplied UDF dependencies are spliced into the pipeline crate's |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
btw, can't ROLE_ARN ( and other vars we are whitelisting ) be part of ~/.aws/config under correct profile? or am i missing something? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
It can be. I thought it would be more natural to use these since these env vars are injected directly by k8s/aws and we're just letting them through, but I can write it into the config file instead. What's the reason we decided to go with this profile approach in the first place? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I'll go with profile instead, which won't require any changes here |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.