GeezerVison

Summary

Three defects in setup-env's BrowserStack re-run delivery path, surfaced by a Cypress + GitHub Actions re-run investigation (SDK-7461).

1. BROWSERSTACK_BUILD_RUN_IDENTIFIER has been silently dropped since 2026-05-07.

The rebuild/details response carries three variables. Verified against the live API (runId 29728341550, HTTP 200):

{"data":{"variables":{
  "BROWSERSTACK_RERUN": "true",
  "BROWSERSTACK_RERUN_TESTS": "cypress/tests/FlightOffer/FO-E2E-05.ts,...",
  "BROWSERSTACK_BUILD_RUN_IDENTIFIER": "1784546348645-29728341550"
}}}

ALLOWED_RERUN_ENV_VARS (added in 8ade0a3, APS-19076) lists only the first two plus BUILD_NAME, so the third is filtered out — the action logs Ignoring non-allowlisted env var from BrowserStack rerun API: BROWSERSTACK_BUILD_RUN_IDENTIFIER and moves on. Before 8ade0a3 the export loop was unfiltered, so this variable did reach the runner.

The SDKs read it as build_run_identifier in the build-start payload (browserstack-cypress-cli bin/testObservability/helper/helper.js:412, bin/testhub/testhubHandler.js:65) — it is what files a re-run as an attempt of its parent build run rather than as an unrelated build.

2. Every way re-run delivery can fail was silent.

With github-token left at its action.yml default of 'none', the job log on a re-run is byte-identical to an ordinary run — checkIfBStackReRun() returns false from two branches that log nothing. The CLI then runs the full spec set, because its only re-run switch is the presence of BROWSERSTACK_RERUN_TESTS. So "all my tests ran again instead of just the failed ones" was undiagnosable from the log, which is how SDK-7124 and SDK-7461 both stalled.

checkIfBStackReRun() now names the missing input, explains a human-triggered re-run, and an API failure is a core.warning rather than core.info. An absent GITHUB_TRIGGERING_ACTOR gets its own message rather than the App-not-installed one — that variable comes from the runner binary, so on a self-hosted runner it can simply be missing, and blaming the GitHub App there sends the reader after an install that is fine. Attempt 1 stays silent — an ordinary run is not a degraded re-run.

PR #85 predicted this. The allowlist's own PR body carries the caveat verbatim:

If the rerun API legitimately sets additional names in production, the list will need to grow — they will be visible as core.warning lines in workflow logs.

BROWSERSTACK_BUILD_RUN_IDENTIFIER is exactly such a name. The warning has been emitted on every BrowserStack-triggered re-run since 2026-05-07; nobody was watching for it. This PR is that predicted growth.

3. An unreported GITHUB_TRIGGERING_ACTOR aborted delivery on an otherwise correct setup.

checkIfBStackReRun() bailed when that variable was absent. But it comes from the runner binary, so a self-hosted runner can simply not set it — and the actor check is only a cheap pre-filter: rebuild/details is the authority and returns no variables when BrowserStack did not trigger the re-run. So the bail converted a working re-run into a full-suite run. It now warns and asks the API instead. This surfaced on a self-hosted Windows runner fleet, where the variable was not being set.

Security note (APS-19076 preserved)

The allowlist still blocks arbitrary names; only one BrowserStack-owned, BrowserStack-consumed name is added. The existing negative test asserting NODE_OPTIONS / PATH / GITHUB_TOKEN are rejected is unchanged and passing.

Testing

• 47 unit tests passing (43 before, +4): identifier is exported; a missing input is named; a human-triggered re-run is explained; an unreported triggering actor is distinguished from a non-BrowserStack one.
• eslint clean; dist rebuilt with ncc, reproducible (a second npm run build gives no further diff).
• Runner-level A/B on the built dist/index.js, run exactly as a GitHub runner does (INPUT_* env, output read from $GITHUB_ENV), against the real details API:

CLI-side behaviour, measured on a real project

Driving the real published CLI (1.36.20, current latest) over a real reported project with that project's own --spec argument:

This establishes the CLI half: the full set runs precisely when the list is absent, and the list is honoured in either format when present.

It is not, by itself, diagnostic of any particular failing build — a healthy first run also dispatches the full set, because the list is legitimately absent there. Distinguishing the two requires GITHUB_RUN_ATTEMPT > 1, which is not recoverable from build data alone. That project's workflow was correctly configured (step present with github-token, same job, before Run Tests, no env shadowing, CLI installed unpinned), which is what pointed at change 3.

Scope — what this PR does and does not claim

These are three independent defects in the re-run delivery path, each evidenced on its own. The PR does not claim to resolve any specific support ticket: attribution of a particular customer failure to any one of these three is not established, and change 1 in particular is ruled out as a cause of full-suite re-runs (the spec list is delivered correctly while the identifier is dropped). Change 1 stands on its own merit — it has silently affected every setup-env re-run user since 2026-05-07.

Blast radius

Small, but not zero. Change 3 means workflows whose runner does not report a triggering actor will now make one extra authenticated GET on re-run attempts (>1) and, when BrowserStack did trigger the re-run, will start receiving the failed-test list they should always have had. Workflows on a BrowserStack-triggered re-run will now additionally receive BROWSERSTACK_BUILD_RUN_IDENTIFIER — the SDKs already read it and it was present before APS-19076, so this restores prior behaviour rather than introducing new behaviour. The rest is log-only. All eight delivery states were exercised end-to-end through the built action into the real CLI: pre-fix seven of eight produce a full-suite re-run, post-fix six of eight do (S4 and S5 now deliver), and each remaining failure carries a distinct job-log signature. Docs recommend pinning @master, so this reaches every setup-env user on merge.

Ref: SDK-7461, SDK-7124.

🤖 Generated with Claude Code