GeezerVison is showing github.com — Retry release 504s in lint-ecosystems installer steps by mikolalysenko · Pull Request #2040 · SocketDev/socket-patch
Conversation
Problem
lint-ecosystems runs on every PR, in the merge queue and nightly, and is part of ci-ok. Three of its steps run scripts/install.sh against real GitHub release assets. The script's curl -fsSL makes one attempt, so a single release-CDN 504 fails the job and ci-ok with it. On 2026-11-09 this happened twice during the same GitHub release-download outage (~05:38–05:47 UTC):
- PR run 42568357606, job 128999767264 (
fix/user-patched-deps-pnpm-yarn-vlt): Downloading SHA256SUMS... curl: (22) ... 504
- Nightly on main, run 42568829044, job 129001245735: the same 504 on
SHA256SUMS
During that outage the same 504s also failed the vexctl download (#2039 fixes that) and the ruby/setup-ruby downloads in two e2e legs. In the merge queue, each of these failures would evict an entry.
Root cause
The test reaches the live network on purpose, because it checks the real latest release end to end. But the download has no tolerance for transient errors.
Fix
A new step writes $RUNNER_TEMP/curl-retry/.curlrc containing retry = 7. The three sh scripts/install.sh invocations run with CURL_HOME pointing there, and curl reads its config from that directory. That gives the installer's curl a retry with exponential backoff (~2 min in all) on transient errors only: timeouts and HTTP 408/429/5xx. A 404, a missing checksum entry or a checksum mismatch still fails immediately. The probe's curl -sfI -m 20 in the Socket-origin step is not affected.
scripts/install.sh is not changed: install.socket.dev must serve a byte-for-byte copy of it (installer-drift.yml), so changing it needs a depscan redeploy. Adding --retry to the user-facing installer may be worth doing separately, as a coordinated change.
Proof
-
Local repro: a server that answers 504 three times and then 200. Plain curl -fsSL exits 22. With CURL_HOME pointing at the retry = 7 .curlrc, curl retries and exits 0 with the body (curl 8.5.0; ubuntu-24.04 runners ship the same 8.5.x).
-
The full CURL_HOME=... sh scripts/install.sh installs the latest release locally.
-
actionlint: same 175 findings before and after, none new. zizmor --offline: 54 findings before and after. YAML parses. python3 -m unittest discover -s scripts/tests: 411 tests OK.
-
On this PR's CI, during the ongoing release-download degradation: attempt 1 (job 129039901803) got 504s on all 8 tries over ~2.3 min and failed. Attempt 2 (job 129042262806) got 504s on 6 tries (07:51:39 → 07:52:10), then installed on the 7th and passed. All three installer steps succeeded. Without this change, that run would have failed on the first 504 as well.
Tests moved or removed
None. Every step runs where it ran before. Job and check names are unchanged.
https://claude.ai/code/session_011f9aeSwgKAzXeLTNSLn5pg
Generated by Claude Code
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
The three installer steps in lint-ecosystems run scripts/install.sh
against real GitHub release assets, and its single-attempt
`curl -fsSL` turned every release-CDN 504 into a red lint-ecosystems
and so a red ci-ok: twice on 2026-11-09 (a PR run and the nightly).
Write a .curlrc with `retry = 7` under RUNNER_TEMP and point CURL_HOME
at it only for the install.sh invocations. Transient errors (timeouts,
408/429/5xx) retry with exponential backoff, ~2 min in all; a 404 or
checksum mismatch still fails at once. install.sh is left untouched
because install.socket.dev must serve a byte-for-byte copy of it.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_011f9aeSwgKAzXeLTNSLn5pg
cursor[bot] reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit e6775d9. Configure here.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
lint-ecosystems (and so ci-ok) failed on e6775d9. The cause is ongoing GitHub degradation, not this diff, but it is the failure this PR targets: job 129039901803.
- The new retry fired as intended: 8 attempts at 1/2/4/8/16/32/64 s backoff (07:44:33 → 07:46:52), each a 504 on
releases/download/v5.0.1/socket-patch-x86_64-unknown-linux-musl.tar.gz.
- I reproduced it from outside CI at the same time. That URL takes ~11 s per request and returns 504 on about 1 in 3 requests (302 otherwise).
SHA256SUMS from the same release returns 200. The same endpoint has been returning 504s since ~05:38 UTC; that is also why the nightly failed.
- Every other job on this head passed. I re-ran the failed jobs once.
The retry turns short 504 bursts into a pass. It can't outlast a release-download outage that lasts several minutes, and a longer window would run into the job's 10-minute timeout across three installer steps. If this degradation keeps up, the more durable follow-up is to have only the first installer step download the release and point the alternate-origin step at a local mirror. That changes what the step tests, so it needs an owner's call and isn't part of this PR.
Generated by Claude Code
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Ready for review at e6775d93381628c2149eeb257da0fe6c06efa987.
- CI: 32/32 checks pass (including
ci-ok and lint-ecosystems on the re-run of the 504-hit job), 19 skipped. No main-wide failures.
- Bugbot reviewed
e6775d9 and found no issues.
- For the reviewer: the retry rides out short release-CDN 504 bursts but not a multi-minute outage (see the previous comment for the follow-up idea, which needs an owner's call).
Generated by Claude Code
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Final review brief
What it does. The three lint-ecosystems steps that run scripts/install.sh against real release assets now set CURL_HOME to a temp dir whose .curlrc says retry = 7. A release-CDN 504 then gets retried with backoff (~2 min) and no longer fails ci-ok. install.sh itself is unchanged, so the hosted copy at install.socket.dev doesn't drift.
Risk: low. This is a CI-only change in one job. curl's --retry covers only transient errors (timeouts, 408/429/5xx), so a 404 or a checksum mismatch still fails at once. install.sh calls plain curl -fsSL without -q, so it does read the .curlrc.
Look here
Verified
- Read the full diff. It is 1 file, +17/−1, and
CHANGELOG.md is untouched.
- Confirmed that
install.sh downloads with curl -fsSL (no -q/--disable), so CURL_HOME takes effect. The wget fallback isn't used on these runners.
- The job is Linux-only (
depot-ubuntu-24.04-4 / ubuntu-latest), so there's no Windows path concern.
- One edge case, not a blocker: if every download in all three steps used up its retries, the job would reach its 10-minute timeout. The result is the same failure, just slower.
- CI on
e6775d9: 32 passed, 19 path-skipped, ci-ok and clippy green. The PR body links an attempt that installed on the 7th retry during the 504 outage.
- Cursor Bugbot: success, no comments. No review threads.
Changes I made. None.
Auto-merge is armed, so approving sends this straight to the merge queue.
Generated by Claude Code
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Labels
ci-janitor
Opened by the CI janitor routine (flakes, redundant tests, CI perf)
Ready for review
Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|