GeezerVison is showing github.com — chore: update Go from 1.25.7 to 1.26.1 by blinkagent[bot] · Pull Request #22262 · coder/coder
Conversation
blinkagent
Bot
commented
Feb 23, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Something is very wrong with Windows on this.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
I've been investigating this. Here's a summary:
What's happening
The Windows test binary is hitting Go's default 10-minute per-package timeout. All failures show (unknown) status — no test actually fails an assertion, they're just killed mid-execution. The goroutine dumps in the logs confirm this: goroutines are stuck in normal operations (selectgo, IO wait, sync.WaitGroup.Wait) when the timeout fires.
Run 1: DONE 2010 tests, 72 skipped, 781 failures in 56.673s (cold cache)
Run 2: DONE 2096 tests, 81 skipped, 59 failures in 94.478s (warm cache, got further)
Root cause
Go 1.26 introduces the Green Tea GC as the default garbage collector. There are known Windows-specific issues:
- golang/go#77567: High memory usage on Windows with Go 1.26 (Green Tea GC related)
- golang/go#77407:
os.RemoveAll regression on Windows (backported to Go 1.26.1)
The test suite runs heavyweight integration tests (HTTP servers, wireguard tunnels, embedded postgres) in parallel. The new GC appears to cause enough overhead on Windows to push the tests past their timeout.
Options
GOEXPERIMENT=nogreenteagc — disable Green Tea GC at build time for the Windows test job to confirm this is the cause
- Increase Windows test timeout — add
-timeout 20m to GOTEST_FLAGS (currently using Go's 10m default)
- Wait for Go 1.26.1 — which will include fixes for the Windows-specific issues
- Reduce Windows test parallelism — currently 8 packages × 16 tests = 128 concurrent tests
|
johnstcn reacted with heart emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
blinkagent
Bot
changed the title
chore: update Go from 1.25.7 to 1.26.0
chore: update Go from 1.25.7 to 1.26.1
Mar 24, 2026
Windows CI Failure Investigation
The test-go-pg (windows-2022) job is failing with 621 out of 2386 tests marked as failures. This is caused by a known Go runtime bug affecting Go 1.26.0 and 1.26.1 on Windows — not related to any code changes in this PR.
Root Cause
The crash signature is:
fatal error: unexpected signal during runtime execution
panic during panic
[signal 0xc0000005 code=0x0 addr=0xe8 pc=0x7ff70ebdab05]
This is an ACCESS_VIOLATION in runtime.(*unwinder).next during runtime.scanstack → runtime.markroot (GC stack scanning). The underlying issue is in internal/poll on Windows: WSARecvFrom writes to the rsan (address length) parameter after overlapped I/O completes. When rsan is a local stack variable inside a closure, and the goroutine's stack is moved/shrunk between the syscall initiation and completion, Windows writes to the old (now-invalid) stack address, corrupting return addresses.
Notably, this is not caused by the Green Tea GC — GOEXPERIMENT=nogreenteagc still reproduces the crash.
Upstream Tracking
Ubuntu CI Failure
The test-go-pg (ubuntu-latest) failure is unrelated — only 2 test failures out of 17,351 (TestRun/RPTY timeout), which appears to be a flaky test.
Recommendation
The Windows failure can be ignored until Go 1.26.2 is released with the fix. This PR should be safe to merge once rebased onto the latest main and the non-Windows checks pass.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Reviewers
jdomeracki-coder
Awaiting requested review from jdomeracki-coder
jdomeracki-coder is a code owner
johnstcn
Awaiting requested review from johnstcn
Uh oh!
There was an error while loading. Please reload this page.
Labels
stale
This issue is like stale bread.
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|