GeezerVison

mihaibudiu left a comment

…le race

start_pipeline()/resume_pipeline()/start_pipeline_as_paused() (wait=True)
only wait for the control plane's deployment_status to report "running";
the pod itself can still be briefly unreachable through its k8s service
afterward (the control plane's own readiness probe is a point-in-time
check, not re-verified before the next caller's request). Calls through
the SDK client already retry transient 503s, but tests/platform/helper.py's
raw REST helpers are deliberately no-retry (other tests rely on that to see
raw status codes), so they're exposed whenever they're the first pod-facing
call after a start/resume.

Add wait_for_pipeline_reachable(), which polls /stats until it stops
returning 503 PipelineInteractionUnreachable, and call it at each of the 25
sites across 7 test files where that pattern occurs. test_pipeline_error.py
(the one that actually flaked in CI) now uses the shared helper instead of
its original bespoke retry loop.

Signed-off-by: Ben Pfaff <[email protected]>

mihaibudiu left a comment