GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Don't record an overlay status when the job was cancelled by henrymercer · Pull Request #4122 · github/codeql-action

Conversation

@henrymercer

@henrymercer henrymercer commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

The overlay status mechanism records a marker in the Actions cache when building an overlay-base database fails, so that later runs skip overlay analysis instead of failing the same way, but it records one whenever the analyze step did not report success, which also covers runs that were cancelled, since the init post step still runs in that case. A cancellation tells us nothing about whether the analysis would have succeeded, and a marker disables overlay analysis for the whole repository until the next CodeQL release, so this adds an internal job-status input defaulting to ${{ job.status }} and skips recording when the job was cancelled.

Risk assessment

For internal use only. Please select the risk level of this change:

  • Low risk: Changes are fully under feature flags, or have been fully tested and validated in pre-production environments and are highly observable, or are documentation or test only.

Which use cases does this change impact?

Workflow types:

  • Advanced setup - Impacts users who have custom CodeQL workflows.
  • Managed - Impacts users with dynamic workflows (Default Setup, Code Quality, ...).

Products:

  • Code Scanning - The changes impact analyses when analysis-kinds: code-scanning.

Environments:

  • Dotcom - Impacts CodeQL workflows on github.com and/or GitHub Enterprise Cloud with Data Residency.
  • GHES - Impacts CodeQL workflows on GitHub Enterprise Server.

How did/will you validate this change?

  • Test repository - This change will be tested on a test repository before merging, specifically to confirm that a cancelled run reports cancelled to the post step.
  • Unit tests - I am depending on unit test coverage (i.e. tests in .test.ts files).

If something goes wrong after this change is released, what are the mitigation and rollback strategies?

  • Feature flags - The behaviour change only takes effect where overlay_analysis_status_save is enabled, so disabling that flag restores the previous behaviour. The new input is inert on its own.

How will you know if something goes wrong after this change is released?

  • Telemetry - I rely on existing telemetry. The codeql-action/overlay-disabled diagnostic reports skipped-due-to-cached-status, so the rate of that reason shows whether we are recording fewer markers as intended, and overlay adoption shows whether we have stopped recording ones we needed.

Are there any special considerations for merging or releasing this change?

  • No special considerations - This change can be merged at any time.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Consider adding a changelog entry for this change. Not added, since the behaviour is gated behind an existing feature flag, matching earlier changes to this mechanism.
  • Confirm the readme and docs have been updated if necessary.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
henrymercer and others added 2 commits September 3, 2026 18:21
@henrymercer
Add an internal job-status input to the init Action
2f3c1c9
Co-authored-by: Copilot App <[email protected]>
@henrymercer
Don't record an overlay status when the job was cancelled
38dd4a0
Co-authored-by: Copilot App <[email protected]>
@henrymercer
henrymercer requested a balanced review from Copilot September 3, 2026 17:24
@github-actions github-actions Bot added the size/M Should be of average difficulty to review label Sep 3, 2026
Copilot started reviewing on behalf of henrymercer September 3, 2026 17:25
View session
Copilot AI reviewed Sep 3, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

  • Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.

Copilot review overview

🟢 Approval recommended

The focused implementation is consistent with existing input handling and covers the new behavior with unit tests.

Review tier: Balanced
Findings: None

What changed in this PR

Prevents cancelled jobs from incorrectly disabling overlay analysis through cached failure markers.

Changes:

  • Passes the runtime job status into post-processing.
  • Skips overlay failure recording for cancellations unless CodeQL already reported an error.
  • Adds unit coverage for cancellation scenarios.
File Description
src/​init-action-post.ts Captures post-step job status before restoring inputs.
src/​init-action-post-helper.ts Adds cancellation-aware overlay status handling.
src/​init-action-post-helper.test.ts Tests cancellation and prior-error behavior.
init/​action.yml Defines the internal job-status input.
lib/​entry-points.js Generated output; excluded from review by policy.
Files excluded by content exclusion policy (1)
  • lib/entry-points.js

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@henrymercer
henrymercer marked this pull request as ready for review September 3, 2026 17:36
@henrymercer
henrymercer requested a review from a team as a code owner September 3, 2026 17:36
oscarsj
oscarsj previously approved these changes Sep 3, 2026
View reviewed changes

@oscarsj oscarsj left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@henrymercer
henrymercer added this pull request to the merge queue Sep 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 4, 2026

Uh oh!

There was an error while loading. Please reload this page.

@henrymercer
henrymercer added this pull request to the merge queue Sep 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 4, 2026

Uh oh!

There was an error while loading. Please reload this page.

mbg
mbg reviewed Sep 4, 2026
View reviewed changes

@mbg mbg left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Broadly looks good and makes sense, thanks! Just a few considerations that came to my mind while reviewing this.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread init/action.yml
[Internal] The ID of the check run, as provided by the Actions runtime environment. Do not set this value manually.
default: ${{ job.check_run_id }}
required: false
job-status:

@mbg mbg Sep 4, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never nice to have to feed this in through an extra action input, but I don't see a better approach for getting hold of the job status. An alternative option might be to set CODEQL_ACTION_STEP_(init|analyze|...) environment variables / state that we set to e.g. starting when the respective action starts and then to success or failure depending on the outcome. That should then allow us to identify which step started, succeeded, or failed (gracefully or not). For the overlay status, we could then check that all available environment variables with a CODEQL_ACTION_STEP_ prefix are success and none are starting or failure. The downside is that it wouldn't catch if the failure isn't related to what happens in CodeQL Action steps, or we fail to even set the starting value.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@henrymercer henrymercer Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That approach is also more complex, though it might be interesting to explore later, particularly if we also want to evaluate excluding failures from non-CodeQL Action steps.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread src/init-action-post-helper.ts Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/init-action-post-helper.ts Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread src/init-action-post-helper.ts
Comment on lines +426 to +429
return (
jobStatus === JobStatus.FailureStatus ||
jobStatus === JobStatus.ConfigErrorStatus
);

@mbg mbg Sep 4, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we end up with one of these if the workflow job was cancelled? E.g. because it caused a thread abort style exception to be thrown at an inconvenient moment?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@henrymercer henrymercer Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expect that it's possible, but the most common case would be the job genuinely failing in some way. It's worth looking into telemetry more once we've tackled the low-hanging fruit, but I'll leave as is for now.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

mbg reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
henrymercer and others added 2 commits September 4, 2026 17:31
@henrymercer
Read the job status from a ReadOnlyEnv in recordOverlayStatus
657964c
Co-authored-by: Copilot App <[email protected]>
@henrymercer
Record an overlay status only for conclusive job statuses
a48f2d3
Co-authored-by: Copilot App <[email protected]>
@henrymercer
henrymercer dismissed oscarsj’s stale review via a48f2d3 September 4, 2026 16:40
mbg
mbg approved these changes Sep 4, 2026
View reviewed changes

@mbg mbg left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes addressing the feedback from my previous review look good, thanks! Agreed on the other two points and happy to leave those for now. We can look into/address them later if necessary.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@henrymercer
henrymercer added this pull request to the merge queue Sep 4, 2026
Merged via the queue into main with commit 313a0b9 Sep 4, 2026
223 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@henrymercer
henrymercer deleted the henrymercer/friendly-potato branch September 4, 2026 17:10
This was referenced Sep 9, 2026
Merge main into releases/v4 #4131
Merged
Merge releases/v4 into releases/v3 #4133
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@mbg mbg mbg approved these changes
@oscarsj oscarsj oscarsj left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

size/M Should be of average difficulty to review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@henrymercer @mbg @oscarsj
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.