GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — [O2B-1567] Add resync GAQ summary button by isaachilly · Pull Request #2180 · AliceO2Group/Bookkeeping

Conversation

@isaachilly

@isaachilly isaachilly commented Jun 29, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Collaborator

I have a JIRA ticket

  • branch and/or PR name(s) include(s) JIRA ID
  • issue has "Fix version" assigned
  • issue "Status" is set to "In review"
  • PR labels are selected

Notable changes for users:

  • Resync button for GAQ now available for DPG admins
  • Can resync a specific dataPass and runNumber pair or via the action dropdown the whole dataPass's GAQ summaries

Notable changes for developers:

  • N/A

Changes made to the database:

  • N/A

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
isaachilly added 30 commits March 5, 2026 14:51
@isaachilly
[O2B-1545] Add GAQ summary models, adapters and migration
bb5a9c3
@isaachilly
[O2B-1545] Add GAQ summary repositories and timestamps
4fade0f
Add GaqSummaryRepository and GaqSummaryInvalidationRepository and export them from the repositories index.

Update migration to replace the previous invalidated_at field with created_at and add updated_at to mirror default Sequelize tables.
@isaachilly
[O2B-1545] Use default Sequelize createdAt/updatedAt instead of compu…
15afdaa
…tedAt
@isaachilly
[O2B-1545] Use mcReproducibleCoverage float instead of boolean
363f09b
Rename and change mcReproducible to be the coverage float not the boolean.
@isaachilly
[O2B-1545] Remove mcReproducible field
002f301
Forgot to remove.
@isaachilly
[O2B-1563] Invalidate GAQ summaries on related changes
5bebbb5
Add GAQ summary invalidation whenever underlying data affecting GAQ changes.

These changes ensure GAQ summary caches/records are marked for recomputation whenever detectors, QC flags, or run QC times that influence GAQ summaries are modified.
@isaachilly
Add GAQ worker and summary recalculation
ee55f59
Introduce background processing for GAQ summary invalidations and wire it into the app scheduler. Changes include:

- Add gaq config to services config.
- Move GaqService to lib/server/services/gaq and update imports across controllers/use-cases/tests.
- Extend GaqService to pop invalidations and recalculate summaries within a transaction.
- Add GaqWorker that guards concurrent runs and calls GaqService to process a batch of invalid summaries.
- Schedule the GaqWorker in application startup when GAQ recalculation is enabled.
- Add a soft-delete filter (where: { deleted: false }) when querying QC flags to map ids to run numbers as otherwise summary invalidations occur even on already deleted flags.

These changes enable periodic recalculation of GAQ summaries when invalidations are queued in the table.
@isaachilly
[O2B-1564] Create computeSummary to separate getting and computing
530216b
@isaachilly
[O2B-1564] Exclude reproducible coverage from badEffectiveRunCoverage
e1f3223
@isaachilly
[O2B-1545] Rename GAQ summary coverage fields
16d3563
@isaachilly
Merge branch 'feature/O2B-1545/Create-GAQ-Summary-Tables' into featur…
9bf8027
…e/O2B-1563/Create-GAQ-summary-invalidation-mechanism
@isaachilly
Merge branch 'feature/O2B-1563/Create-GAQ-summary-invalidation-mechan…
edcd9f7
…ism' into feature/O2B-1564/Create-background-GAQ-summary-worker
@isaachilly
[O2B-1564] Rename coverage fields to match table cols in GaqService o…
87afd8c
…utput
@isaachilly
[O2B-1565] Use GaqSummaryRepository in getSummary
37e97a3
Replace manual aggregation via getGaqCoverages with a direct query to GaqSummary table.

getSummary now builds a where clause and formats the summaries, if returned, via a new _formatSummary helper that now the mcReproducibleAsNotBad logic resides in.
@isaachilly
[O2B-1567] Allow passing fetch options (headers) to remote fetch
903a0e8
This will be useful for when we want to set the cache header.
@isaachilly
[O2B-1567] Add GAQ recalc endpoint and service logic
7f4ce67
Add a POST /gaq/recalculate API and corresponding handler to trigger recalculation of GAQ summaries for a data pass and optional run ranges.

In GaqService: include invalidation flag on fetched summaries, propagate it to formatted output, ensure upserts update updatedAt, adjust transaction handling, and add recalculateSummaries(dataPassId, runNumbers) which resolves run ranges, enqueues invalidations for matching DataPassRun entries, and returns the number of summaries scheduled for recalculation.
@isaachilly
[O2B-1567] Add GAQ summary recalculation UI support
cfd0538
- Add new getGAQSummaryDisplay.js to render GAQ state and per-run recalculate button for admins.
- Update getQcSummaryDisplay to accept an options.classes param and render an "invalidated" clock indicator when summary.invalidated is set.
  - implement recalculateGaqSummary(runNumbers) to POST to /api/qcFlags/summary/gaq/recalculate and refresh summaries.
@isaachilly
[O2B-1565] Forgot to rename db column names in formatSummary
86bdfdb
@isaachilly
Merge branch 'improv/O2B-1565/Migrate-GAQ-Summary-get-to-use-table' i…
d91a734
…nto feature/O2B-1567/Add-resync-GAQ-summary-button
@isaachilly
[O2B-1563] Add GAQ summary invalidation trigger tests
170c1b3
Validate GAQ summary invalidation behaviour on QC flag create/verify/delete, deleteAllForDataPass, explicit/default GAQ detector changes, and run QC time updates.
@isaachilly
[O2B-1563] Use dataPass.id in GAQ invalidation TYPO
78a2181
@isaachilly
Merge branch 'feature/O2B-1563/Create-GAQ-summary-invalidation-mechan…
a41b312
…ism' into feature/O2B-1564/Create-background-GAQ-summary-worker
@isaachilly
Make GAQ worker singleton and add pause/resume
abb263b
Converted gaqWorker to a singleton.

This allows adding of pause() and resume() methods to prevent the worker from processing invalidated summaries during test execution.

Reduced the default GAQ recalculation period from 1 minute to 10 seconds to improve test suite performance.
@isaachilly
[O2B-1564] Add GAQ worker tests; pause worker in DB reset
10fac20
Adds tests that verify the worker removes an invalidation and adds a summary, upserts for an already present summary, batch processes correctly, and doesn't run concurrent recalculations.

Update resetDatabaseContent to pause/resume the GAQ worker to avoid worker failures when the invalidation table is dropped.
@isaachilly
[O2B-1564] Add GaqService tests and register suite
6aa9617
GaqService summary recalculation and invalidation processing functions tested directly circumventing worker.

Tests verify correct computed fields, summary upsert, behaviour when no coverage exists, and batched invalidation processing.
@isaachilly
Merge branch 'feature/O2B-1564/Create-background-GAQ-summary-worker' …
bcc4005
…into feature/O2B-1567/Add-resync-GAQ-summary-button
@isaachilly
Add calculationFailed to GAQ summaries
c2ccbd4
Add a calculation_failed boolean column to gaq_summaries. This allows us to know whether a summary has been attempted to be calculated but unsuccessful due to limited data etc.
@isaachilly
Merge branch 'feature/O2B-1545/Create-GAQ-Summary-Tables' into featur…
1e78cac
…e/O2B-1563/Create-GAQ-summary-invalidation-mechanism
@isaachilly
Merge branch 'feature/O2B-1563/Create-GAQ-summary-invalidation-mechan…
f0dea1f
…ism' into feature/O2B-1564/Create-background-GAQ-summary-worker
@isaachilly
[O2B-1545] Make GAQ summary columns nullable
35112e5
Remove not-null constraints from several GAQ summary columns in the migration to allow NULL when values are unavailable.
isaachilly added 11 commits June 29, 2026 16:57
@isaachilly
Merge branch 'feature/O2B-1564/Create-background-GAQ-summary-worker' …
deb510e
…into improv/O2B-1565/Migrate-GAQ-Summary-get-to-use-table
@isaachilly
[O2B-1565] Fix for test to wait a little longer to be cleared
c2d24f0
@isaachilly
[O2B-1565] Debug logs to fix CI failing test
6b72a6a
@isaachilly
[O2B-1564] Add missing docker env vars to CI dockerfile
4a97602
@isaachilly
Merge branch 'feature/O2B-1564/Create-background-GAQ-summary-worker' …
c40cb9e
…into improv/O2B-1565/Migrate-GAQ-Summary-get-to-use-table
@isaachilly
Revert "[O2B-1565] Debug logs to fix CI failing test"
f379844
This reverts commit 6b72a6a.
@isaachilly
Update ca-certificates version in Dockerfile (#2138)
5fee0a4
@isaachilly
Merge branch 'feature/O2B-1545/Create-GAQ-Summary-Tables' into featur…
0b2dab4
…e/O2B-1563/Create-GAQ-summary-invalidation-mechanism
@isaachilly
Merge branch 'feature/O2B-1563/Create-GAQ-summary-invalidation-mechan…
6ee99b3
…ism' into feature/O2B-1564/Create-background-GAQ-summary-worker
@isaachilly
Merge branch 'feature/O2B-1564/Create-background-GAQ-summary-worker' …
ec04c1e
…into improv/O2B-1565/Migrate-GAQ-Summary-get-to-use-table
@isaachilly
Merge branch 'improv/O2B-1565/Migrate-GAQ-Summary-get-to-use-table' i…
d9cb7bf
…nto feature/O2B-1567/Add-resync-GAQ-summary-button
@isaachilly
isaachilly requested a review from graduta June 29, 2026 16:24
@isaachilly isaachilly self-assigned this Jun 29, 2026
@isaachilly isaachilly added the frontend label Jun 29, 2026
github-code-quality[bot]
github-code-quality Bot found potential problems Jun 29, 2026
View reviewed changes
Comment thread lib/public/Model.js Fixed

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/public/Model.js Fixed

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/public/Model.js Fixed

Uh oh!

There was an error while loading. Please reload this page.

Comment thread lib/public/components/common/form/inputs/DateTimeInputModel.js Fixed

Uh oh!

There was an error while loading. Please reload this page.

Comment thread test/public/runs/runsPerDataPass.overview.test.js
getPopoverSelector,
getInnerText,
getPopoverInnerText,
getColumnCellsInnerTexts,
Comment thread test/public/runs/runsPerDataPass.overview.test.js
const DataPassRepository = require('../../../lib/database/repositories/DataPassRepository.js');
const { BkpRoles } = require('../../../lib/domain/enums/BkpRoles.js');
const { navigateToRunsPerDataPass } = require('./navigationUtils.js');
const { invalid } = require('joi');
Comment thread test/public/runs/runsPerDataPass.overview.test.js
* @param {number} ms milliseconds to wait
* @return {Promise<void>}
*/
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
@isaachilly isaachilly changed the title Feature/o2 b 1567/add resync gaq summary button [O2B-1567] Add resync GAQ summary button Jun 29, 2026
isaachilly added 4 commits June 29, 2026 18:36
@isaachilly
Reapply "Merge branch 'main' into feature/O2B-1563/Create-GAQ-summary…
467630a
…-invalidation-mechanism"

This reverts commit 23f6e04.
@isaachilly
Merge branch 'feature/O2B-1563/Create-GAQ-summary-invalidation-mechan…
79fc244
…ism' into feature/O2B-1564/Create-background-GAQ-summary-worker
@isaachilly
Merge branch 'feature/O2B-1564/Create-background-GAQ-summary-worker' …
3f29232
…into improv/O2B-1565/Migrate-GAQ-Summary-get-to-use-table
@isaachilly
Merge branch 'improv/O2B-1565/Migrate-GAQ-Summary-get-to-use-table' i…
3ca19c2
…nto feature/O2B-1567/Add-resync-GAQ-summary-button
github-code-quality[bot]
github-code-quality Bot found potential problems Jun 29, 2026
View reviewed changes
Comment thread lib/public/views/Runs/RunPerDataPass/RunsPerDataPassOverviewPage.js Fixed

Uh oh!

There was an error while loading. Please reload this page.

@codecov

codecov Bot commented Jun 29, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.10145% with 77 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.60%. Comparing base (1758c68) to head (b252167).

Files with missing lines Patch % Lines
...uns/RunPerDataPass/RunsPerDataPassOverviewModel.js 0.00% 24 Missing ⚠️
...c/views/Runs/ActiveColumns/getGAQSummaryDisplay.js 0.00% 23 Missing ⚠️
...Runs/RunPerDataPass/RunsPerDataPassOverviewPage.js 0.00% 9 Missing ⚠️
...ic/views/Runs/ActiveColumns/getQcSummaryDisplay.js 0.00% 6 Missing ⚠️
lib/public/utilities/fetch/RemoteDataSource.js 0.00% 4 Missing ⚠️
lib/server/services/gaq/GaqWorker.js 92.59% 4 Missing ⚠️
lib/database/adapters/GaqSummaryAdapter.js 50.00% 3 Missing ⚠️
...ons/v1/20260223120000-create-gaq-summary-tables.js 66.66% 2 Missing ⚠️
lib/public/utilities/fetch/getRemoteData.js 0.00% 1 Missing ⚠️
lib/server/controllers/qcFlag.controller.js 90.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2180      +/-   ##
==========================================
+ Coverage   46.17%   46.60%   +0.42%     
==========================================
  Files        1038     1044       +6     
  Lines       17123    17341     +218     
  Branches     3122     3175      +53     
==========================================
+ Hits         7907     8081     +174     
- Misses       9216     9260      +44     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

isaachilly added 6 commits June 29, 2026 19:05
@isaachilly
[O2B-1567] Correct incorrect & complicated merge
b4600f6
@isaachilly
[O2B-1567] Fix failing tests
a437aa5
@isaachilly
[O2B-1567] Fix failing tests with missing fields
9bb5dbe
@isaachilly
[O2B-1567] Improve error / no summary handling
f240a16
@isaachilly
[O2B-1567] Fix failing test
4e83650
Test had a leftover button click that was meant to be deleted.
@isaachilly
Merge branch 'main' into feature/O2B-1567/Add-resync-GAQ-summary-button
b252167
@isaachilly
isaachilly marked this pull request as ready for review July 21, 2026 12:47
@graduta
graduta force-pushed the main branch from 544a0ee to 1758c68 Compare July 29, 2026 09:49

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@graduta graduta Awaiting requested review from graduta
1 more reviewer
@github-code-quality github-code-quality[bot] github-code-quality[bot] left review comments
Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@isaachilly isaachilly

Labels

frontend

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

1 participant

@isaachilly
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.