GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Add opt-in retention for generic artifacts by profiling · Pull Request #513 · git-pkgs/proxy

Conversation

@profiling

@profiling profiling commented Oct 18, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Orca publishes frequent releases with large Linux AppImage and Windows installer assets. I use the generic GitHub Release proxy to cache these downloads; fetching or prewarming new releases leaves older installers in storage, and their accumulated size can exhaust the available disk. storage.max_size provides a global LRU limit, but cannot express an inactivity limit for selected GitHub repositories.

This adds explicit opt-in retention for generic artifacts while preserving the behavior of existing storage.retention.default configurations.

Compatibility and configuration

The discussion in #306 deliberately excluded generic because release assets may disappear upstream. This PR preserves that behavior unless a user explicitly opts in. The global storage.retention.default never applies to generic, even when packages contains generic entries. Other ecosystems retain their existing behavior.

Use case Generic behavior
No storage.retention configuration No age-based eviction; existing behavior is preserved.
Only storage.retention.default: "30d" Generic remains excluded; the default still applies to other supported ecosystems.
Explicit generic entries in packages, without ecosystems.generic Only the listed packages use their configured retention. All other generic packages remain excluded, even if default is set.
Explicit ecosystems.generic: "30d" All generic packages use 30 days unless a package rule overrides it.
Both ecosystems.generic and a matching package rule The package rule wins, including "0" to disable age-based eviction for that package.

1. Preserve existing behavior: no opt-in. Leave storage.retention unset. Existing installations that already use only a global default also keep generic excluded:

storage:
  retention:
    default: "30d"  # Other supported ecosystems only; generic stays excluded.

2. Opt in selected packages only. This example enables retention for one GitHub repository. It does not enable retention for any other generic repository. default is optional and is omitted here to make the scope clear:

storage:
  retention:
    packages:
      "pkg:generic/owner%2Frepo": "30d"

Adding default: "30d" to this example would affect other supported ecosystems, but would still not opt in unlisted generic packages. A package rule covers that package's versions and assets; each cached asset ages individually based on its last access, with recently fetched files protected as before.

3. Opt in all generic packages. Set the generic ecosystem rule explicitly:

storage:
  retention:
    ecosystems:
      generic: "30d"

4. Override or exclude a package after opting in the ecosystem. This keeps one repository outside age-based eviction and gives another a longer retention:

storage:
  retention:
    ecosystems:
      generic: "30d"
    packages:
      "pkg:generic/owner%2Fkeep": "0"
      "pkg:generic/owner%2Flong-lived": "90d"

For generic, precedence is matching package rule → explicit generic ecosystem rule → no age-based eviction. There is no fallback to the global default.

Explicit rules also apply to existing cached assets, so the initial sweep can evict older unused files. Users opting in accept that an evicted upstream asset may no longer be available for refetch. storage.max_size remains independent: exclusion from age-based eviction, including "0", does not prevent capacity-based LRU eviction.

Implementation

  • Register generic with its existing stored package PURL form, such as pkg:generic/jqlang%2Fjq.
  • Resolve explicit package and ecosystem rules before returning zero for generic instead of falling back to the global default.
  • Keep generic in the startup log's default_not_applied list.
  • Reuse the existing sweeper, deletion queue, and grace period, without new configuration fields or schema changes. Metadata/API responses are not retention candidates.
  • Document the opt-in behavior and upstream availability tradeoff.

Validation

Tests cover legacy global-default preservation, explicit ecosystem and package opt-in, isolation from other generic repositories, package-level disabling, actual Release package PURL storage, recent-access protection, eviction, and refetch through the generic HTTP route. The integration upstream is an httptest.Server, not live GitHub.

Passed on Linux with Go 1.26.7, against a source snapshot containing only this PR's changes:

go test -p=1 ./... -timeout 5m
go vet -p=1 ./...
go build -p=1 -buildvcs=false ./...
go test -p=1 -race ./internal/config ./internal/retention ./internal/handler ./internal/server -run 'Generic|Retention|NotCovered|RulesFor|RulesMinPositive' -count=1 -timeout 3m

Changed Go files pass gofmt, and git diff --check passes. -buildvcs=false was used because the validation snapshot has no .git directory. PostgreSQL integration tests were skipped (PROXY_DATABASE_URL was unset); local tests used SQLite. Full race and lint checks across the CI operating-system matrix are left to CI.

Related to #306 and #428.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
ztzhou added 2 commits October 18, 2026 20:02
ztzhou
Add opt-in retention for generic artifacts
f1ea481
ztzhou
Remove constant parameter from generic retention test helper
39d10b1
andrew
andrew approved these changes Oct 18, 2026
View reviewed changes

@andrew andrew left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The explicit opt-in preserves existing retention configurations. Verified that upgrading with only a global default keeps generic assets cached, and a package rule evicts only the selected repository's assets, which can then be fetched again. Approval is for the code; merge should wait for CI to pass.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@andrew
andrew merged commit d643836 into git-pkgs:main Oct 19, 2026
6 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@profiling
profiling deleted the feat/generic-retention branch October 21, 2026 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrew andrew andrew approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@profiling @andrew
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.