GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — npm: read the stored publish time in the cooldown download check by VictorCodesseira · Pull Request #296 · git-pkgs/proxy

Conversation

@VictorCodesseira

@VictorCodesseira VictorCodesseira commented Sep 1, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

With cooldown enabled, versionInCooldown runs on every npm tarball request —
cache hits included — and loads the package's full packument through
FetchOrCacheMetadata, unmarshalling the entire body to read one entry of the
time map. For packages like typescript or @swc/core that is a multi-megabyte
buffer and parse per tarball, and with an S3 storage backend it is also one
storage GET per tarball.

Under a heavy load (bursts of several hundred concurrent requests) this
dominates the proxy's cost: I measured ~12-16ms of CPU per request with cooldown on versus
well under 1ms with it off, install fetch times of ~100s that dropped to ~9s
when cooldown was disabled, and higher memory usage with it enabled, causing OOM kills.

A version's publish time is immutable, so the check now reads the stored
versions.published_at first and only falls back to the packument for a
version the proxy has never seen, persisting the parsed time so the packument
is fetched and parsed at most once per version. Same shape as the PyPI fix
from #242.

Changes:

  • versionInCooldown consults versions.published_at before fetching
    metadata, and persists the parsed time after the packument fallback
  • Add DB.SetVersionPublishedAt, an upsert that writes only the publish time
    so it never disturbs enrichment data on an existing row
  • UpsertVersion now preserves a stored published_at when the incoming
    value is NULL (COALESCE, both dialects). Without this, caching the
    artifact right after the check upserts the versions row without a publish
    time and erases the value the same request just stored; it also means a
    null-bearing upsert can no longer erase enrichment data
  • Tests: downloads with a stored publish time never touch the metadata
    upstream (served and withheld cases); two downloads of an uncached version
    fetch the packument exactly once, proving the stored time survives the
    artifact-cache upsert; database test for preserve-on-NULL and
    update-on-value in both dialects

go test ./..., go fmt, go vet clean.

Companion PR: #297 (serving publish times without enabling cooldown).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@VictorCodesseira
Read the stored publish time in the npm cooldown download check
5acbfcf
- Consult versions.published_at before fetching the packument, and persist
  the parsed time after the packument fallback, so each version's metadata
  is fetched and parsed at most once
- Add DB.SetVersionPublishedAt, an upsert that writes only the publish time
- Preserve a stored published_at in UpsertVersion when the incoming value
  is NULL, so the artifact-cache upsert cannot erase it
- Add handler tests for stored-time downloads and single-fetch behavior,
  and a database test for preserve-on-NULL in both dialects
@VictorCodesseira VictorCodesseira mentioned this pull request Sep 1, 2026
npm: add upstream.npm_full_metadata to serve publish times without cooldown #297
Merged
@VictorCodesseira
VictorCodesseira marked this pull request as ready for review September 1, 2026 18:53
andrew
andrew approved these changes Sep 2, 2026
View reviewed changes

@andrew andrew left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice — same shape as the PyPI fix in #242, and the COALESCE on UpsertVersion is a strict improvement (only caller is updateCacheDB, which never sets PublishedAt). Thanks for the measurements.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@andrew
andrew merged commit 76fcd07 into git-pkgs:main Sep 2, 2026
5 checks passed

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrew andrew andrew approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@VictorCodesseira @andrew
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.