GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Add Chef Supermarket proxy support for Berkshelf and Policyfiles by abhinavgautam01 · Pull Request #518 · git-pkgs/proxy

Conversation

@abhinavgautam01

@abhinavgautam01 abhinavgautam01 commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

Closes #264

Adds read-only caching proxy support for Chef Supermarket at /chef, so Berkshelf and Policyfiles can resolve and download cookbooks through the proxy. Chef moves to ✓ in the supported registries table.

Usage

# Berksfile
source "http://localhost:8080/chef"

# Policyfile.rb
default_source :supermarket, "http://localhost:8080/chef"

A private Supermarket is configured with upstream.chef (or PROXY_UPSTREAM_CHEF), which defaults to https://supermarket.chef.io. Credentials go under upstream.auth, scoped to its URL.

How it works

The protocol was checked against the live supermarket.chef.io API:

Route Behavior
GET /chef/universe Rewrites each version's location_path and download_url to the proxy. Dependencies and all other fields are kept byte for byte.
GET /chef/api/v1/cookbooks/{name} Rewrites latest_version and versions[]
GET /chef/api/v1/cookbooks/{name}/versions/{version} Rewrites cookbook and file. Also accepts 7_5_22, the form Berkshelf uses in URLs.
GET /chef/api/v1/cookbooks/{name}/versions/{version}/download Follows Supermarket's 302 to its storage host and caches the archive as pkg:chef/{name}@{version}
  • Metadata goes through the existing metadata cache, rewrite cache and serveRewrittenMetadata, the same path npm and Composer use. It follows metadata_ttl, falls back to the stored copy when the upstream fails and supports ETag/304. The rewritten universe (about 7.6 MB) is cached rather than rebuilt on every request.
  • Downloads are served from the artifact cache once cached, without contacting the upstream. Denylisted versions are refused, as for every artifact.
  • Dependencies stay unchanged. JSON is re-encoded without HTML escaping, so constraints such as >= 0.0.0 keep their spelling rather than becoming \u003e= 0.0.0.
  • Unreadable upstream documents return 502 instead of being passed through, since passing them through would send clients around the proxy.
  • Cache keys include a hash of the upstream URL, so pointing upstream.chef at another Supermarket never serves the previous one's documents.
  • Authentication uses the existing upstream.auth transport. Because credentials are scoped by URL, the storage host behind the download redirect never receives them.
  • Input validation: cookbook names and versions are validated before any upstream URL is built from them.

Wiring

  • Config: upstream.chef with its default, env var and serve -help entry
  • Route mount in server.go, plus the chef label in request metrics
  • Dashboard: ecosystem filter pill, badge color and install instructions for Berkshelf and Policyfiles
  • Docs: README section and API endpoint row, config.example.yaml and docs/configuration.md with a private Supermarket auth example

Tests

New internal/handler/chef_test.go, against a fake Supermarket shaped like the real API, with downloads redirecting to a separate storage host:

  • universe rewriting, with every dependency set unchanged and constraints not re-escaped
  • cookbook and version metadata rewriting, with unrelated fields unchanged
  • both version spellings (7.5.22 and 7_5_22), fetched from upstream once
  • the Berkshelf flow (universe, then version at location_path, then file download), covering the redirect, a cache hit and offline reads of both the cookbook and the universe
  • upstream authentication, including the storage host not receiving the token
  • 400 for invalid names or versions, 404 for unknown cookbooks, 502 for an unreadable universe

Chef is also added to the existing config, serve -help, route-label and missing-artifact 404 tests. Removing the URL rewrite, the underscore handling or the credentials each makes a test fail.

Live check against supermarket.chef.io

Using proxy serve:

  • /chef/universe returned 4,074 cookbooks with every entry rewritten. Compared with the upstream universe, all 31,355 versions have identical dependencies and fields.
  • apt 7.5.22 downloaded through the redirect as a valid tarball. Its 24,561 bytes match Supermarket's tarball_file_size.

go build, go vet, golangci-lint (v2.13.1), go test -race ./... and Swagger regeneration all pass locally.

Not included

  • Storage retention for chef: each ecosystem has received retention in its own PR (Add per-ecosystem artifact retention (age-based eviction) #306). Chef archives are still limited by storage.max_size.
  • Cooldown: the universe carries no publish dates.
  • Hiding denylisted versions from metadata: like Helm, Conan and Swift, they are blocked at download only.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@abhinavgautam01
Add Chef Supermarket proxy support for Berkshelf and Policyfiles
23ed256
andrew
andrew requested changes Oct 18, 2026
View reviewed changes

@andrew andrew left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing upstream.chef can serve a cached cookbook archive from the previous Supermarket. Please scope archive cache identity to the upstream and add a regression test covering an upstream switch with the same cookbook name and version.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread internal/handler/chef.go Resolved

Uh oh!

There was an error while loading. Please reload this page.

@abhinavgautam01
Scope cached Chef cookbook archives to the upstream Supermarket
558e9ea
@abhinavgautam01

abhinavgautam01 commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, fixed. Archives are now cached under a filename that carries the upstream's key (apt-7.5.22-<key>.tgz), the same 16-character digest the metadata cache keys use, so switching upstream.chef refetches from the new Supermarket. The package and version records stay pkg:chef/[email protected], so the denylist, dashboard and retention are unaffected. TestChefArchiveCacheDependsOnUpstream covers the switch through the download route and it fails with the old filename.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

andrew
andrew approved these changes Oct 19, 2026
View reviewed changes

@andrew andrew left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The upstream cache isolation issue is fixed. Archive filenames now include the upstream identity, and the regression test covers switching upstreams and retrieving each cached archive.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@andrew
andrew merged commit bf0c784 into git-pkgs:main Oct 19, 2026
6 checks passed

Uh oh!

There was an error while loading. Please reload this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrew andrew andrew approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add Chef Supermarket proxy support

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@abhinavgautam01 @andrew
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.