GeezerVison is showing github.com — Short term index cache · Issue #72 · python/pymanager
Short term index cache #72
Description
We should probably cache the index locally for a short period of time (e.g. 5 minutes), to more efficiently handle rapid queries.
The period should be short enough to not really worry about other invalidation mechanisms - we can just skip the network traffic entirely. Passing -f should skip it, of course.
Activity
AA-Turner commented on May 6, 2025
If we add a cache we could reasonably consider making it longer than 5 minutes, as we make releases on the order of months. I'm maybe biased though by the constant 40MB conda index downloads!
A
zooba commented on May 7, 2025
The thing about a longer cache is then we need to do network calls to check whether to invalidate it. For a very short one, we're really just trying to catch a flurry of calls (e.g. a user doing py list --online ... a few times then a py install), and so we can get away with only looking at the local timestamp.
zooba commented on May 7, 2025
I'm maybe biased though by the constant 40MB conda index downloads!
And yeah, I'm all too familiar with these. It's why I set up the index chaining system like I did. I just repartitioned (#5), so now we have:
Wrote index-windows.json (36 entries, 27461 bytes)
Wrote index-windows-recent.json (225 entries, 170621 bytes)
Wrote index-windows-legacy.json (160 entries, 254675 bytes)
Most people just doing installs/updates will only get the first one (27KB). You only see the rest if you ask for a specific version that isn't the latest. New releases will always go into the first, so eventually it'll bloat, but we can repartition whenever we want to balance them again.
added a commit that references this issue
on May 16, 2025This issue will close once commit c835a4b is merged into the 'main' branch.c835a4b
Metadata
Metadata
Development
No branches or pull requestsIssue actions