Garnet is a remote cache-store from Microsoft Research that offers strong performance (throughput and latency), scalability, storage, recovery, cluster sharding, key migration, and replication features. Garnet can work with existing Redis clients.

scalable key-value cache cluster store remote persistent concurrent low-latency cache-storage hash-table larger-than-memory

• Updated Nov 12, 2026
• C#

Big-file data prep that never runs out of memory - an interactive shell and CLI, powered by DuckDB.

python shell cli data-science data csv etl pandas data-engineering parquet data-wrangling command-line-tool dataframe data-cleaning out-of-core duckdb larger-than-memory

• Updated Oct 30, 2026
• Python

A performant feature store, implemented in Rust 🦀 with a Fjall 🗻online store and DuckDB 🐤 query engine for offline workflows.

python rust machine-learning point-in-time apache-arrow mlops feature-store duckdb larger-than-memory fjall

• Updated Oct 2, 2026
• Rust

A Dask library for Big Data processing in Python demo

python insurance distributed-computing python3 distributed dask reinsurance dask-distributed larger-than-memory

• Updated Mar 21, 2024
• Python

A Bioinformatics demo in Python working with FASTQ files and using the Modin library

python bioinformatics computational-biology trie python3 biopython fastq modin larger-than-memory

• Updated Mar 21, 2024
• Python