The Patterns of Scalable, Reliable, and Performant Large-Scale Systems

computer-science lists devops distributed-systems machine-learning awesome web-development programming big-data system backend architecture scalability resources design-patterns interview awesome-list interview-practice interview-questions system-design

• Updated Jan 4, 2026

ClickHouse® is a real-time analytics database management system

rust embedded sql database big-data ai analytics cpp clickhouse dbms self-hosted distributed olap cloud-native mpp hacktoberfest lakehouse

• Updated Nov 12, 2026
• C++

Apache Spark - A unified analytics engine for large-scale data processing

python java r scala sql big-data spark jdbc

• Updated Nov 12, 2026
• Scala

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

python aws data-science machine-learning caffe theano big-data spark deep-learning hadoop tensorflow numpy scikit-learn keras pandas kaggle scipy matplotlib mapreduce

• Updated Mar 20, 2024
• Python

Apache Flink

python java scala sql big-data flink

• Updated Nov 12, 2026
• Java

All-in-one IoT Platform - Device management, data collection, processing and visualization.

visualization http mqtt iot middleware cloud microservices big-data snmp iot-platform dashboards iiot coap-server iot-framework thingsboard iot-analytics lwm2m-server iot-solutions

• Updated Nov 12, 2026
• Java

An open source cybersecurity protocol for syncing decentralized graph data.

machine-learning cryptography crypto encryption database big-data graph offline-first protocol end-to-end dapp decentralized blockchain realtime p2p artificial-intelligence crdt web3 metaverse dweb

• Updated Oct 15, 2026
• JavaScript

大数据入门指南 ⭐

phoenix scala kafka big-data spark yarn hive hadoop storm bigdata hbase zookeeper hdfs mapreduce flume azkaban sqoop

• Updated Jan 5, 2024
• Java

The official home of the Presto distributed SQL query engine for big data

java data query sql big-data presto hive hadoop lakehouse

• Updated Nov 12, 2026
• Java

The Data Engineering Cookbook

big-data best-practices cookbook data-engineering data-engineer

• Updated Oct 16, 2026
• Python

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

java distributed-systems data-science sql database big-data presto hive hadoop analytics jdbc databases distributed-database query-engine iceberg datalake prestodb trino delta-lake

• Updated Nov 12, 2026
• Java

PredictionIO, a machine learning server for developers and ML engineers.

scala big-data predictionio

• Updated Jan 9, 2021
• Scala

A distributed, fast open-source graph database featuring horizontal scalability and high availability

distributed-systems database big-data cpp graph raft scalability distributed knowledge-graph graph-database graphdb knowledge-base hacktoberfest nebula nebula-graph nebulagraph graphrag

• Updated Sep 8, 2026
• C++

Open-Source Web UI for Apache Kafka Management

opensource kafka big-data web-ui streams kafka-connect apache-kafka kafka-producer kafka-client kafka-streams hacktoberfest streaming-data kafka-manager kafka-cluster event-streaming cluster-management kafka-ui kafka-brokers

• Updated Jul 26, 2024
• Java

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

sql database big-data analytics olap join distributed-database realtime-database mpp cloudnative iceberg real-time-analytics datalake vectorized real-time-updates star-schema hudi delta-lake lakehouse lakehouse-platform

• Updated Nov 12, 2026
• Java

CMAK is a tool for managing Apache Kafka clusters

scala kafka big-data cluster-management

• Updated Aug 2, 2023
• Scala

Cloud-native OSS search engine for observability

rust open-source search-engine big-data logs cloud-storage cloud-native log-management distributed-tracing tantivy

• Updated Nov 11, 2026
• Rust

The most widely used Python to C compiler

python c performance big-data cpp cython cpython cpython-extensions

• Updated Nov 10, 2026
• Cython

Apache DataFusion SQL Query Engine

python rust sql big-data arrow olap query-engine dataframe datafusion

• Updated Nov 12, 2026
• Rust

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

python data-science machine-learning data-mining tutorial r big-data gpu cuda kaggle gbdt gbm gpu-computing decision-trees gradient-boosting coreml catboost categorical-features

• Updated Nov 12, 2026
• C++