Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

python rust data-science machine-learning computer-vision deep-learning embeddings dataops data-analytics data-analysis data-format apache-arrow data-centric mlops duckdb llms

• Updated Nov 12, 2026
• Rust

Redpanda Console is a developer-friendly UI for managing your Kafka/Redpanda workloads. Console gives you a simple, interactive approach for gaining visibility into your topics, masking data, managing consumer groups, and exploring real-time data with time-travel debugging.

react go typescript kafka web-ui dataops apache-kafka kafka-ui kafka-gui

• Updated Nov 12, 2026
• TypeScript

Scalable and efficient data transformation framework - backwards compatible with dbt.

python sql etl dataops dbt elt transformation dataengineering

• Updated Nov 12, 2026
• Python

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

python data-science machine-learning analytics logging constraints dataset dataops data-pipeline data-quality calculate-statistics data-constraints mlops model-performance ml-pipelines ai-pipelines approximate-statistics statistical-properties

• Updated Jan 10, 2025
• Jupyter Notebook

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

open-source tap data opensource integration pipelines data-engineering target dataops loaders elt extract-data data-pipelines singer connectors targets taps meltano dataops-platform meltano-sdk

• Updated Nov 12, 2026
• Python

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.

bigquery snowflake data-warehouse dataops data-analysis redshift dbt data-pipelines data-pipeline lineage data-governance data-lineage analytics-engineer dbt-packages data-observability data-reliability dbt-artifacts

• Updated Nov 5, 2026
• HTML

Kafka Docker for development. Kafka, Zookeeper, Schema Registry, Kafka-Connect, , 20+ connectors

docker kafka schema-registry dataops kafka-rest-proxy

• Updated Sep 10, 2026
• Shell

Cloud Native DataOps & AIOps Platform | 云原生数智运维平台

kubernetes engineering application devops ops saas dataops maintenance k8s sre flink cloudnative operation aiops oam

• Updated Dec 13, 2025
• Java

Support agile Ontology DataOps Based on Flink, DataX and Flink-CDC with Web-UI

java etl ontology dataops flink cdc datax chatbi

• Updated Nov 12, 2026
• Java

📙 Awesome Data Catalogs and Observability Platforms.

open-source metadata awesome opensource oss big-data opendata ml data-engineering dataops data-catalog data-discovery awesome-list observability data-quality metadata-management datacatalog datadiscovery

• Updated Aug 14, 2025

Tenzir is the data pipeline engine for security teams.

security netflow pcap incident-response pipelines dataops suricata siem sigma soc hacktoberfest zeek investigation threathunting secdataops

• Updated Nov 12, 2026
• C++

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

golang bigquery airflow automation etl analytics data-transformation data-warehouse business-intelligence dataops elt workflows data-pipelines data-modelling analytics-engineering

• Updated Jun 8, 2024
• Go

DataOps for Microsoft Data Platform technologies. https://aka.ms/dataops-repo

devops data fabric azure dataops cicd databricks datafactory automatedtesting

• Updated Feb 25, 2026
• Shell

A list of tools for annotating data, managing annotations, etc.

nlp awesome computer-vision image-annotation annotations dataops awesome-list data-annotation

• Updated Aug 1, 2024

Engine for AI/ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.

tracking data-science statistics spark tensorflow plotly pandas data-visualization pytorch dataops matplotlib dask data-exploration pandas-summary dataframes data-quality-checks data-quality data-profiling explainable-ai mlops

• Updated Nov 11, 2026
• Python

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL.

python data-science data sql database pipeline etl analytics snowflake data-warehouse data-structures data-engineering dataops warehouse elt data-pipelines data-engineer trino data-lineage data-engineering-pipeline

• Updated Sep 1, 2026
• Python

Titan Core - Snowflake infrastructure-as-code. Provision environments, automate deploys, CI/CD. Manage RBAC, users, roles, and data access. Declarative Python Resource API. Change Management tool for the Snowflake data warehouse.

devops snowflake data-warehouse secops data-engineering dataops rbac data-governance compliance-as-code

• Updated Mar 13, 2025
• Python

The data-validation toolkit for enhanced dbt (data build tool) PR review

data data-validation data-engineering dataops dbt elt analytics-engineering

• Updated Nov 12, 2026
• TypeScript

Power BI DevOps & Source Control Tool

devops power-bi dataops source-control pbix

• Updated Jan 30, 2026
• C#

Open data platform based on Kubernetes. Scaleph supports SeaTunnel、Flink and Doris backended by SeaTunnel on Flink engine、Flink Kubernetes Operator and Doris operator.

dataops data-platform flink dag doris flink-sql seatunnel flink-sql-gateway flink-kubernetes flink-kubernetes-operator doris-operator doris-manager

• Updated Sep 13, 2026
• Java