data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

python data extract data-warehouse data-engineering data-lake load transform elt data-loading

• Updated Nov 12, 2026
• Python

WarpDrive is the lightweight data library for ambitious web apps — universal, typed, reactive, and ready to scale.

fetch ember vuejs query typescript spa reactjs restful signals mpa sveltejs data-framework reactive-js data-loading data-patterns local-first fine-grained-reactivity warp-drive

• Updated Nov 12, 2026
• TypeScript

TypeScript/JavaScript client library for Convex

javascript typescript database websocket realtime-database convex data-loading

• Updated Nov 12, 2026
• TypeScript

Fast, flexible, elastically deterministic data loading for foundation model training

deep-learning pytorch deterministic data-loader distributed-training multimodal data-loading elastic-training megatron-lm llm-training torchtitan

• Updated Nov 11, 2026
• Python

Accelerating AI Training and Inference from Storage Perspective (Must-read Papers on Storage for AI)

deep-learning tensorflow data-storage pytorch checkpoint dataloader data-preprocessing data-preparation storage-system model-training data-loading mlsys model-storage model-inference storage-for-ai

• Updated Nov 11, 2026

Podium: a framework agnostic Python NLP library for data loading and preprocessing

python nlp natural-language-processing datasets preprocessing data-loading

• Updated Dec 12, 2022
• Python

Lightweight data loading and caching library for android

android kotlin lightweight opensource cache annotations kotlin-android android-library android-development annotation-processor cache-storage kapt caching-library coldstorage data-loading viewbi

• Updated Feb 20, 2020
• Kotlin

Reusable data engineering toolkit My personal data infrastructure

infrastructure performance data-engineering preprocessing data-engineering-pipeline data-loading

• Updated Oct 29, 2025
• Jupyter Notebook

Local-first federated analytics query engine using DuckDB.

query federated federated-query data-loading data-load duckdb

• Updated Jan 30, 2025
• Go

Contains all my data science projects.

python data-science numpy pandas-dataframe linear-regression pandas python3 dimensionality-reduction matplotlib outlier-detection feature-engineering data-cleaning hyperparameter-tuning kfold-cross-validation gridsearchcv pandas-python colab-notebook data-loading matplotlib-python numpy-array

• Updated May 1, 2020
• Jupyter Notebook

This repository is a working ETL framework which utilizes user data from Spotify API using ➲Python for Extraction and Transformation ➲SQL for Data Loading and Staging ➲Airflow for Data Orchestration and Monitoring ➲PowerBI for Reporting

scheduling data-transformation data-visualization orchestration api-call etl-pipeline data-loading data-modelling

• Updated Apr 16, 2023
• Python

High-level API for tar-based dataset

tar tensorflow-datasets data-loading webdataset huggingface-datasets torchdata

• Updated Feb 3, 2024
• Python

A fast and easy-to-use Remote sensing Image format COnverter for High-throughput Deep-Learning (rico-hdl).

converter deep-learning remote-sensing high-throughput hydro data-loading eurosat bigearthnet hyspecnet-11k bigearthnet-s1 bigearthnet-s2 bigearthnet-mm uc-merced ssl4eo-s12 major-tom-core

• Updated May 7, 2025
• Python

Very fast loader for libsvm / svmlight sparse data files in Python (Cython): SciPy sparse matrices, gzip and bz2 input, an alternative to sklearn's load_svmlight_file.

python fast machine-learning cython scikit-learn loader sparse-data scipy libsvm-format sparse-matrix libsvm svmlight data-loading

• Updated Oct 20, 2026
• Python

Fast, accurate streaming of training data from cloud storage for PyTorch. Community-maintained fork of MosaicML Streaming.

python machine-learning streaming deep-learning cloud-storage pytorch dataset distributed-training data-loading

• Updated Nov 6, 2026
• Python

An Android library for data load with cache and loading state

android rxjava2 disklrucache lru-cache invalidation lrucache data-cache data-loading ui-status

• Updated May 11, 2026
• Kotlin

Data loading with combined async Rust stream and Python

structured-concurrency data-loading

• Updated Dec 14, 2023
• Rust

Aggregate multiple Google Drive accounts into one pool with automatic uploads to the least-used account for expanded free cloud storage.

fetch ember go http crawler query typescript storage samba ethereum restful signals data-loading data-patterns driverstore

• Updated Oct 14, 2026
• TypeScript

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

etl data-warehouse data-engineering data-lake warehouse dataloader warehouse-management data-pipeline datalake dlt data-ingestion etl-framework data-loading data-load duckdb datalake-etl data-loading-framework data-loaders dlt-destination

• Updated Nov 12, 2026
• Python

Syracuse University, Masters of Applied Data Science - IST 722 Data Warehouse

sql big-data power-bi sql-server-database data-warehouse business-intelligence extract-transform-load etl-framework data-mart sql-server-management-studio data-loading rolap data-warehouse-architecture dimensional-modeling kimball-lifecycle etl-patterns tdwi-mdm-report microsoft-ssis microsoft-ssas operational-data-store

• Updated Mar 24, 2020
• TSQL