Scalable data pre processing and curation toolkit for LLMs

python data data-processing data-preparation deduplication data-quality data-curation data-prep fine-tuning fast-data-processing data-processing-pipelines datacuration large-language-models llm llmapps large-scale-data-processing datarecipes semantic-deduplication llm-data-quality

• Updated Nov 11, 2026
• Python

Open source project for data preparation for GenAI applications

python data spark malware code-quality data-preprocessing ray data-preparation deduplication data-prep finetuning data-preprocessing-pipelines datacuration large-language-models llm llmapps large-scale-data-processing datarecipes

• Updated Sep 29, 2026
• HTML

99+ CLI tools to build, browse, and blend your media library

music cli files playlist ffmpeg command-line mpv sqlite media videos folders data-collection ffprobe curation qbittorrent-nox datasette-tool yt-dlp gallery-dl datacuration broadcatching

• Updated Nov 10, 2026
• Python

Wikidata-based scholarly profiles

latex dataviz bibtex bibliography sparql wikidata citations literature scientometrics code4lib hacktoberfest linked-open-data bibliometrics wikicite fairdata datacuration

• Updated Nov 12, 2026
• JavaScript

🐍👨‍💻This introductory🐍 notebook is designed for beginners who are📘starting their journey with📘Python, especially in the context of data science and data handling. As I am currently learning Data Curation, this will be an easy, readable file for good practice and learning.👨‍💻🐍

python nlp science data machine-learning guide numpy pandas matplotlib deeplearning beginner-friendly datacuration

• Updated May 20, 2026
• Jupyter Notebook

An image + data web scraper build to crawl the CarMax website and store relevant information for vehicle identification projects.

python selenium selenium-webdriver webcrawler webscraping datacuration

• Updated Dec 4, 2021
• Python

Data Lorry Nigeria is a comprehensive open-data platform aggregating Nigerian socioeconomic, political, and demographic datasets into a single, accessible resource for researchers, journalists, policymakers, and civic technologists.

devops data-science nigeria-api datacuration

• Updated Oct 18, 2026
• TypeScript

Exploration and data curation of a dataset given by a Kaggle competition (https://www.kaggle.com/dansbecker/melbourne-housing-snapshot) related to properties that were sold in Melbourne in 2016 and 2017. The meaning of this project is to prepare a well-structured matrix, so it can be used to run a model in order to estimate their prices.

price estimation properties melbourne exploration machinelearning datacuration

• Updated Jul 1, 2021
• Jupyter Notebook

This is a capstone project for the course of Business Analytics and Business Data Management at IIT Madras. The project involves analyzing sales data of Uttam Supermarket in Indore, which has 5 franchises, collected over a year. The analysis includes store-wise and monthly sales, the effect of holidays on sales, and weekly sales analysis.

data analysis excel powerpoint sheets business-analytics datacleaning conclusions datacuration

• Updated Dec 14, 2022

Acest repo conține materiale, seturi de date și soluții care au fost folosite în cadrul Școlii de vară Astra, prima ediție, 2021

javascript metadata openrefine jq datawrangling catmandu datacuration

• Updated Sep 7, 2021

SpiralDB connectors for NVIDIA NeMo Curator

python data computer-vision vortex data-processing spiral data-preparation deduplication data-quality data-curation data-prep multimodal fast-data-processing nvidia-nemo datacuration datarecipes multimodal-ai physical-ai

• Updated Oct 28, 2026
• Python