NCRF++, a Neural Sequence Labeling Toolkit. Easy use to any sequence labeling tasks (e.g. NER, POS, Segmentation). It includes character LSTM/CNN, word LSTM/CNN and softmax/CRF components.

natural-language-processing crf cnn pytorch artificial-intelligence lstm named-entity-recognition neural-networks chunking ner char-rnn part-of-speech-tagger sequence-labeling nbest lstm-crf char-cnn

• Updated Jun 30, 2022
• Python

Content-Addressable Data Synchronization Tool

http synchronization download tar upload archive file-system chunking delivery

• Updated Sep 22, 2025
• C

A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.

python nlp text splitting chunking text-chunking text-splitting semantic-chunking isaacus

• Updated Jun 13, 2026
• Python

Alternative casync implementation

golang synchronization archive chunking casync

• Updated Nov 10, 2026
• Go

An extensible Java framework for building event-driven applications that break up XML and non-XML data into chunks for data integration

java etl analytics xml pipelines sax stream-processing event-driven chunking smooks enterprise-integration

• Updated Nov 24, 2025
• Java

Fully neural approach for text chunking

ai ml chunking rag text-splitter llms semantic-chunking

• Updated Oct 23, 2025
• Python

Adaptive Chunking: automatically select the best chunking method per document for RAG. Accepted at LREC 2026.

nlp information-retrieval chunking rag llm text-splitting

• Updated Jul 6, 2026
• Python

The RAG Experiment Accelerator is a versatile tool designed to expedite and facilitate the process of conducting experiments and evaluations using Azure Cognitive Search and RAG pattern.

experiment information-retrieval azure evaluation indexing openai sparse vectors chunking acs embedding dense rag llm genai

• Updated Nov 9, 2026
• Python

A new chunking strategy developed by ZeroEntropy for general semantic chunking using Llama-70B.

retrieval chunking llm

• Updated Jan 28, 2025
• Python

Visual document analysis studio powered by Docling — configure the extraction pipeline, inspect text, tables and bounding boxes in the browser, then chunk, embed and index into OpenSearch and Neo4j.

ocr vue neo4j self-hosted embeddings knowledge-graph chunking opensearch table-extraction rag fastapi vector-search pdf-extraction document-ai document-parsing docling

• Updated Nov 11, 2026
• Python

A package for parsing PDFs and analyzing their content using LLMs.

nlp ocr chunking document-analysis pdf-parser pdfparser rag llm text-chunking

• Updated Aug 6, 2024
• Python

smart-llm-loader is a lightweight yet powerful Python package that transforms any document into LLM-ready chunks. Spend less time on preprocessing headaches and more time building what matters. From RAG systems to chatbots to document Q&A, SmartLLMLoader handles the heavy lifting so you can focus on creating exceptional AI applications.

markdown chatbot pdf-converter gemini openai chunking pdf-parser claude rag langchain llama-index pdf-to-markdown

• Updated Aug 3, 2026
• Python

A TensorFlow implementation of Neural Sequence Labeling model, which is able to tackle sequence labeling tasks such as POS Tagging, Chunking, NER, Punctuation Restoration and etc.

tensorflow python3 named-entity-recognition chunking punctuation sequence-labeling pos-tagger sentence-boundary-detection lstm-networks

• Updated Nov 27, 2018
• Python

A local, open-source visual workspace for RAG document preparation: inspect PDF conversion, refine Markdown, compare chunking strategies, and enrich chunks before indexing.

markdown chunking document-processing document-extraction rag pdf-processing chunking-algorithm text-splitter llms langchain retrieval-augmented-generation rag-evaluation rag-pipeline pdf-to-markdown chonkie docling document-chunking semantic-chunker chunk-validation rag-chunk

• Updated Nov 9, 2026
• Python

Pure Rust PDF library for AI/RAG: structure-aware chunking, no ML, no C deps.

rust pdf ocr encryption ai text-extraction embeddings data-extraction invoice chunking pdf-generation pdf-manipulation pdfa table-extraction pdf-parser digital-signatures document-processing rag langchain llamaindex

• Updated Nov 11, 2026
• Rust

PDFStract - Extract, Chunking and Embedding Layer in Your RAG Pipeline - Available as CLI - WEBUI - API

pdf ocr ai knowledgebase data-extraction chunking dataengineering unstructured rag rag-pipeline docling pdfconversion

• Updated Mar 18, 2026
• Python

🍱 Semantically create chunks from large document for passing to LLM workflows

vector embeddings chunking text-splitter llm text-chunking text-splitting semantic-chunking equill-library

• Updated Aug 14, 2026
• JavaScript

A Python CLI to test, benchmark, and find the best RAG chunking strategy for your Markdown documents.

python nlp ia chunking rag vector-search embedding-vectors llm langchain retrieval-augmented-generation text-splitting rag-pipeline document-chunking

• Updated Jan 18, 2026
• Python

Deprecated/unmaintained Postgres extensions to support end-to-end Retrieval-Augmented Generation (RAG) pipelines

postgresql embeddings chunking rag pgrx

• Updated Jun 15, 2026
• Rust

Live TS segmenter and HLS manifest creation in Go

golang video hls chunk chunking transport-stream chunked lhls

• Updated Oct 31, 2021
• Go