SQLite-AI

On-device LLM inference and embeddings inside SQLite.
Run Llama, Phi, Qwen, and embedding models directly from SQL queries — local, private, no API calls. GGUF support, GPU acceleration on Metal/CUDA.

Free managed instance → · Docs · Website · Blog

_{Data:
Vector ·
Sync ·
Columnar ·
JS

AI:
AI ·
Agent ·
Memory ·
MCP}

Want hybrid local + cloud AI? SQLite-AI runs models on-device; combine with SQLite Cloud to sync embeddings, share agent memory, or offload heavy inference. Free tier available.

SQLite-AI

SQLite-AI is an extension for SQLite that brings artificial intelligence capabilities directly into the database. It enables developers to run, fine-tune, and serve AI models from within SQLite using simple SQL queries — ideal for on-device and edge applications where low-latency and offline inference are critical. The extension is actively developed by SQLite AI, some API and features are still evolving.

Features

Embedded AI Inference: Run transformer models directly from SQL queries.
Streaming I/O: Token-by-token streaming via SQL aggregate functions.
Fine-tuning & Embedding: On-device model customization and vector embedding.
Full On-Device Support: Works on iOS, Android, Linux, macOS, and Windows.
Offline-First: No server dependencies or internet connection required.
Composable SQL Interface: AI + relational logic in a single unified layer.
Audio Transcription: Speech-to-text via Whisper models (WAV, MP3, FLAC).
Vision / Multimodal: Analyze images via multimodal models (JPG, PNG, BMP, GIF).
Supports any GGUF model: available on Huggingface; Qwen, Gemma, Llama, DeepSeek and more

SQLite-AI supports text embedding generation for search and classification, a chat-like interface with history and token streaming, automatic context save and restore across sessions, audio transcription via Whisper models, and vision/multimodal image understanding — making it ideal for building conversational agents, memory-aware assistants, and voice-enabled applications.

Getting Started

# Start SQLite CLI
sqlite3 myapp.db

-- Load the extension
.load ./ai

Text Generation

-- Load a text generation model
SELECT llm_model_load('./models/Qwen2.5-3B-Q4_K_M.gguf', 'gpu_layers=99');
SELECT llm_context_create_textgen();

-- Generate text
SELECT llm_text_generate('What is the most beautiful city in Italy?');

Embedding Generation

-- Load an embedding model
SELECT llm_model_load('./models/nomic-embed-text-v1.5-Q8_0.gguf', 'gpu_layers=99');
SELECT llm_context_create_embedding('embedding_type=FLOAT32');

-- Generate an embedding vector
SELECT llm_embed_generate('Hello world');

-- Generate an embedding as JSON
SELECT llm_embed_generate('Hello world', 'json_output=1');

Chat

-- Load a chat model
SELECT llm_model_load('./models/Llama-3.2-3B-Instruct-Q4_K_M.gguf', 'gpu_layers=99');
SELECT llm_context_create_chat();

-- Send a message and get a complete response
SELECT llm_chat_respond('Tell me a joke.');

-- Or stream the reply token by token
SELECT reply FROM llm_chat('Tell me another joke.');

Audio Transcription

-- Load a Whisper model
SELECT audio_model_load('./models/ggml-tiny.bin');

-- Transcribe from a file path
SELECT audio_model_transcribe('./audio/speech.wav');

-- Transcribe with options
SELECT audio_model_transcribe('./audio/speech.mp3', 'language=it,translate=1');

-- Transcribe from a BLOB column
SELECT audio_model_transcribe(audio_data) FROM recordings WHERE id = 1;

Vision / Multimodal

-- Load a multimodal model and its vision projector
SELECT llm_model_load('./models/Gemma-3-4B-IT-Q4_K_M.gguf', 'gpu_layers=99');
SELECT llm_context_create_textgen();
SELECT llm_vision_load('./models/mmproj-Gemma-3-4B-IT-f16.gguf');

-- Describe an image
SELECT llm_text_generate('Describe this image', './photos/cat.jpg');

-- Use vision in a chat conversation
SELECT llm_context_create_chat();
SELECT llm_chat_respond('What do you see in this photo?', './photos/landscape.jpg');

-- Analyze multiple images
SELECT llm_text_generate('Compare these two images', './img1.jpg', './img2.jpg');

Documentation

For detailed information on all available functions, their parameters, and examples, refer to the comprehensive API Reference.

Installation

Pre-built Binaries

Download the appropriate pre-built binary for your platform from the official Releases page:

Linux: x86 and ARM
macOS: x86 and ARM
Windows: x86
Android
iOS

Loading the Extension

-- In SQLite CLI
.load ./ai

-- In SQL
SELECT load_extension('./ai');

Swift Package

You can add this repository as a package dependency to your Swift project. After adding the package, you'll need to set up SQLite with extension loading by following steps 4 and 5 of this guide.

Here's an example of how to use the package:

import ai

...

var db: OpaquePointer?
sqlite3_open(":memory:", &db)
sqlite3_enable_load_extension(db, 1)
var errMsg: UnsafeMutablePointer<Int8>? = nil
sqlite3_load_extension(db, ai.path, nil, &errMsg)
var stmt: OpaquePointer?
sqlite3_prepare_v2(db, "SELECT ai_version()", -1, &stmt, nil)
defer { sqlite3_finalize(stmt) }
sqlite3_step(stmt)
log("ai_version(): \(String(cString: sqlite3_column_text(stmt, 0)))")
sqlite3_close(db)

Android Package

Add the following to your Gradle dependencies:

implementation 'ai.sqlite:ai:0.7.55'

Here's an example of how to use the package:

SQLiteCustomExtension aiExtension = new SQLiteCustomExtension(getApplicationInfo().nativeLibraryDir + "/ai", null);
SQLiteDatabaseConfiguration config = new SQLiteDatabaseConfiguration(
    getCacheDir().getPath() + "/ai_test.db",
    SQLiteDatabase.CREATE_IF_NECESSARY | SQLiteDatabase.OPEN_READWRITE,
    Collections.emptyList(),
    Collections.emptyList(),
    Collections.singletonList(aiExtension)
);
SQLiteDatabase db = SQLiteDatabase.openDatabase(config, null, null);

Note: Additional settings and configuration are required for a complete setup. For full implementation details, see the complete Android example.

Python Package

Python developers can quickly get started using the ready-to-use sqlite-ai package available on PyPI:

pip install sqlite-ai

For usage details and examples, see the Python package documentation.

Flutter Package

Add the sqlite_ai package to your project:

flutter pub add sqlite_ai  # Flutter projects
dart pub add sqlite_ai     # Dart projects

Usage with sqlite3 package:

import 'package:sqlite3/sqlite3.dart';
import 'package:sqlite_ai/sqlite_ai.dart';

sqlite3.loadSqliteAiExtension();
final db = sqlite3.openInMemory();
print(db.select('SELECT ai_version()'));

For a complete example, see the Flutter example.

☁️ Hosted version

Need to coordinate AI across devices, fleets, or agents? SQLite Cloud is the managed backend for SQLite-AI — sync embeddings, share memory, and offload heavier inference when the device can't keep up.

Start free →

Part of the SQLite AI stack

SQLite-AI is one piece of a larger ecosystem that turns SQLite into a runtime for intelligent, distributed data:

Data layer

sqlite-vector — ANN vector search inside SQLite
sqlite-sync — Offline-first CRDT sync across devices
sqlite-columnar — Column-oriented analytics for OLAP queries
sqlite-js — Custom SQLite functions written in JavaScript

AI layer

sqlite-ai — On-device LLM inference and embeddings (you are here)
sqlite-agent — Autonomous AI agents running inside SQLite
sqlite-memory — Persistent, searchable memory for agents
sqlite-mcp — Call MCP tools directly from SQL queries

Managed platform

SQLite Cloud — Hosted SQLite with sync, auth, edge functions, and analytics. Free tier →

Built by SQLite AI. Questions? Contact us.

GeezerVison

Latest commit

History

Folders and files

Repository files navigation

SQLite-AI

SQLite-AI

Features

Getting Started

Text Generation

Embedding Generation

Chat

Audio Transcription

Vision / Multimodal

Documentation

Installation

Pre-built Binaries

Loading the Extension

Swift Package

Android Package

Python Package

Flutter Package

☁️ Hosted version

Part of the SQLite AI stack

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages