GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com β€” GitHub - KipData/KiteSQL: SQL as a Function for Rust β€” an embedded SQL database with a typed ORM and pluggable LMDB/RocksDB storage.

BranchesTags
Open more actions menu

Latest commit

 

History

352 Commits
352 Commits

Folders and files

NameName
Last commit message
Last commit date
.cargo
.cargo
 
 
.devin
.devin
 
 
.github
.github
 
 
assets
assets
 
 
benchmarks
benchmarks
 
 
docs
docs
 
 
examples
examples
 
 
fuzz
fuzz
 
 
kite_sql_serde_macros
kite_sql_serde_macros
 
 
rfc
rfc
 
 
scripts
scripts
 
 
src
src
 
 
static/images
static/images
 
 
tests
tests
 
 
tpcc
tpcc
 
 
.gitignore
.gitignore
 
 
.gitmodules
.gitmodules
 
 
AGENT.md
AGENT.md
 
 
Cargo.lock
Cargo.lock
 
 
Cargo.toml
Cargo.toml
 
 
Dockerfile
Dockerfile
 
 
LICENSE
LICENSE
 
 
Makefile
Makefile
 
 
README.md
README.md
 
 
codecov.yml
codecov.yml
 
 
deepwiki.json
deepwiki.json
 
 
rust-toolchain
rust-toolchain
 
 

Repository files navigation

KiteSQL Logo

SQL as a Function for Rust

 

CI Codecov Ask DeepWiki Discord github star github fork

Introduction

KiteSQL is a lightweight embedded relational database written in Rust, inspired by MyRocks and SQLite. It runs inside your application with no external service: execute SQL directly, or use typed ORM models, migrations, and builder-style queries.

  • Most of the SQL 2016 syntax
  • All metadata and data stored in KV storage: RocksDB, LMDB, in-memory, or SlateDB on S3
  • Zero-copy reads: on LMDB, strings go from the mmap'd file into your structs as Cow::Borrowed, without a copy (how)
  • Async API on object storage: the same engine as async, on SlateDB over Amazon S3 or an S3-compatible store such as MinIO or R2, see Async API on S3
  • Typed ORM with schema migration (orm feature), see src/orm/README.md
  • WebAssembly and Python bindings

πŸ‘‰ More features Β· Blog

πŸ“ Zero-Copy SQL Computation on LMDB (δΈ­ζ–‡): how strings travel from the mmap'd file to your struct without a copy, benchmarked against SQLite.

Example

use kite_sql::db::DataBaseBuilder;
use kite_sql::errors::DatabaseError;
use kite_sql::orm::OrmQueryResultExt;
use kite_sql::Model;

#[derive(Default, Debug, PartialEq, Model)]
#[model(table = "users")]
struct User {
    #[model(primary_key)]
    id: i32,
    #[model(varchar = 64)]
    name: String,
    #[model(default = "18", index)]
    age: Option<i32>,
}

fn main() -> Result<(), DatabaseError> {
    let mut database = DataBaseBuilder::path("./data").build_rocksdb()?;
    database.migrate::<User>()?;

    database.insert_many([
        User { id: 1, name: "Alice".to_string(), age: Some(18) },
        User { id: 2, name: "Bob".to_string(), age: Some(24) },
    ])?;

    let users = database
        .bind(|ctx| {
            ctx.from::<User>()?
                .filter(|e| e.column(User::age())?.gte(18))?
                .project_scalars((User::id(), User::name()))?
                .order_by(User::name())?
                .finish()
        })?
        .project_tuple::<(i32, String)>();
    for user in users {
        println!("{:?}", user?);
    }

    // Plain SQL works too.
    database.run("select count(*) from users")?.done()?;
    Ok(())
}

More: hello_world, transaction, program, borrowed_parse (zero-copy reads against SQLite).

Storage Backends

Builder Storage
build_rocksdb() RocksDB (default feature rocksdb), stronger for write-heavy workloads
build_lmdb() LMDB (feature lmdb), stronger for read-heavy workloads
build_in_memory() In-memory, for tests and temporary data
build_optimistic() RocksDB with optimistic transactions
build_with_storage(SlateDbStorage::open_s3(..)) SlateDB on S3 (feature slatedb-s3, async API), see Async API on S3

Feature flags, checkpoints, and transaction isolation: docs/features.md.

Shell, WebAssembly, Python

Shell

cargo run --bin kitesql-shell
cargo run --bin kitesql-shell -- -e "select 1"

Type .help for metacommands.

WebAssembly

wasm-pack build --release --target nodejs
import { WasmDatabase } from "./pkg/kite_sql.js";

const db = new WasmDatabase();
await db.ddl("create table demo(id int primary key, v int)");
console.log(db.run("select * from demo").rows());

Python

Requires the python feature.

import kite_sql

db = kite_sql.Database.in_memory()  # or Database(path, backend="lmdb")
db.execute("create table demo(id int primary key, v int)")
print(list(db.run("select * from demo")))

TPC-C

make tpcc runs the benchmark (--backend kitesql-lmdb|kitesql-rocksdb|sqlite); make tpcc-dual cross-checks every statement against SQLite.

720-second runs on an i9-13900HX (32 GB, KIOXIA EXCERIA PLUS G3), without fsync on commit. Latencies are p90 in Β΅s, including commit. SlateDB runs on a local S3 server (s3s-fs) with its block cache in memory and on local disk.

1 thread, 1 warehouse, every backend pinned to the same P-core:

Backend TpmC New-Order Payment Order-Status Delivery Stock-Level
KiteSQL LMDB 254023 211 38 25 293 175
KiteSQL RocksDB 47488 372 85 415 15471 389
KiteSQL SlateDB (S3) 12016 677 240 399 74047 1638
SQLite (WAL, synchronous=OFF) 214001 251 37 32 338 227

4 threads, 4 warehouses (one home warehouse per terminal), pinned to four P-cores:

Backend TpmC New-Order Payment Order-Status Delivery Stock-Level
KiteSQL LMDB 240584 930 780 29 890 207
KiteSQL RocksDB 212936 505 109 285 8991 400
KiteSQL SlateDB (S3) 19010 1749 1816 2481 66495 2523
SQLite (WAL, synchronous=OFF) 205896 254 41 45 336 380

πŸ‘‰ Details and how to reproduce

Roadmap

  • Get SQL 2016 mostly supported
  • LLVM JIT: Perf: TPCC

License

KiteSQL uses the Apache 2.0 license to strike a balance between open contributions and allowing you to use the software however you want.

Contributors

About

SQL as a Function for Rust β€” an embedded SQL database with a typed ORM and pluggable LMDB/RocksDB storage.

kitesql.kould2333.org/

Topics

asyncaws-s3datadatabaseembeddingsmyrocksoltpquery-enginerocksdbrustsqlsql-querysqlitesqlite-databasewebzero-copy

Resources

Readme
Apache-2.0 license
Activity
Custom properties

Stars

760 stars

Watchers

7 watching

Forks

55 forks
Report repository

Sponsor this project

Used by

Contributors

Languages

You are visitor 01402503 β€” GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800Γ—600. No cookies required.