GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GitHub - rstmdb/rstmdb: rstmdb is a deterministic, versioned state machine database with WAL-backed durability and TCP command protocol

BranchesTags
Open more actions menu

Latest commit

 

History

69 Commits
69 Commits

Folders and files

NameName
Last commit message
Last commit date
.githooks
.githooks
 
 
.github/workflows
.github/workflows
 
 
docs
docs
 
 
grafana
grafana
 
 
rstmdb-backup
rstmdb-backup
 
 
rstmdb-bench
rstmdb-bench
 
 
rstmdb-cli
rstmdb-cli
 
 
rstmdb-client
rstmdb-client
 
 
rstmdb-core
rstmdb-core
 
 
rstmdb-protocol
rstmdb-protocol
 
 
rstmdb-server
rstmdb-server
 
 
rstmdb-storage
rstmdb-storage
 
 
rstmdb-wal
rstmdb-wal
 
 
scripts
scripts
 
 
specs
specs
 
 
src
src
 
 
.dockerignore
.dockerignore
 
 
.gitignore
.gitignore
 
 
CONTRIBUTING.md
CONTRIBUTING.md
 
 
Cargo.lock
Cargo.lock
 
 
Cargo.toml
Cargo.toml
 
 
Dockerfile
Dockerfile
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
ROADMAP.md
ROADMAP.md
 
 
config.dev-tls.yaml
config.dev-tls.yaml
 
 
config.dev.yaml
config.dev.yaml
 
 
config.example.yaml
config.example.yaml
 
 
config.primary.yaml
config.primary.yaml
 
 
config.production.yaml
config.production.yaml
 
 
config.replica1.yaml
config.replica1.yaml
 
 
config.replica2.yaml
config.replica2.yaml
 
 
docker-compose.prod.yml
docker-compose.prod.yml
 
 
docker-compose.replication.yaml
docker-compose.replication.yaml
 
 
docker-compose.yml
docker-compose.yml
 
 
test_commands.sh
test_commands.sh
 
 

Repository files navigation

rstmdb

A state machine database with WAL durability and snapshot compaction.

Features

  • State Machine Engine - Define state machines with states, transitions, and guards
  • WAL Durability - Write-ahead log ensures no data loss on crash
  • Snapshot Compaction - Automatic snapshots and WAL cleanup
  • TCP Protocol (RCP) - Binary JSON protocol for high performance
  • TLS Support - Optional TLS encryption with mutual TLS (mTLS) support
  • Guard Expressions - Conditional transitions based on context
  • Idempotency - Safe retries with idempotency keys
  • Batch Operations - Execute multiple operations atomically
  • Event Streaming - Watch instance changes in real-time with WATCH_INSTANCE and WATCH_ALL

Quick Start

# Build
cargo build --release

# Run server
./target/release/rstmdb

# Or with config file
RSTMDB_CONFIG=config.yaml ./target/release/rstmdb

CLI Usage

# Ping server
rstmdb-cli ping

# Define a state machine
rstmdb-cli put-machine -n order -v 1 '{
  "states": ["created", "paid", "shipped"],
  "initial": "created",
  "transitions": [
    {"from": "created", "event": "PAY", "to": "paid"},
    {"from": "paid", "event": "SHIP", "to": "shipped", "guard": "ctx.ready"}
  ]
}'

# Create instance
rstmdb-cli create-instance -m order -V 1 -i order-001 -c '{"ready": false}'

# Apply event
rstmdb-cli apply-event -i order-001 -e PAY -p '{"amount": 99.99}'

# Get instance
rstmdb-cli get-instance order-001

# Compact WAL
rstmdb-cli compact --force

Event Streaming

rstmdb supports real-time event streaming for building event-driven architectures.

Watch a Specific Instance

Subscribe to state changes on a single instance:

# Watch instance and receive events
rstmdb-cli watch-instance order-001

# Output when state changes:
# {"type":"event","subscription_id":"sub-xxx","instance_id":"order-001",
#  "machine":"order","version":1,"from_state":"created","to_state":"paid",
#  "event":"PAY","wal_offset":42,"payload":{...},"ctx":{...}}

Watch All Events

Subscribe to all state changes across all instances:

# Watch all events
rstmdb-cli watch-all

# Filter by machine type
rstmdb-cli watch-all --machines order

# Filter by target state (e.g., only "shipped" transitions)
rstmdb-cli watch-all --to-states shipped

# Filter by event type
rstmdb-cli watch-all --events PAY,SHIP

# Combine filters
rstmdb-cli watch-all --machines order --to-states shipped

# Exclude context from events (smaller payloads)
rstmdb-cli watch-all --no-ctx

Use Cases

Notification Service:

# Watch for orders that ship, send customer notifications
rstmdb-cli watch-all --machines order --to-states shipped

Audit Log:

# Consume all events and write to external log
rstmdb-cli watch-all | jq -c >> audit.jsonl

CQRS Projection:

# Build a read model from state machine events
# Use a consumer that tracks the last processed wal_offset
rstmdb-cli watch-all --machines order | ./projection-builder

Unsubscribe

# Unsubscribe using subscription ID (when not using Ctrl+C)
rstmdb-cli unwatch sub-xxxxxxxx

Configuration

Config File (YAML)

# config.yaml
network:
  bind_addr: "127.0.0.1:7401"
  max_connections: 1000

auth:
  required: false
  # token_hashes:
  #   - "<sha256-hash>"  # Generate with: rstmdb-cli hash-token <token>

storage:
  data_dir: "./data"
  wal_segment_size_mb: 64
  fsync_policy: every_write

compaction:
  enabled: true
  events_threshold: 10000
  size_threshold_mb: 100

Authentication

When auth.required: true, clients must authenticate before executing commands.

# Generate a token hash for config
rstmdb-cli hash-token "my-secret-token"

# Use token with CLI
rstmdb-cli --token "my-secret-token" list-machines

# Or via environment variable
export RSTMDB_TOKEN="my-secret-token"
rstmdb-cli list-machines

Token hashes can be stored in config or loaded from an external secrets file:

auth:
  required: true
  secrets_file: "/etc/rstmdb/tokens.secret"  # One hash per line

TLS

Enable TLS encryption for secure communication:

# config.yaml
tls:
  enabled: true
  cert_path: "/path/to/server-cert.pem"
  key_path: "/path/to/server-key.pem"
  # For mutual TLS (mTLS):
  require_client_cert: true
  client_ca_path: "/path/to/client-ca.pem"

Generate development certificates:

./scripts/generate-dev-certs.sh ./dev-certs

CLI usage with TLS:

# Standard TLS (verify server certificate)
rstmdb-cli --tls --ca-cert ./dev-certs/ca-cert.pem -t my-token ping

# Insecure mode (skip verification - dev only)
rstmdb-cli --tls --insecure -t my-token ping

# Mutual TLS (client certificate required)
rstmdb-cli --tls \
    --ca-cert ./dev-certs/ca-cert.pem \
    --client-cert ./dev-certs/client-cert.pem \
    --client-key ./dev-certs/client-key.pem \
    -t my-token ping

Environment variables for TLS:

export RSTMDB_TLS=true
export RSTMDB_CA_CERT=/path/to/ca-cert.pem
export RSTMDB_CLIENT_CERT=/path/to/client-cert.pem
export RSTMDB_CLIENT_KEY=/path/to/client-key.pem

Environment Variables

Variable Default Description
RSTMDB_CONFIG - Path to YAML config file
RSTMDB_BIND 127.0.0.1:7401 Server bind address
RSTMDB_DATA ./data Data directory
RSTMDB_AUTH_REQUIRED false Require authentication
RSTMDB_AUTH_TOKEN_HASH - Single token hash
RSTMDB_AUTH_SECRETS_FILE - Path to secrets file
RSTMDB_COMPACT_EVENTS 10000 Auto-compact after N events
RSTMDB_COMPACT_SIZE_MB 100 Auto-compact when WAL > N MB
RSTMDB_TOKEN - CLI: Auth token for commands

Environment variables override config file values.

State Machine Definition

{
  "states": ["pending", "approved", "rejected"],
  "initial": "pending",
  "transitions": [
    {
      "from": "pending",
      "event": "APPROVE",
      "to": "approved",
      "guard": "ctx.amount <= 1000"
    },
    {
      "from": "pending",
      "event": "REJECT",
      "to": "rejected"
    }
  ]
}

Guard Expressions

Guards are boolean expressions evaluated against instance context:

ctx.amount <= 1000
ctx.approved && ctx.manager_id
ctx.items > 0
!ctx.cancelled

Monitoring

rstmdb exposes Prometheus metrics on port 9090 by default.

Grafana Dashboard

Grafana Dashboard

Import the official dashboard: rstmdb Dashboard #24778

Includes:

  • Request rate and latency (p50/p90/p99)
  • Error rates by code
  • Active connections and subscriptions
  • WAL size, throughput, and I/O operations
  • Instance and machine counts
  • System resources (CPU, memory, file descriptors)

Prometheus Configuration

scrape_configs:
  - job_name: 'rstmdb'
    static_configs:
      - targets: ['localhost:9090']

Architecture

┌─────────────────────────────────────────────┐
│                  Client                      │
└─────────────────┬───────────────────────────┘
                  │ RCP Protocol (TCP)
┌─────────────────▼───────────────────────────┐
│               Server                         │
│  ┌─────────────────────────────────────┐    │
│  │         Command Handler              │    │
│  └─────────────────┬───────────────────┘    │
│                    │                         │
│  ┌─────────────────▼───────────────────┐    │
│  │       State Machine Engine           │    │
│  │  • Definitions  • Instances          │    │
│  │  • Guards       • Transitions        │    │
│  └─────────────────┬───────────────────┘    │
│                    │                         │
│  ┌────────────┬────┴────┬─────────────┐    │
│  │    WAL     │ Snapshot │  Compaction │    │
│  │            │  Store   │   Manager   │    │
│  └────────────┴──────────┴─────────────┘    │
└─────────────────────────────────────────────┘

Data Directory Structure

data/
├── wal/
│   ├── 0000000000000001.wal
│   ├── 0000000000000002.wal
│   └── ...
└── snapshots/
    ├── index.json
    ├── snap-xxx.snap
    └── ...

Development

# Run tests
cargo test --workspace

# Run with logging
RUST_LOG=debug cargo run

# Format code
cargo fmt

# Lint
cargo clippy

Git Hooks

# Setup pre-commit hooks (fmt, clippy, tests)
./.githooks/setup.sh

About

rstmdb is a deterministic, versioned state machine database with WAL-backed durability and TCP command protocol

rstmdb.com

Topics

databasedeterministicdistributed-systemsevent-sourcinginfrastructureruststate-machinewalworkflow-engine

Resources

Readme
License

Contributing

Contributing
Activity
Custom properties

Stars

40 stars

Watchers

3 watching

Forks

2 forks
Report repository

Releases

Packages

Contributors

Languages

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.