OxideSwarm

Run builds and tests on the machines you already own.
One Rust binary turns your Macs, Windows PCs and Linux boxes into a compute grid that you and your coding agents can send work to: cargo test, scripts, compile jobs, file processing.

Quick start · Install · How it works · Benchmarks · Docs · Changelog

• A second machine for your test suite. oxideswarm submit --workspace myproject -- cargo test syncs your project to another machine and runs it there. The next run sends only the files that changed, and the remote target/ directory is reused, so Cargo builds incrementally.
• Made for coding agents. oxideswarm mcp is a Model Context Protocol server with the tools status, workers, submit, task_result, cancel, doctor and workspaces. Claude Code, Codex, Cursor and other MCP clients can submit tasks, wait for results and use workspaces (setup guide).
• No broker, no cloud account. One master and the workers you add talk over TCP on your LAN. Across networks they use iroh QUIC through NATs, so no router ports need opening.
• Authenticated and encrypted beyond loopback. A master that other machines can reach requires a shared cluster secret, and cluster traffic is encrypted (ChaCha20-Poly1305). On macOS, Linux and Windows, a worker started with --isolation fs limits what each task can write, and keeps the task out of your home directory and away from the secret.
• Easy to add a machine. Join a machine with a single token (oxideswarm join), or set it up from a USB stick: one double-click on macOS and Windows, or bash install.sh on Linux. oxideswarm doctor checks the whole cluster.

Each number links to the dated record of its run. The first two rows come from a run by hand, and the others from benchmarks/grid/bench_grid.py. Each record states its machines, its load and its limits.

Offload cargo test to a second machine.

Two machines on one network: on A you edit code (with or without a coding agent), B has Rust installed and does the work. Neither machine has to build OxideSwarm.

1. Install the release archive for each system on both machines (Install).

2. Start the master on A. The first start creates ~/.oxideswarm/cluster.secret:

   oxideswarm master --listen 0.0.0.0:8088

3. Join B with one token (design 0048). The token carries the secret: move it over a channel you trust (never chat or e-mail), and pipe it in rather than typing it as an argument:

   oxideswarm secret join-token --master <A_IP>:8088 > join.token   # on A
   oxideswarm join < join.token                                     # on B: saves the secret and the address, runs the doctor checks
   oxideswarm worker --workspaces                                   # on B

4. Check from A that B offers workspaces and Cargo: in oxideswarm workers --json, B's tags include feat:workspaces and tool:cargo.

5. Offload from the root of a Rust project on A:

   oxideswarm submit --workspace myproject --require-tool cargo --wait -- cargo test

   The first run sends every file that .gitignore does not exclude and builds from scratch. Run it again: only changed files travel, and B reuses its target/ directory.

To set up several machines, a USB stick can carry the program for every system and the token: see packaging/usb/README.md. The stick installs, joins and starts the worker with one double-click on macOS and Windows, or bash install.sh on Linux. With a token that carries the master's P2P ticket, it also works on school or office Wi-Fi where the machines cannot reach each other. The kit scripts are in the repository and install release 0.1.1 or newer (0.1.2 or newer for Windows workers that run workspace tasks under -Isolation fs).

A coding agent does step 5 through oxideswarm mcp (guides/mcp-client-setup.md): its submit tool takes the same workspace and tools fields. The full walk-through, with the expected output, the MCP calls and troubleshooting, is guides/offload-cargo-test.md, and measured times are in benchmarks/grid/README.md.

Download the archives from the Releases page (the latest is v0.1.2), or build from source.

Each release carries one archive per system, oxideswarm-<version>-<target>.tar.gz (Linux, macOS) or .zip (Windows), and a -gpu variant built with --features gpu-wgpu for a worker whose GPU should do work. Targets: x86_64-unknown-linux-gnu (built on Ubuntu 22.04, needs glibc 2.35 or newer), aarch64-apple-darwin, x86_64-apple-darwin, x86_64-pc-windows-msvc. An archive holds oxideswarm, rusty-grid, ox-mode, the licences and the installer scripts for that system under scripts/. SHA256SUMS lists every archive.

# macOS / Linux: verify, unpack, put the binaries on PATH (the file names below are an example)
shasum -a 256 -c SHA256SUMS --ignore-missing      # Linux: sha256sum -c SHA256SUMS --ignore-missing
tar -xzf oxideswarm-0.1.2-aarch64-apple-darwin.tar.gz
# The binaries are not signed or notarized: a browser download is quarantined and Gatekeeper refuses to run it.
xattr -dr com.apple.quarantine oxideswarm-0.1.2-aarch64-apple-darwin      # macOS only
sudo install -m 755 oxideswarm-0.1.2-aarch64-apple-darwin/{oxideswarm,rusty-grid,ox-mode} /usr/local/bin/
oxideswarm --help

# Windows: compare the hash with the SHA256SUMS line, unpack, add the folder to PATH
(Get-FileHash .\oxideswarm-0.1.2-x86_64-pc-windows-msvc.zip -Algorithm SHA256).Hash
Expand-Archive .\oxideswarm-0.1.2-x86_64-pc-windows-msvc.zip -DestinationPath C:\oxideswarm
$user = [Environment]::GetEnvironmentVariable('Path', 'User')
[Environment]::SetEnvironmentVariable('Path', "$user;C:\oxideswarm\oxideswarm-0.1.2-x86_64-pc-windows-msvc", 'User')   # then open a new terminal

Then continue with the Quick start or Set up a cluster. The background-service installers in scripts/ (install_mac_daemon.sh --bin <PATH>, install_windows_service.ps1) take the unpacked rusty-grid binary.

cargo build --release --bin oxideswarm --bin rusty-grid     # the two names are the same program
# Add --features gpu-wgpu on a machine whose GPU should do work (Apple Silicon Mac, Windows GPU PC);
# without it the worker cannot use the GPU and advertises none:
cargo build --release --bin oxideswarm --features gpu-wgpu

flowchart LR
    subgraph clients["You and your agents"]
        CLI["oxideswarm submit / status / doctor"]
        MCP["oxideswarm mcp<br/>(Claude Code, Codex, Cursor)"]
    end
    M["Master<br/>queue, scheduler, blob store<br/>TCP :8088 · dashboard :8080"]
    W1["Worker<br/>macOS"]
    W2["Worker<br/>Windows"]
    W3["Worker<br/>Linux"]
    CLI -- "authenticated, encrypted" --> M
    MCP --> M
    W1 -- "outbound TCP or iroh QUIC" --> M
    W2 --> M
    W3 --> M

• Master. One master holds the task queue.
  • A task goes from submitted to queued, scheduled and running, and ends completed, failed or cancelled. A failed attempt with retries left waits in retrying, then goes back to queued.
  • The master places each task on a worker that fits it: OS, architecture, tags, GPU, tools such as cargo, a workspace, and free slots.
  • When a worker is lost, its tasks are requeued and run elsewhere (run_chaos_test.sh checks this).
  • With --state-dir, the queue is kept in a journal, so a master restart loses no tasks (design 0046).
• Workers. Workers connect out to the master, so they need no open port. A worker runs tasks as its own OS user, with a timeout, bounded output and the whole process tree killed at the end. With --isolation fs, a task can write only its own sandbox and cannot read the cluster secret (design 0033). With --workspaces, a worker keeps named project trees between tasks (design 0039).
• Files. Inputs and outputs move as content-addressed blobs (BLAKE3). A blob that the other side already holds is not sent again (design 0006).
• Clients. oxideswarm submit, status, workers, result, cancel, doctor and the MCP server all speak the same protocol. Before they change anything, they check that the master runs a compatible version (design 0040).
• Network. On a LAN everything uses TCP port 8088. Across networks, master --p2p publishes an iroh ticket, and machines then connect directly through NAT or through a relay. That needs outbound Internet access to the public n0 relays, or to your own relay (--relay-url, guide).

Also included:

• Map/reduce: oxideswarm mapreduce has built-in kernels, or it ships your own scripts (design 0016).
• Web dashboard: live workers, tasks and links over plain HTTP. It listens on loopback by default, and needs a token once the master has a secret.
• Discovery: --master auto finds a master on the LAN through an authenticated UDP beacon. An HTTP probe and a rendezvous registry exist, with gaps that FEAT-7 tracks.
• Host protection: above 85 % host CPU, the master stops sending new tasks to that worker (--max-host-cpu-pct). Tasks run at below-normal priority.
• Output capture: up to 10 MiB per stream, keeping both the head and the tail, so a long build log still ends with its summary (design 0022).
• GPU: a worker built with --features gpu-wgpu multiplies matrices on Metal, Vulkan or DirectX 12 and checks every result against a CPU sample. It is a benchmark, not a general GPU runtime. --simulate-gpu is a labelled CPU mode for tests, and the scheduler prefers real GPUs over it.
• 3D tasks: procedural mesh generation and auto-rigging. They are available only in a worker built from source with the Cargo feature 3d; the release archives do not include it (details).
• Agent Mesh: a separate WebSocket hub through which agents on several machines exchange messages and run each other's advertised commands, including shell_exec (below).

Details and the full table: Network port topology and docs/SECURITY.md.

A macOS and Windows walk-through, checked end to end, is in docs/guides/two-machine-cluster.md.

# macOS / Linux: cluster port 8088 on every interface, dashboard on this machine only, plus a local worker
./switch_role.sh master
./switch_role.sh master --p2p              # also publish a P2P ticket for machines on other networks (see below)
./switch_role.sh master --dashboard-lan    # also open the dashboard (:8080) to phones / the LAN
./switch_role.sh master --no-worker        # master only

# the same by hand
./target/release/oxideswarm master --listen 0.0.0.0:8088
./target/release/oxideswarm master --listen 0.0.0.0:8088 --p2p --p2p-ticket-file ~/.oxideswarm/master_ticket.txt

Windows master: windows\switch_role.cmd master [--p2p] [--lan-dashboard].

The first start of a master that other machines can reach creates ~/.oxideswarm/cluster.secret and prints where it is. Copy that file to every machine that should join (workers and anyone running submit), over a channel you trust, for example:

scp user@master:~/.oxideswarm/cluster.secret ~/.oxideswarm/cluster.secret && chmod 600 ~/.oxideswarm/cluster.secret

(Windows: %USERPROFILE%\.oxideswarm\cluster.secret; a USB stick works too. Never chat, e-mail or cloud-synced folders.) Quick path: join with one token. Instead of copying the file and typing the address, create a token on the master and pipe it into oxideswarm join on the new machine (the token is as sensitive as the secret: send it over a channel you trust, never through chat or e-mail, and never as a command line argument):

# on the master
./target/release/oxideswarm secret join-token --master 192.168.1.100:8088 > join.token
# on the new machine (token on standard input; saves the secret at mode 600 and the address, then runs the doctor checks)
./target/release/oxideswarm join < join.token && ./target/release/oxideswarm worker

A master bound to loopback only (plain oxideswarm master) needs no secret. The dashboard token is oxideswarm secret show --dashboard-token. More: docs/SECURITY.md.

For a Mac and a Windows PC, follow docs/guides/two-machine-cluster.md: it covers the link (Wi-Fi, Ethernet, P2P), both roles and the check with oxideswarm doctor.

Put the master's cluster.secret into %USERPROFILE%\.oxideswarm\, then in the windows\ folder:

• Double-click windows/run_worker_silent.vbs (looks for the master with auto), or
• run windows\switch_role.cmd worker <MASTER_IP>:8088, or windows\start_worker.cmd <MASTER_IP>:8088 (the worker uses the GPU only if rusty-grid.exe was built with --features gpu-wgpu, e.g. ./build_windows_worker.sh --gpu; it then finds the card itself and prints the adapter at start-up, and without that feature it says so and advertises no GPU, also with --gpu), or
• install a service that starts at boot: windows\install_service.cmd <MASTER_IP>:8088 C:\path\to\cluster.secret (runs as the low-privilege virtual account NT SERVICE\OxideSwarmWorker, not SYSTEM; see packaging/windows/README.md).

• Termux worker (works with the secret today): packaging/android/termux: run install_termux_daemon.sh, put the master's cluster address (<MASTER_IP>:8088) and the copied secret in place, then start_worker.sh --daemon.
• Android app (JNI): build liboxideworker.so first with packaging/android/build_android_jni.sh --release and then the APK (guide). Supports native cluster secret configuration via nativeSetClusterSecret (note: the Gradle APK project in packaging/android/app/ does not compile yet; see AND-1 in docs/BACKLOG.md).
• Dashboard from the phone: start the master with --dashboard-lan, open http://<MASTER_IP>:8080 and log in with the token from oxideswarm secret show --dashboard-token. The page's "Scan Master" button probes the current host, hosts saved in the browser and hosts the master lists in /api/status; it does not scan your subnet.

• On the Windows PC: windows\switch_role.cmd master (or menu option 1).
• On the Mac: ./switch_role.sh worker <WINDOWS_IP>:8088 (or menu option 2). Both machines need the same cluster.secret first. Nothing else is killed: the launchers stop only the processes they started (PID files in ~/.oxideswarm/run).

This needs both the ticket and the secret. When two machines are on different networks (e.g., laptop on remote Wi-Fi, desktop compute node at home/office):

1. On the Master coordinator, enable P2P: ./switch_role.sh master --p2p (or oxideswarm master --listen 0.0.0.0:8088 --p2p --p2p-ticket-file ~/.oxideswarm/master_ticket.txt). The master requires outbound Internet access to reach public Iroh (n0) DERP relays.
   • The connection ticket is written to ~/.oxideswarm/master_ticket.txt, printed in console output, or copied via the "📋 Copy P2P Ticket" button in the Web Dashboard.
   • The ticket remains deterministic across restarts if ~/.oxideswarm/master_key.bin is preserved and relays are reachable during startup.
2. Copy the cluster secret ~/.oxideswarm/cluster.secret from the Master to the worker machine via scp, SSH, or physical USB drive. The P2P ticket provides address routing only and does not establish authorization; workers refuse pairing if the secret is missing (unless --insecure is explicitly passed).
3. On the Worker machine:
   • macOS / Linux: Run ./connect_remote.sh "<TICKET>" --secret-file /path/to/cluster.secret (or execute without arguments to paste interactively). The ticket is persisted to ~/.oxideswarm/remote-worker.toml (mode 0600) without modifying default configs. Pass --service to install a user-level LaunchDaemon on macOS.
   • Windows: Double-click connect_remote.cmd and paste the ticket when prompted (avoid passing JSON via command-line arguments to prevent shell character stripping). The script requests elevation, installs the OxideSwarmWorker service under the virtual account NT SERVICE\OxideSwarmWorker, and awaits registration confirmation.
4. Automated Path Routing:
   • Nodes attempt UDP hole-punching for direct zero-broker connectivity (🟢 Direct P2P | <RTT>ms).
   • If UDP traffic is restricted by carrier CGNAT or firewalls, traffic automatically routes through encrypted HTTPS relays (🟣 DERP Relay | <RTT>ms).
   • Comparative performance benchmarks are documented in the Interconnect Benchmark Report.
   • P2P communication is encrypted via TLS by Iroh, in addition to HMAC-SHA256 cluster secret verification.

A lightweight utility for two machines sharing a cloud-synchronized folder (e.g., Google Drive, OneDrive, or local NFS/SMB share) to verify peer visibility:

• Role Assignment (SERVER / CLIENT): Mediated via ephemeral claim_<platform>.json tokens in the shared directory (claims older than 180 seconds are invalidated).
• HTTP Ping on Port 8085: Direct reachability check between hosts; writes ping_success.txt upon verification or generates detailed troubleshooting logs.
• Process & Secret Isolation: The probe does not interact with the Rust compute grid (ports 8088, 8080, 8089) and never exchanges tickets or secrets. Secrets must never be stored in cloud-synced storage.
• For complete details, see scripts/sync_network/README.md.

Start with oxideswarm doctor --expect-workers <N> on any machine that holds the secret: it names the check that fails.

Each archive has a -gpu variant built with --features gpu-wgpu.

OxideSwarm 0.1.2 covers the user story above. What it does not do yet:

The plan and the order of the work are in docs/ROADMAP.md. Every item, with its evidence and its "done when", is in docs/BACKLOG.md.

Long-term goal. Grow OxideSwarm from one owner's machines into an open compute network, where anyone can offer idle compute and anyone can use it, safely and with results that can be checked. It takes years and stages that are each useful alone: a WebAssembly task type, federation between trusted clusters, then a public pool with verified results. The draft is design 0054 and Phase H of the roadmap.

• A master that other machines can reach (--listen 0.0.0.0:8088 or --p2p) creates ~/.oxideswarm/cluster.secret and refuses peers without it. Workers and clients refuse a remote master without a secret unless you pass --insecure.
• The handshake authenticates both sides, and every later frame is a ChaCha20-Poly1305 record.
• The dashboard listens on loopback by default. Once the master has a secret, it needs a token (oxideswarm secret show --dashboard-token).
• --insecure turns off the secret requirement; use it only on a network you fully control. The current handshake (OXS-AUTH v2) has no forward secrecy.
• Every machine of a cluster must run the same release: the oldest protocol version a peer accepts is the current one. Upgrade and restart the master first, then the workers and clients (CHANGELOG, DEPLOYMENT.md section 3.5).
• Anyone who holds the secret can run commands on every worker. Share it only with machines you trust. Run workers with --isolation fs where you can, and never run a worker as root or SYSTEM.

The threat model, what each port exposes and how to rotate the secret are in docs/SECURITY.md.

OxideSwarm also contains an agent-to-agent communication layer (crates/agent_mesh) that lets coding agents on Windows, macOS, Ubuntu Linux and Android (Termux / ADB) discover peers, transmit data payloads and route structured commands. Nodes execute only 5 advertised commands (echo, ping, system_info, sha256_verify, shell_exec); shell_exec can be disabled with --no-shell. Treat the hub token like a password.

                      ┌─────────────────────────────────┐
                      │         OxideRelay Hub          │
                      │    (axum / tokio-tungstenite)   │
                      │      Active Node Catalog        │
                      └───────▲───────▲────────▲────────┘
                              │       │        │
                Persistent WS │       │ WS     │ Persistent WS
                    (outbound)│       │        │ (outbound)
                              ▼       ▼        ▼
            ┌─────────────────┐ ┌───────────┐ ┌───────────────────┐
            │ Node 1: Windows │ │Node 2: Mac│ │ Node 3: Android   │
            │ Coding Agent    │ │Coding Agt │ │ Coding Agent      │
            └─────────────────┘ └───────────┘ └───────────────────┘

• Outbound-only nodes: nodes connect out to the hub over WebSocket, so they need no inbound port (the hub host does).
• Explicit point-to-point addressing (from: "node-1", to: "node-3"); the hub enforces that from is the registered node id and rejects duplicate ids.
• Dual runtime: Rust binary agent-mesh, and a dependency-light Python client scripts/agent_node.py.
• Delivery tracking & integrity: correlation ids, DeliveryAck / DeliveryNack, SHA-256 verification of data payloads.
• Authentication: the hub listens on 127.0.0.1:8088 by default; a non-loopback --listen needs --token (or OXIDE_MESH_TOKEN), or --insecure. ws:// is not encrypted, so use wss:// behind a TLS proxy or a VPN / SSH tunnel, and never expose the hub to the Internet.

# 1. Build the Agent Mesh CLI:
cargo build --release -p agent_mesh

# 2. Pick a hub token and start the hub. The hub also defaults to port 8088, which the grid master uses,
#    so give it another port when both run on one machine:
export OXIDE_MESH_TOKEN="$(openssl rand -hex 24)"
./target/release/agent-mesh hub --listen 127.0.0.1:8090          # loopback; use 0.0.0.0:8090 + token for a LAN

# 3. Connect an agent node (Windows / Mac / Linux / Android); every command below reads OXIDE_MESH_TOKEN:
./target/release/agent-mesh node --hub ws://127.0.0.1:8090/ws --id node-mac-1 --platform macos
python3 scripts/agent_node.py --hub ws://127.0.0.1:8090/ws --id node-android-3 --platform android
# To disable shell command execution on untrusted nodes, add --no-shell:
./target/release/agent-mesh node --hub ws://127.0.0.1:8090/ws --id node-mac-1 --platform macos --no-shell

# 4. Route a command from any node or terminal:
./target/release/agent-mesh send --hub ws://127.0.0.1:8090/ws --to node-android-3 --command shell_exec --args '{"cmd": "uname -m"}'

# 5. Query the online nodes catalog:
./target/release/agent-mesh list --hub ws://127.0.0.1:8090/ws

--grid-master [ADDR] makes the hub start its own embedded Master Grid coordinator on ADDR. It does not connect to a master that is already running, so give it a free port. A non-loopback ADDR needs a cluster secret (--secret / OXIDESWARM_SECRET, else the default ~/.oxideswarm/cluster.secret) or an explicit --insecure: the hub refuses to start an unauthenticated master that the network can reach. Pass --max-active-mb <MB> (or set RUSTY_GRID_MAX_ACTIVE_MB) to configure the embedded master's active queue byte budget (default 256 MiB, minimum 1 MiB).

# Start the hub with an embedded master on 127.0.0.1:9088 (loopback, so no secret is needed):
./target/release/agent-mesh hub --listen 127.0.0.1:8090 --grid-master 127.0.0.1:9088 --max-active-mb 512

# When --grid-master is configured, agent mesh nodes can dispatch compute envelopes:
# grid_compute, grid_compile, grid_submit, and grid_status execute end-to-end against the grid.
# When --grid-master is omitted, grid envelopes return ERR_BRIDGE_NOT_CONFIGURED.

Platform guides: DEPLOYMENT.md.

oxideswarm and rusty-grid are the same program (rusty-grid is the older name; use either). Every command accepts --help. Clients and workers read the cluster secret from --secret-file, OXIDESWARM_SECRET_FILE, or ~/.oxideswarm/cluster.secret.

# Master. Default listen is 127.0.0.1:8088 (this machine only); other machines need 0.0.0.0:8088.
./target/release/oxideswarm master --listen 0.0.0.0:8088
# Persist the task queue across master restarts (snapshot.bin + journal.bin; defaults --blob-dir to <DIR>/blobs):
./target/release/oxideswarm master --listen 0.0.0.0:8088 --state-dir ~/.oxideswarm/state
# Raise active and finished-task memory budgets (each defaults to 256 MiB):
./target/release/oxideswarm master --listen 0.0.0.0:8088 --max-active-mb 512 --max-retained-mb 512
# Dashboard on the LAN as well (explicit; default is loopback):
./target/release/oxideswarm master --listen 0.0.0.0:8088 --web-ui-addr 0.0.0.0:8080
# Register with Tier 3 HTTP rendezvous registry:
./target/release/oxideswarm master --listen 0.0.0.0:8088 --rendezvous http://rendezvous.example.com:8080

# Join a machine with one token (design 0048): print on the master, read from standard input on the new machine
./target/release/oxideswarm secret join-token --master 192.168.1.100:8088 [--p2p-ticket-file ~/.oxideswarm/master_ticket.txt]
./target/release/oxideswarm join [--replace-secret] [--expect-workers N] [--no-doctor] < join.token

# Worker: --master takes the CLUSTER port (default 127.0.0.1:8088; a bare host gets :8088; 'auto' = LAN discovery)
./target/release/oxideswarm worker --master 192.168.1.100:8088
./target/release/oxideswarm worker --master 192.168.1.100:8088                  # a --features gpu-wgpu build finds and uses the GPU itself
./target/release/oxideswarm worker --master 192.168.1.100:8088 --gpu            # insist: warns and advertises no GPU if there is none / no wgpu
./target/release/oxideswarm worker --master 192.168.1.100:8088 --no-gpu         # keep this machine's GPU out of the grid
./target/release/oxideswarm worker --master auto --rendezvous http://rendezvous.example.com:8080  # discover via Tier 3 fallback

# Submit work (--command is the executable, arguments follow after --; the file must exist on the worker)
./target/release/oxideswarm submit --master 127.0.0.1:8088 --command echo --wait -- hello
./target/release/oxideswarm submit --master 127.0.0.1:8088 --command sha256sum --wait -- /path/present/on/the/worker
./target/release/oxideswarm submit --master 127.0.0.1:8088 --type gpu --gpu --wait   # routed to a worker that advertises a GPU

# Input and output files: send a file or a directory, bring back what the patterns select
# (files the master already holds are not sent again; outputs land in --output-dir at their sandbox paths; --output-dir requires --output)
./target/release/oxideswarm submit --input input.dat --output results.csv --output-dir out/ --command process_data -- input.dat results.csv
./target/release/oxideswarm submit --input ./my-crate:. --output 'target/release/my-crate' --output-dir out/ --timeout-secs 600 --command cargo -- build --release
./target/release/oxideswarm submit --input data/ --output 'reports/**/*.json' --type shell --script-file analyse.sh

# Worker with user tags (matched by `submit --require-tag`); os:/arch: tags are added automatically
./target/release/oxideswarm worker --master <MASTER_IP>:8088 --tag laptop --tag office
# A second worker on the same machine: give it its own name and no standby portal on :8080
./target/release/oxideswarm worker --master <MASTER_IP>:8088 --name <NAME>-2 --no-portal

# Pin a task to an OS, an architecture, a tag or one worker (a name, or an id from `workers --json`)
./target/release/oxideswarm submit --os windows --wait --command cmd -- /C ver
./target/release/oxideswarm submit --worker <WORKER_NAME> --wait -- uname -a
./target/release/oxideswarm submit --require-tag laptop --arch x86_64 --wait -- hostname

# Send input with the task (each file at most 32 MiB): a script, standard input, environment variables
./target/release/oxideswarm submit --type shell --interpreter python3 --script-file job.py --wait
./target/release/oxideswarm submit --stdin-file data.txt --env MODE=fast --wait --command wc -- -l

# GPU task of a chosen size (default 64; up to 8192 on a real GPU, 2048 on the simulated one)
./target/release/oxideswarm submit --type gpu --matrix-dim 2048 --wait

# 3D mesh generation: default runs on CPU via procedural SDF and writes a glTF 2.0 binary asset
./target/release/oxideswarm submit --prompt "a solid cube" -o cube.glb --wait
./target/release/oxideswarm submit --prompt "robot" --engine sdf -o robot.glb --wait

# Inspect the grid, fetch task results, or cancel a task
./target/release/oxideswarm workers --master 127.0.0.1:8088
./target/release/oxideswarm workspaces --master 127.0.0.1:8088
./target/release/oxideswarm status --master 127.0.0.1:8088 --json
./target/release/oxideswarm result <TASK_ID> --wait --output-dir out/
./target/release/oxideswarm cancel <TASK_ID>

# Persistent agent workspaces: sync a project tree once and reuse its build directory across runs
./target/release/oxideswarm submit --workspace oxideswarm --wait --command cargo -- test -p rusty_grid_core --lib
./target/release/oxideswarm submit --workspace oxideswarm --no-sync --cwd crates/core --wait --command cargo -- test --lib
./target/release/oxideswarm mapreduce --master 127.0.0.1:8088 --input data.txt --mapper word_count --reducer sum
# `--mapper` / `--reducer` ship local .sh scripts to workers; `--map-cmd` / `--reduce-cmd` run scripts/binaries that are already on every worker
./target/release/oxideswarm mapreduce --master 127.0.0.1:8088 --input data.txt --mapper ./map.sh --reducer ./reduce.sh
./target/release/oxideswarm mapreduce --master 127.0.0.1:8088 --input data.txt --map-cmd "python3 /tmp/map.py" --reduce-cmd "python3 /tmp/reduce.py"

# Model Context Protocol (MCP) stdio server for AI coding assistants
./target/release/oxideswarm mcp --master 127.0.0.1:8088

# Check the whole cluster: every worker, process start-up, every GPU, spreading
# (exit code 0 = all checks passed, 1 = a check failed, 6 = authentication failed)
./target/release/oxideswarm doctor --expect-workers 2
./target/release/oxideswarm doctor --json --skip-gpu

# Secret management
./target/release/oxideswarm secret generate
./target/release/oxideswarm secret show --dashboard-token

Exit code 6 means authentication failed. Remote or auto-discovered masters are refused without a secret unless you pass --insecure (RUSTY_GRID_INSECURE=1, or insecure = true under [master], [worker], [submit], or [status] in rusty-grid.toml).

Every subcommand inherits the global flags below and responds to --help (-h) and --version (-V).

OxideSwarm resolves configuration through a 4-tier hierarchical engine: $$\text{CLI Flags} > \text{Environment Variables} > \text{Configuration File (rusty-grid.toml)} > \text{Defaults}$$

ox-mode runs the project's own test, development, documentation and research workflows:

# Launch the interactive terminal UI (TUI) menu:
./ox-mode

# Code Quality & Testing Mode (Linting + Flaky test detection):
./ox-mode test lint
./ox-mode test flaky --iterations 5

# Development Mode (Live-reload file watcher and local dev cluster):
./ox-mode dev watch
./ox-mode dev cluster   # binds 127.0.0.1:9090 and 127.0.0.1:8080 (or falls back to free ephemeral ports recorded in .dev_cluster.json)
./ox-mode dev status    # probes the dashboard port from .dev_cluster.json (or --port / 8080)
./ox-mode dev stop

# Research & Performance Profiling Mode:
./ox-mode research bench
./ox-mode research distributed

The architecture map is CLAUDE.md; the documentation index and the writing standard are docs/README.md. A change to behaviour, a CLI flag, an environment variable or the wire format comes with a design record in docs/design/ and an update of the reference tables in this file, in the same pull request.

OxideSwarm/
├── Cargo.toml                     # Workspace root manifest (no root package)
├── README.md                      # This file
├── CLAUDE.md                      # Architecture map for AI coding assistants
├── docs/                          # README.md (index + writing standard), SECURITY.md (threat model),
│                                  # guides/ (runbooks), design/ (design records)
├── DEPLOYMENT.md                  # Agent Mesh deployment guide
├── ANDROID_ARCHITECTURE.md        # Android architecture (parts are design only, marked as such)
├── switch_role.sh                 # Master <-> Worker role switcher for macOS / Linux
├── connect_remote.sh              # P2P WAN pairing for macOS / Linux
├── connect_remote.cmd             # P2P WAN pairing for Windows
├── test_integration.sh            # E2E: 1 master + 3 workers on loopback, private HOME
├── run_chaos_test.sh              # local failover test (kills a worker that holds running tasks)
├── run_cluster_benchmark.sh       # two-node benchmark (two local workers, or a phone via adb)
├── ox-mode -> scripts/mode.sh     # Unified CLI & TUI workflow automation runner
├── windows/                       # Windows launchers (oxctl.ps1 engine + .cmd/.vbs wrappers)
├── assets/                        # Vector branding assets
├── benchmarks/interconnect/       # Interconnect comparison: static recorded data + a validator script
├── engines/
│   ├── generative_3d/             # Tier-1 3D generation Python entrypoint (`entrypoint.py`)
│   └── autorig/                   # Headless Blender 4.5 LTS auto-rigging driver (`blender_rig.py`)
├── scripts/
│   ├── sync_network/              # Shared-folder reachability probe (not part of the grid)
│   ├── agent_node.py              # Lightweight Python agent node client
│   └── mode.sh                    # Workflow mode automation script
├── packaging/
│   ├── agent_mesh/                # Agent node launchers (windows / macos / ubuntu / android)
│   ├── windows/                   # Windows service: ServiceWrapper.cs (csc.exe) or WinSW, installer, verifier
│   ├── android/                   # Android app project, Termux worker daemon, JNI build script
│   └── macos/                     # LaunchDaemon installer / verifier / uninstaller
├── crates/
│   ├── core/                      # [Layer 1] Wire protocol, codec, cluster_auth (secret handshake), discovery, workflow modes
│   ├── master/                    # [Layer 1] Scheduler, GPU routing, FSM, LRU bounding, Axum dashboard, RBAC auth
│   ├── worker/                    # [Layer 1] Task runner, process-tree kill, CPU SIMD / wgpu compute, gen3d, autorig, standby portal
│   ├── cli/                       # [Layer 1] `oxideswarm` / `rusty-grid` / `ox-mode`; connect.rs (client plumbing), secret_cmd.rs
│   ├── android_bridge/            # [Layer 2] JNI bridge (liboxideworker.so)
│   └── agent_mesh/                # [Layer 3] Axum WebSocket relay hub & agent client
└── tests/                         # Integration tests (registered per crate in Cargo.toml) and Python suites

These task types build only with the Cargo feature 3d (cargo build --release --bin oxideswarm --features 3d). A worker built without it does not advertise them, and the master never places a 3D task on it. The code is in crates/worker/src/gen3d, crates/worker/src/autorig and engines/.

The worker executes two 3D asset pipeline task types (architecture and limits in docs/design/0023-generative-3d-and-autorig-subsystem.md, 0015, and 0017):

cargo test --workspace runs the unit, integration and stress tests of every crate, and CI runs it on Linux, macOS and Windows for every pull request. Tests that need the public iroh relays are #[ignore] by default and run weekly in the CI job p2p-wan-relay. Targets that need the 3d or gpu-wgpu features run in their own CI jobs.

# Rust unit and integration tests (slow: many stress tests)
cargo test --workspace

# Agent Mesh against the RUST hub (build first with: cargo build -p agent_mesh)
cargo test -p agent_mesh
python3 tests/test_agent_mesh_token_auth.py
python3 tests/test_agent_mesh_rust_hub_cross_interop.py

# Protocol simulation against a PYTHON reference hub (tests/run_mesh_simulation_tests.py does NOT start the Rust hub)
python3 tests/run_mesh_simulation_tests.py

# Launcher / installer scripts (switch_role.sh, connect_remote.sh, Termux supervisor, macOS installer dry-run,
# line endings, no hard-coded owner values): runs them against fake binaries in a private HOME; no build needed
python3 tests/test_launcher_scripts.py

# End-to-end on loopback: 1 master + 3 workers, one simulated-GPU worker. Uses a private HOME and
# ephemeral ports, touches no running master/worker and only signals the PIDs it started.
./test_integration.sh

# Failover: starts its own cluster, SIGKILLs a worker that holds running tasks, checks re-execution
./run_chaos_test.sh

# Throughput of two nodes (two local worker processes by default; --adb-id SERIAL uses a phone)
./run_cluster_benchmark.sh

# Static comparison data check + two local Rust benches (does NOT measure Tailscale or Cloudflare)
./benchmarks/interconnect/run_comparative_benchmark.sh

The scripts exit non-zero and say FAIL when a check fails; their reports are computed from what was observed in that run.

Benchmarks. benchmarks/grid measures no-op task overhead with 1 to 4 workers and a first versus a repeated upload of the same input tree on loopback, with the script that produced each dated record in benchmarks/grid/results. The records state their machine, load and limits.

OxideSwarm is dual-licensed under:

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT License (LICENSE-MIT)

You may choose either license at your option.