Commands as byte streams.

Command buffers are a Go vocabulary for automation: commands and files are byte streams, piping and copying are both io.Copy, and one Machine interface runs them everywhere — your own system, a container, a host across the network.

// Stream a database dump from a remote host into a local file.
db := ssh.Machine(sys.Machine(), "ssh", "[email protected]")
_, err := io.Copy(
    fs.CreateBuffer(ctx, command.FS(sys.Machine()), "backup.sql"),
    command.NewReader(ctx, db, "pg_dumpall"),
)

No YAML, no scp, no shell quoting. Bytes stream from pg_dumpall on the database host straight into a local file. Swapping the source or destination machine is a one-line change.

Automation as code

Command buffers are for the work that surrounds software: building and releasing it, moving files and data between machines, setting up servers and containers. That work usually lives in shell scripts, task runners, and pipeline configuration. As it grows conditionals, loops, and templating, the configuration becomes a programming language without a debugger, a formatter, or a test framework — and the tool itself is often replaced before the investment pays off.

Command buffers start from the other end: a real programming language, with the pipelines and file operations automation needs. Scripts get functions, libraries, and tests from the first line; they ship as one static binary; and the same program runs on a laptop, in CI, and across Linux, macOS, and Windows. Every automation tool is a language to learn; few say so. This one's is Go. The Tour of Go covers the basics this page assumes.

Adoption is incremental. One script or one CI job — usually the flaky one — can move first while everything around it stays put, because the result is an ordinary program that any existing pipeline job or playbook task can call. The exit is as cheap as the entry: these are small libraries on Go primitives behind a one-method interface, so the worst case is maintaining a fork or returning to os/exec — not migrating off a platform.

The model

• A Buffer is a command's execution as an io.Reader: the command starts on the first Read and is finished at io.EOF.
• A Machine is anything that can run a command — one interface, one method: Command(ctx, args...) Buffer.
• Machines compose. ssh, ctr, and sub each take a Machine and return a new one, so a container on a remote host is just two calls.
• Files are buffers too. command.FS gives any Machine a filesystem, copying between machines is io.Copy, and directories stream as tar archives.
• A Shell declares an execution environment — the commands a program may run, with portable operations for everything else — and that declaration is what makes automation portable.

Every example on this page compiles and runs against the current releases of lesiw.io/command and lesiw.io/fs.

Reading runs the command

A buffer is a command lifted into Go's io vocabulary. It executes because something consumes it — the way a shell pipeline moves when something reads the pipe — so everything built for io.Reader and io.Writer already works on commands, from bufio.Scanner to io.MultiWriter.

package main

import (
    "context"
    "fmt"
    "io"
    "log"

    "lesiw.io/command"
    "lesiw.io/command/mem"
)

func main() {
    ctx, m := context.Background(), mem.Machine()

    out, err := io.ReadAll(command.NewReader(ctx, m, "echo", "Hello, world!"))
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%s", out)
}

Run this program in the Go Playground — mem.Machine is an in-memory machine, so the example works anywhere.

Because a pipe is just bytes moving from a reader to a writer, piping one command into another is io.Copy.

// echo "hello, pipes" | tr a-z A-Z
_, err := io.Copy(
    command.NewWriter(ctx, m, "tr", "a-z", "A-Z"),
    command.NewReader(ctx, m, "echo", "hello, pipes"),
)

For pipelines of three or more stages, use command.Copy with command.NewFilter for the middle commands — run one in the Go Playground.

Three helpers cover the everyday cases, so most code never touches a buffer directly.

// version=$(go version)     — capture output, like command substitution.
version, err := command.Read(ctx, m, "go", "version")

// go test ./... >/dev/null  — run for the side effect, discard output.
err = command.Do(ctx, m, "go", "test", "./...")

// go vet ./...              — attach to the terminal, stream output live.
err = command.Exec(ctx, m, "go", "vet", "./...")

Environment variables travel on the context, so they compose the way the rest of Go does.

ctx = command.WithEnv(ctx, map[string]string{"CGO_ENABLED": "0"})
err := command.Exec(ctx, m, "go", "build", ".")

Anything that runs a command is a machine

The interface has one method. Everything else in the package is built on it.

type Machine interface {
    Command(ctx context.Context, arg ...string) Buffer
}

m := sys.Machine()

m := ssh.Machine(sys.Machine(),
    "ssh", "user@host")

m := ctr.Machine(sys.Machine(),
    "alpine:latest")

m := sub.Machine(sys.Machine(),
    "busybox")

m := mem.Machine()

m := new(mock.Machine)

SSH is the remote transport, and by now it is close to universal — every major operating system ships an OpenSSH server, Windows included. A Windows host is a remote like any other; commands travel to it as a base64-encoded PowerShell script, which survives every quoting layer in between — the quoting is fuzz-tested. The script targets Windows PowerShell 5.1, stock on Windows Server, and encoded commands run without execution-policy changes.

win := ssh.Machine(sys.Machine(), "ssh", "[email protected]")
sh := command.Shell(win)
fmt.Println(sh.OS(ctx), sh.Arch(ctx))     // windows amd64
err := sh.WriteFile(ctx, "app.yaml", cfg) // PowerShell idioms underneath

Machines take machines. ssh.Machine needs somewhere to run ssh; ctr.Machine needs somewhere to run docker. That somewhere is another Machine — so environments nest by construction.

// A container on a remote build host.
host := ssh.Machine(sys.Machine(), "ssh", "[email protected]")
m := ctr.Machine(host, "golang:latest")
defer command.Shutdown(ctx, m)

sh := command.Shell(m, "go")
if err := sh.Exec(ctx, "go", "test", "./..."); err != nil {
    log.Fatal(err)
}

A prefix glued onto a command — ssh host …, docker exec …, env FOO=bar … — is a machine that hasn't been named yet. Once it is one, the code that runs commands no longer depends on where they run: the same program works on a laptop, in a container, and on a production host.

A machine doesn't have to be a computer. It can be a single command, enriched — here, a shim that injects an environment variable into every go invocation.

m := command.HandleFunc(sys.Machine(), "go",
    func(ctx context.Context, args ...string) command.Buffer {
        ctx = command.WithEnv(ctx, map[string]string{
            "GOFLAGS": "-trimpath",
        })
        return sys.Machine().Command(ctx, args...)
    })

flags, err := command.Read(ctx, m, "go", "env", "GOFLAGS")
// flags == "-trimpath"

Files are buffers too

command.FS turns any machine into a filesystem, accessed through lesiw.io/fs — a context-aware cousin of io/fs with write support. On machines without native filesystem access, file operations are implemented with the commands the target system has: tee on Unix, Remove-Item on Windows. Your code doesn't see the difference — and Windows is an origin, not just a destination. The same program compiles for and runs on a Windows host.

sh := command.Shell(sys.Machine())
fmt.Println(sh.OS(ctx), sh.Arch(ctx))          // windows amd64
err := sh.WriteFile(ctx, "out.txt", data)      // PowerShell idioms underneath
err = sh.MkdirAll(ctx, `logs\2026`)            // no mkdir -p in sight

fsys := command.FS(m)

err := fs.WriteFile(ctx, fsys, "greeting.txt", []byte("Hello from fs!\n"))

data, err := fs.ReadFile(ctx, fsys, "greeting.txt")

Run this program in the Go Playground.

Copying a file between machines is the same io.Copy as piping commands. The operations scp, docker cp, and cp share one implementation.

local := command.Shell(sys.Machine())
remote := command.Shell(ssh.Machine(sys.Machine(), "ssh", "[email protected]"))

// Stream a local build to the remote host.
_, err := io.Copy(
    remote.CreateBuffer(ctx, "/opt/app/server"),
    local.OpenBuffer(ctx, "bin/server"),
)

A directory is a buffer too: a tar archive, indicated by a trailing slash. Copying a directory into a container looks exactly like copying a file.

box := ctr.Machine(sys.Machine(), "alpine:latest")
defer command.Shutdown(ctx, box)

local, remote := command.Shell(sys.Machine()), command.Shell(box)

dst, err := remote.Create(ctx, "/app/")   // trailing slash: a directory
if err != nil {
    log.Fatal(err)
}
defer dst.Close()
src, err := local.Open(ctx, "src/")
if err != nil {
    log.Fatal(err)
}
defer src.Close()
if _, err := io.Copy(dst, src); err != nil {
    log.Fatal(err)
}

Because commands and files are the same shape, they mix in one pipeline. This example dumps a database, compresses it in flight, and lands it on another machine, touching no disk along the way.

backup := ssh.Machine(m, "ssh", "[email protected]")
_, err := command.Copy(
    fs.CreateBuffer(ctx, command.FS(backup), "db.sql.gz"),
    command.NewReader(ctx, m, "pg_dumpall"),
    command.NewFilter(ctx, m, "gzip"),
)

Shells make automation portable

command.Shell exists to keep automation portable. It wraps a machine with two things: operations that work on any machine for everything a filesystem can do, and an explicit list of the external commands your automation is allowed to run. Anything you didn't declare fails with command not found at the moment of the call — the dependency list lives at the top of the file, and it stays short because most of what shell scripts shell out for doesn't need a command at all.

sh := command.Shell(sys.Machine(), "go")

if err := sh.Exec(ctx, "go", "vet", "./..."); err != nil {
    log.Fatal(err)
}

ver, err := sh.ReadFile(ctx, "VERSION")
if err != nil {
    ver = []byte("dev")
}

if err := sh.MkdirAll(ctx, "bin"); err != nil {
    log.Fatal(err)
}
err = sh.Exec(
    command.WithEnv(ctx, map[string]string{"CGO_ENABLED": "0"}),
    "go", "build", "-ldflags", "-X main.version="+string(ver),
    "-o", "bin/app", ".",
)

The build declares only go. Reading VERSION and creating the output directory go through the Shell's portable methods, which work on any machine — including Windows, where cat and mkdir -p do not exist. Both libraries' CI suites (command, fs) run the tests — not just the build — on Linux, macOS, Windows, FreeBSD, and Alpine.

Automation you can test

Automation written against Machine is code, so it tests like code. mock.Machine records every call and returns whatever you program.

func Deploy(ctx context.Context, sh *command.Sh) error {
    branch, err := sh.Read(ctx, "git", "branch", "--show-current")
    if err != nil {
        return fmt.Errorf("read branch: %w", err)
    }
    return sh.Exec(ctx, "git", "push", "origin", branch)
}

func TestDeploy(t *testing.T) {
    m := new(mock.Machine)
    m.Return(strings.NewReader("main\n"), "git", "branch", "--show-current")

    sh := command.Shell(m, "git")
    if err := Deploy(t.Context(), sh); err != nil {
        t.Fatal(err)
    }

    got := mock.Calls(sh, "git")
    want := []mock.Call{
        {Args: []string{"git", "branch", "--show-current"}},
        {Args: []string{"git", "push", "origin", "main"}},
    }
    if !cmp.Equal(want, got) {
        t.Errorf("git calls mismatch (-want +got):\n%s", cmp.Diff(want, got))
    }
}

No git repository, no network, no side effects — and the test asserts the exact commands your automation would have run in production.

The guide carries these patterns and the rest of the vocabulary in a form sized for a coding agent's context window.

Unprogrammed commands succeed with empty output, so tests only stub what they assert. For a strict mock, set the default response instead — Return with no command arguments applies to every unprogrammed command, and a zero-code failure reads as command not found.

m.Return(command.Fail(&command.Error{Err: fmt.Errorf("unexpected command")}))

err := command.Do(ctx, m, "anything")   // fails
command.NotFound(err)                   // true

When a command fails

Automation spends much of its life handling failure, so failures carry their evidence. A failed command returns a *command.Error with the exit code and captured stderr.

err := command.Do(ctx, m, "go", "build", ".")

var cerr *command.Error
if errors.As(err, &cerr) {
    fmt.Println("exit code:", cerr.Code)
    fmt.Printf("stderr:\n%s", cerr.Log)
}

When a multi-stage pipeline fails, the error reports every stage and its outcome, identifying the failing machine and command.

_, err := command.Copy(
    command.NewWriter(ctx, m, "wc", "-c"),
    command.NewReader(ctx, m, "echo", "not gzip data"),
    command.NewFilter(ctx, m, "gzip", "-d"),
)
fmt.Println(err)
// <*command.reader>
//     <success>
//
// <*command.filter>
//     exit status 1
//         gzip: unknown compression format

Output produced before a failure is not lost: reads return the bytes that arrived, and the error follows. Cancellation follows the context. Cancel the ctx and a running command is killed; the pending Read returns the error, and no process is left behind. NewReader's Close does the same for early exits from a pipe.

And failures are programmable in tests, so the error path gets exercised as easily as the happy path.

m.Return(command.Fail(&command.Error{Code: 1}), "git", "push")

if err := Deploy(t.Context(), sh); err == nil {
    t.Error("Deploy() should fail when the push fails")
}

Inside the CI you already have

Command buffers don't replace a CI system; they replace the scripts inside it. A pipeline job runs a Go program.

deploy:
  stage: deploy
  script: go run ./ops deploy

Reproducing that job on a laptop is the same command: go run ./ops deploy. The YAML shrinks to a list of triggers; the logic — the part that breaks — lives in a program that runs anywhere, so a failing job can be reproduced and debugged without pushing a commit to find out. The orchestrator stops mattering: GitLab, Jenkins, GitHub Actions, or nothing at all — a job becomes a program invocation, and an artifact becomes a file or directory stream the program moves itself. Porting can start with a single job, usually the flaky one, while the rest of the pipeline stays put. The program is an ordinary process, so variables and secrets the CI system injects arrive as plain environment, and caching stays where it was — the runner caches directories, whatever the job runs.

Why Go for automation

Any compiled language could take automation out of configuration formats. Go's particulars make it a good choice:

• It keeps working. The Go 1 compatibility promise means automation written today still builds years from now.
• Dependencies are solved. Modules give automation code the same sharing and versioning as any other code.
• Concurrency is ordinary. Backing up one host and backing up a hundred are the same code plus a goroutine per host.

Command buffers are not a replacement for a five-line shell script. They're for the automation that outgrew one — the build that needs a real conditional, the deploy that spans three machines, the script someone finally asked you to write a test for.

Get started

go get lesiw.io/command

A complete program.

package main

import (
    "context"
    "fmt"
    "log"

    "lesiw.io/command"
    "lesiw.io/command/sys"
)

func main() {
    ctx, m := context.Background(), sys.Machine()

    version, err := command.Read(ctx, m, "go", "version")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(version)
}

Keep reading

• lesiw.io/command — full API documentation, including a cookbook of shell idioms translated to Go.
• lesiw.io/fs — the filesystem abstraction: local, remote, and in-memory filesystems behind one interface.
• The cmdbuf guide — a condensed, example-first reference for writing idiomatic command buffer code. Written with coding agents in mind; humans welcome too.
• Source on GitHub.