GeezerVison

Summary

Adds an extension for Baseshift, which creates masked, writable clones of production databases for development. The extension runs the whole Baseshift flow locally next to LocalStack: replicating and masking a source database, publishing snapshot images to LocalStack ECR, and starting clones from them, so app code running in LocalStack can work against realistic but anonymized data.

• Self-hosted Dubs: POST /dubs creates a Dub for a source database (e.g. LocalStack RDS) and runs the Baseshift agent and replication server as containers next to LocalStack (or optionally in a LocalStack EKS cluster). The extension drives the baseshift CLI, logged in as a system user (BASESHIFT_USER_EMAIL / BASESHIFT_USER_PASSWORD), applies an optional masking policy, and the masked snapshots end up in LocalStack ECR
• Clones API: POST/GET/DELETE http://baseshift.localhost.localstack.cloud:4566/clones starts and stops clones on demand (e.g. one per PR), from the latest snapshot of a Dub ({"dub": "<name>"}) or from any clone image
• Default clone: set BASESHIFT_IMAGE and a clone starts once LocalStack is ready, e.g. for a Dub that runs in your AWS account
• Connectivity: PostgreSQL clones are proxied through the gateway port 4566 (protocol handshake detection, like the ParadeDB extension) and published on host ports. MySQL clones only get a host port, since MySQL is server-speaks-first and can't be detected on the shared port

Architecture

flowchart TB
    dev["Developer / CI<br/>psql, IDE, tests, ELT job"]
    cloud["Baseshift Cloud (SaaS)<br/>control plane"]

    subgraph docker["Docker host"]
        subgraph ls["LocalStack container"]
            gateway["Gateway :4566"]
            subgraph ext["Baseshift extension"]
                dubsapi["Dubs API<br/>(baseshift CLI)"]
                clonesapi["Clones API"]
            end
            rds[("RDS PostgreSQL<br/>source data with PII")]
            ecr[("ECR registry<br/>snapshot images")]
            s3[("S3")]
            snowflake[("Snowflake")]
        end

        subgraph dub["Dub containers"]
            agent["Agent<br/>reads the source DB"]
            repserver["Replication server<br/>masking, snapshots"]
        end

        subgraph clones["Clone containers"]
            clone1[("Clone 'shop-dev'<br/>:5432")]
            clone2[("Clone 'pr-123'<br/>:15432")]
        end
    end

    cloud <-- "1 · create Dub, masking policy" --> dubsapi
    dubsapi -- "2 · run" --> agent
    rds -- "3 · logical replication" --> agent
    agent -- "changes" --> repserver
    cloud -. "manages" .-> repserver
    repserver -- "4 · push masked snapshot" --> ecr
    clonesapi -- "5 · start" --> clone1
    ecr -. "pull" .-> clone1
    gateway -- "SQL" --> clone1
    dev -- "AWS APIs, SQL" --> gateway
    dev -- "CSV export" --> s3
    s3 -- "COPY INTO" --> snowflake
    dev -. "SQL via host port" .-> clone2

1. The Dubs API uses the baseshift CLI to create the Dub in Baseshift Cloud and set its masking policy
2. It starts the agent and replication server as containers (managed via Baseshift Cloud)
3. The agent replicates the source DB via logical replication, the replication server masks it
4. Masked snapshot images get pushed to LocalStack ECR
5. The Clones API starts clone containers from them, reachable through the gateway or their host port

Demo

baseshift/demo/ has an end-to-end flow in one LocalStack container (Snowflake emulator image, which also serves RDS/ECR/S3):

RDS Postgres with PII → self-hosted Dub with masking policy → masked snapshot in LocalStack ECR → clone via the extension → export to S3 → COPY INTO Snowflake → the same record shown at each stage, with a check that no raw PII leaked.

make demo-selfhosted runs this with the real Baseshift components. make demo runs the same flow with a stand-in script for the masking step, which doesn't need a Baseshift user.

Output of a full local run:

column    source (RDS)                  Baseshift clone               Snowflake
--------  ----------------------------  ----------------------------  ----------------------------
name      Jane Roe                      Darren Ortiz                  Darren Ortiz
email     [email protected]            [email protected]  [email protected]
ssn       078-05-1120                   MzeROCjpFL7                   MzeROCjpFL7
phone     +1 415 555 0142               iLC3dS3BsqZDBVK               iLC3dS3BsqZDBVK
address   221B Baker Street, London     35493 North Trailville Stree  35493 North Trailville Stree

No raw PII values reached the clone or Snowflake.

Testing

• CI matrix: emulator aws and snowflake × image tag latest and dev, using lstk, with postgres:17 as a stand-in clone image. All jobs run the integration tests (clones API, gateway and host port access, Dubs API validation), and the snowflake jobs also run make demo
• New selfhosted-demo job runs make demo-selfhosted against the real Baseshift components, using the BASESHIFT_USER_EMAIL / BASESHIFT_USER_PASSWORD secrets. It takes about 8 minutes
• Verified locally with both deployments (Docker and EKS), including restarts and masking policy changes

Notes

• Each selfhosted-demo run creates a new Dub (ci-<run id>) in the Baseshift org, since Dubs can't be deleted via the CLI. That's why the job is skipped for the nightly schedule and for fork PRs. It would be great to get a delete API from Baseshift
• Baseshift ignores the masking policy passed at creation for self-hosted Dubs, so the extension applies it after the initial snapshot and then deletes the unmasked image from ECR
• The replication server enrolls with a one-time token. With the Docker deployment, its certificates live in Docker volumes and survive LocalStack restarts. With EKS they don't, so a Dub needs a new name after a restart there
• LocalStack RDS doesn't apply the rds.logical_replication parameter, so the demo sets wal_level=logical directly and reboots (the master user is a superuser in LocalStack)
• Running together with another extension that serves Postgres on port 4566 (e.g. ParadeDB) isn't supported, both claim the same handshakes
• Not yet added to CODEOWNERS

🤖 Generated with Claude Code