GeezerVison

A new thread in the Amp project sourcegraph/deploy-sourcegraph-docker gets a running Sourcegraph, started from this repository's Compose files, behind a portal URL, with a site admin already created.

What a thread gets

• Sourcegraph with Jaeger and an executor, from docker-compose/, docker-compose/jaeger/, and docker-compose/executors/. Those two are the only optional deployments in the repository which need nothing outside it.
• A "Sourcegraph" link in the thread's Portal tab, plus links to the Executors page and the Jaeger UI.
• A site admin named admin. Its password shows as a note next to the Sourcegraph link, and is in ~/.config/sourcegraph/admin-password in the orb. The agent in the thread is told to give the user the URL, the username, and the password.

How it works

• .agents/setup runs when Amp builds the project's snapshot. It starts Docker, installs the Compose plugin if it is missing, pulls the images, and adds a short section on running Sourcegraph to the orb's AGENTS.md.
• .amp/services.yaml declares the sourcegraph service. Amp starts it when someone opens the Portal tab, or runs amp orb services ensure.
• .amp/sourcegraph.sh is that service. It starts the containers, waits for the frontend, creates the admin, configures the site when its configuration has no executor token, shows the password in the Portal tab, then follows the logs until Amp stops the service.
• .amp/docker-compose.override.yaml makes Caddy publish the port Amp assigned and port 80, gives the executor its token and its queues, and lowers the CPU limit of the two zoekt services to 4.

Risk: none for customers. Nothing under docker-compose/ changes. The files only take effect in an Amp orb, and only once they are on main.

Things to know

• The repository holds no password or token. Each orb makes its own.
• Caddy publishes the port Amp assigned, and port 80. The Amp service cannot use port 80 itself, because Amp checks a service's port as a user who cannot open it. Port 80 is for the containers the executor starts. Caddy does not publish 443, where Amp's own portal proxy listens.
• The orb needs at least 4 vCPUs. zoekt-indexserver-0 and zoekt-webserver-0 set cpus: 8, and Docker refuses more than the machine has, so the override lowers both to 4, in every orb size. The project's default orb has 16 vCPUs and 31 GB.
• After a restart, the Executors page also lists the executor from before the restart, as inactive. The executor registers under a new name on each start.
• The script waits only for the frontend, not for every service, because searcher-0 reports unhealthy for a few seconds while it starts. ci: Make the smoke test wait for health checks, and check more #1487 fixes that.
• The executor takes jobs from all three queues: batches, batches-hooks, and codeintel. It gets them through EXECUTOR_QUEUE_NAMES, because EXECUTOR_QUEUE_NAME takes one name only, and the executor refuses to start when both are set.
• When the script adds the executor token, it writes the site configuration back as plain JSON, so comments in it are lost. That happens on the first start, and again only if someone removes the token. A site configuration the script cannot parse is left alone.
• Only people who can open the thread can open its portal.
• The script sets five things in the site configuration: externalURL (the portal URL), executors.accessToken, executors.frontendURL, codeIntelAutoIndexing.enabled, and observability.tracing.
  • executors.frontendURL is http://host.docker.internal. The containers the executor starts for a job cannot use the portal URL: they reach it from outside Amp's sign-in, and get an HTML page. So the executor sets EXECUTOR_DOCKER_ADD_HOST_GATEWAY=true, and they reach Caddy on the orb's port 80. Sourcegraph labels that setting "Can be insecure": job containers can reach the orb's ports.
  • codeIntelAutoIndexing.enabled lets index jobs be queued. With the default policies, nothing gets indexed on its own.
  • Sourcegraph exports a trace only for a request made with ?trace=1.
• Not tested: the pushed tip in a 4 vCPU orb (the 4 vCPU test ran the same override change at b4347d6), publishing changesets, and a start with a site configuration the script cannot parse.
• Not tested: a job on the batches-hooks queue. Those jobs come from changeset events, so they need published changesets on a code host with a write token.

Checklist

• All images have a valid tag and SHA256 sum (no image changes)

Test plan

Run in an Amp orb (16 vCPUs, 31 GB, Docker Compose 5.7.1), at b4347d6, with the 8.2.0 images:

• Setup takes 40 seconds on a new orb, and 2 seconds when the images are already there. It leaves no process behind. The 24 images take 8.3 GB.
• The first start takes 36 seconds. 27 of 27 containers run, with no restarts, and none unhealthy. At idle the orb uses 3.3 GB of memory.
• The admin signs in through the portal URL in a browser.
• The Executors page shows the executor as active, with batches, batches-hooks, and codeintel. Its logs have no configuration error.
• A restart takes 37 seconds, and the executor is active again afterwards. The site configuration keeps its ID, and a setting added by hand survives.
• With the executor token removed from the site configuration, a restart puts it back, keeps the setting added by hand, and the executor is active again.
• The Jaeger UI loads through the portal, signed in as the admin, and its API answers.
• A server-side batch change runs on the executor: one step in alpine:3 on github.com/sourcegraph/jsonrpc2 completes in 2 to 4 seconds, with a one-line diff.
• An auto-indexing job did not run. Sourcegraph refused to queue it until codeIntelAutoIndexing.enabled was set. With that set, scip-go ran, and the upload step failed: its container got the portal URL, and received an HTML page.
• Sourcegraph sent Jaeger no traces.
• A stop leaves no containers and no portal links.
• bash -n and shellcheck 0.11.0 are clean on both scripts.

Run in the same orb with the change in commit 6f9d138, on new volumes, before it was committed:

• The first start takes 34 seconds, and writes the settings. 27 of 27 containers run, with no restarts. A restart keeps the site configuration's ID.
• An auto-indexing job on github.com/sourcegraph/jsonrpc2 completes in about 15 seconds: scip-go runs, the upload reaches http://host.docker.internal, and Sourcegraph processes the index.
• The server-side batch change still completes.
• A search request with ?trace=1 shows in Jaeger with 93 spans from frontend, searcher, gitserver, and zoekt-webserver, and the "View trace" link in the search results opens it through the portal. A request without ?trace=1 adds no spans.

Run in the same orb at the pushed tip, 48a8471, on new volumes:

• The first start takes 40 seconds. 27 of 27 containers run, with no restarts. searcher-0 reports unhealthy for a few seconds while it starts, as before.
• Both zoekt containers have a 4 CPU limit. Zoekt indexes github.com/sourcegraph/jsonrpc2 within 30 seconds of the repository being added, and an index:only search returns a match from the index.
• An auto-indexing job and a server-side batch change both complete. The batch change waits for the index job, because the executor runs one job at a time.
• Precise code navigation works on the index: a hover shows the type and its doc comment with the "Precise" badge, and go to definition and find references (11) answer.
• A search request with ?trace=1 shows in Jaeger with 64 spans, from frontend and zoekt-webserver.
• A restart takes 38 seconds, and keeps the site configuration's ID. The executor is active again, and the search still answers from the index.
• bash -n and shellcheck are clean on both scripts.

In Buildkite, build 16911 of 48a8471 passed. The smoke test job took 519 seconds; this PR changes no file it uses.

Run in two smaller orbs, at b4347d6, with the 8.2.0 images:

• 8 vCPUs, 16 GB: works unchanged. Setup takes 26 seconds, the first start 40 seconds, and a restart 35 seconds. 27 of 27 containers run, with no restarts, none unhealthy, and none killed for memory, also 5 minutes after the restart. The admin signs in through the portal, and the executor is active with all three queues. At idle the orb uses 3.1 GB of memory.
• 4 vCPUs, 8 GB: did not work. docker compose up failed with range of CPUs is from 0.01 to 4.00, as there are only 4 CPUs available for the two zoekt services, and no container was created. With the change in commit 48a8471, before it was committed, all of the above works there too, and the orb uses 2.8 GB at idle.

Run in the 16 vCPU orb before the rebase, at 696bd15, with the 8.1.4873 images, and not repeated on 8.2.0:

• The service recovers from a killed docker compose up, and from a start while Docker still holds the old port.
• A stop takes 11 seconds.

Closes FE-600