GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GitHub - google/xmc

BranchesTags
Open more actions menu

Latest commit

 

History

1,179 Commits
1,179 Commits

Folders and files

NameName
Last commit message
Last commit date
.agents
.agents
 
 
.githooks
.githooks
 
 
.github
.github
 
 
.vscode
.vscode
 
 
api
api
 
 
cmd
cmd
 
 
deploy
deploy
 
 
docs
docs
 
 
examples
examples
 
 
internal
internal
 
 
pkg
pkg
 
 
scripts
scripts
 
 
sdks/python
sdks/python
 
 
test
test
 
 
tools
tools
 
 
.bazelrc
.bazelrc
 
 
.env.example
.env.example
 
 
.gitignore
.gitignore
 
 
.golangci.yml
.golangci.yml
 
 
.tflint.hcl
.tflint.hcl
 
 
.trivyignore
.trivyignore
 
 
ABSENCE.md
ABSENCE.md
 
 
BUILD.bazel
BUILD.bazel
 
 
CONTRIBUTING.md
CONTRIBUTING.md
 
 
INSTALL.md
INSTALL.md
 
 
LICENSE
LICENSE
 
 
MODULE.bazel
MODULE.bazel
 
 
MODULE.bazel.lock
MODULE.bazel.lock
 
 
OWNERS
OWNERS
 
 
README.md
README.md
 
 
SECURITY.md
SECURITY.md
 
 
go.mod
go.mod
 
 
go.sum
go.sum
 
 
nogo_config.json
nogo_config.json
 
 
oci.bzl
oci.bzl
 
 
pyproject.toml
pyproject.toml
 
 

Repository files navigation

XManager on Cloud (XMC)

License

XManager on Cloud (XMC) serves as a central command platform for both human researchers and autonomous agents building AI. Built on the experiment model of Google DeepMind's battle-tested XManager, XMC is designed to operate at scale to boost research velocity across AI labs and enterprise organizations.

Key capabilities

Capability What XMC provides
Workload execution & agent integration Reliable workload launching alongside first-class agent interaction capabilities.
Persistent experiment storage High-performance, persistent storage for experiment metadata and research artifacts.
Resource visibility & scheduling Out-of-the-box visibility into TPU and GPU availability and scheduling states, with full compatibility and configurable support for Kueue.
Kubernetes-native deployment Built on standard Kubernetes primitives (Kueue, JobSet). GKE is the supported, documented target; other Kubernetes environments (other clouds, on-premises) are possible with additional configuration.
Metrics & profiling (optional) Training metric streaming with charts and dashboards, and Managed XProf hardware profiles attached to work units. Both are off by default.

Note

XMC spans two repositories. This repository holds the XMC services, SDKs, and deployment charts. The Python launcher and xmanager CLI that define and submit experiments live in google-deepmind/xmanager (see Getting started for installation).

How it works

flowchart LR
    Client["Humans & agents<br/>(xmanager CLI, Python, APIs)"] -->|"gRPC / HTTPS"| CP["XMC control plane"]
    CP -->|"Submits JobSets"| K8s["Kubernetes + Kueue<br/>(GPU / TPU / CPU)"]
    K8s -->|"Status"| CP
    CP --> UI["XMC web UI"]
    K8s -.->|"Metrics (optional)"| MP["XMC metrics pipeline"]
    MP -.-> UI
Loading

XMC organizes work around three concepts:

Concept What it represents
Experiment (xid) A group of related runs, such as a sweep, an ablation, or an RL campaign, with shared priority, metadata, and access control.
WorkUnit (wid) One trial or training job inside an experiment. Restarting a work unit creates a new execution and keeps earlier attempts in history.
Artifact (aid) A typed output or link attached to an experiment or work unit: URLs (such as TensorBoard or checkpoint locations), code snapshots, text notes, metric charts, or Managed XProf profiles.

The Architecture Overview and Execution Model cover the services, the execution lifecycle, and how XMC maps work units to Kueue and JobSet.

Getting started

You can deploy XMC to GKE or evaluate it locally on KinD in three steps:

  1. Stand up an XMC environment
    • For GKE deployments, provision a stack with Cluster Toolkit v2 or Terraform and Helm using the Installation & Deployment Guide.
    • For local evaluation, boot a KinD control plane with python3 tools/dev_env/dev_env.py up (requires a GCP project for Cloud Spanner, gcloud, and bazel; see Local & Hybrid Development).
  2. Install the XManager launcher and sign in The launcher and xmanager CLI come from google-deepmind/xmanager, published on PyPI:
    pip install 'xmanager>=0.8.5.post1'
    export XMC_DOMAIN=<your-xmc-domain>
    xmanager login
    When working from a clone of this repository with local protobuf changes or xdash, run python3 deploy/cli/install_xmanager.py instead (see CLI Setup).
  3. Run an example workload Launch a multi-WorkUnit job from one of the ready-to-run templates in examples/:
    • examples/simple_job: multi-WorkUnit CPU/GPU sweep on a Kueue LocalQueue.
    • examples/multipod_job and examples/pathways_job: distributed multi-node and Pathways workloads.
    • examples/metrics_dashboard_job and examples/xdash_dashboard: live metric streaming and programmatic dashboards.
    • examples/managed_xprof_artifact_job: Managed XProf hardware profiling attached to a work unit.
    • examples/xmc_lite_job: standalone xm_local.Kubernetes launch without the XMC server.

Metrics streaming and Managed XProf are optional and off by default. Enable metrics by deploying the Redpanda and ClickHouse add-ons in deploy/addons/metrics and enabling redpanda, clickhouse, and ingestion-gateway in the stable-2026a Component profile (xmc-profile); enable Managed XProf with managed_xprof_gcs_bucket (managedXprofGcsBucket in Helm; see the Cluster Toolkit v2 README).

Documentation

Operations and usage

  • XManager launcher (google-deepmind/xmanager): Python launch API and xmanager CLI that define and submit experiments to XMC.
  • Installation & Deployment Guide: Cluster Toolkit v2 blueprints, Terraform and Helm setup, DNS and TLS configuration, and upgrades.
  • Authentication & Experiment Access: interactive xmanager login, CI service accounts, and experiment ACL sharing.
  • Python Metrics SDK (xmc) and Dashboard SDK (xdash): in-job metric logging and dashboard definitions.

Architecture

  • Architecture Overview: control, execution, telemetry, and component planes, followed by Component Reference, Execution Model, and Use Cases.
  • Main Flows: sequence walkthroughs for experiment lifecycle, work unit actuation, reconciliation, and metrics & observability.
  • Deployment Topologies and Gateway Routing Design: Cluster Toolkit Blueprint v1 vs v2 layouts, Kubernetes Gateway API routing, OAuth2 session handling, and identity mappings.

Contributing

  • Contributing Guide: Google CLA requirements, issue templates, and pull request workflow.
  • Local & Hybrid Development: KinD inner loop (dev_env up, dev_env reload) and hybrid --remote-k8s execution.
  • Developer Guide, E2E & UI Testing, and Agent Workflows.
  • Automation scripts under ./tools and ./deploy require Python 3.13.14 or newer:
    python3 ./tools/common/setup_venv.py --venv-dir ./tools/venv --requirements ./tools/common/requirements.txt

Contact and support

  • For questions about XMC, deployment guidance, or Google Cloud Managed Training Clusters, email [email protected] or reach out to your Google Cloud account team.
  • File bug reports and feature requests in GitHub Issues. Self-hosted deployments receive best-effort support without contractual SLAs; for a managed offering with contractual SLAs, see Google Cloud Managed Training Clusters.
  • Report security vulnerabilities privately through g.co/vulnz rather than in public issues.
  • Released under the Apache 2.0 License.

About

No description, website, or topics provided.

Resources

Readme
Apache-2.0 license

Code of conduct

Code of conduct

Contributing

Contributing

Security policy

Security policy
Activity
Custom properties

Stars

8 stars

Watchers

2 watching

Forks

0 forks
Report repository

Releases

Packages

Used by

Contributors

Languages

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.