The Control Plane for GPUaaS
Run GPU workloads on any NeoCloud or hyperscaler or your own infrastructure through one Kubernetes API.
Nebula turns external GPU capacity into ordinary Pods: ask for an accelerator, and it picks a provider, provisions the instance, and reclaims it when you're done — no per-cloud glue. The API follows a Karpenter-style split:
- NodePool — policy: which providers are allowed, how to pick between them (cost / availability), failover behaviour, and the GPU shape.
- NodeClaim — one provisioned instance and its lifecycle, held by a finalizer so a paid instance is never leaked.
- Kubernetes-native experience. Existing Deployments or Jobs opt in via labels; no new workload type and no per-cloud glue.
- Multi-provider support. Modal, RunPod and AWS today, with more providers coming soon.
- Policy-driven placement. Flexible NodePool spec to pick providers by cost, availability, and region, with failover and network egress policy.
- Cost tracking. Out-of-the-box per-instance cost tracking.
- Observability. Rich metrics and logs for placement, provisioning, and instance lifecycle.
- Opt in. You label a Pod to request an accelerator.
- Gate. A mutating webhook injects a scheduling gate (and a toleration for the
virtual node) at Pod CREATE, so the Pod sits
SchedulingGated. - Place. The placement controller picks a provider from the matching NodePool,
stamps its
nodeSelector, and lifts the gate — or leaves the Pod gated if no provider can serve it. - Provision. The Pod binds to that provider's virtual node; a per-provider virtual kubelet spins up the real instance and reports status (phase, endpoint) back onto the Pod.
- Reclaim. A NodeClaim tracks the instance and guarantees teardown via a finalizer — even if the Pod is force-deleted while the virtual kubelet is down.
A NodePool declares which providers may serve a workload and how to pick between
them. Workloads target it by name via the nebula.inftyai.com/nodepool label. A
single pool can span multiple providers, so a workload fails over across clouds:
apiVersion: nebula.inftyai.com/v1alpha1 kind: NodePool metadata: name: gpu spec: providers: - name: modal # NeoCloud; regions omitted = place anywhere (cheapest) - name: runpod # NeoCloud, OnDemand only; a region is a geography regions: # ("us") or one data center ("EU-RO-1") - us - name: aws # hyperscaler; "us" expands to every US region regions: - us - eu-west-1 # or name one region exactly capacityTypes: # prefer cheap Spot, fall back to OnDemand - Spot - OnDemand strategy: Ordered # try providers in listed order (or LowestPrice) failover: blocklistTTL: 10m # how long a failed placement is skipped egress: # outbound policy; omitted = Open mode: Allowlist # or Blocked to permit nothing targets: # CIDRs, bare IPs and domains, wildcards allowed - 10.0.0.0/8 - "*.huggingface.co"
Three labels on the Pod template (not the Deployment metadata) are all it takes; Nebula fills in the rest:
metadata: labels: nebula.inftyai.com/enabled: "true" # opt in nebula.inftyai.com/nodepool: gpu # which NodePool to place against nebula.inftyai.com/accelerator-type: h100 # GPU type (case-insensitive) annotations: nebula.inftyai.com/regions: eu,uk # optional: narrow within the pool spec: containers: - name: workload image: nvidia/cuda:12.4.1-base-ubuntu22.04 resources: limits: nvidia.com/gpu: "8" # GPU count
# 1. Namespace and provider credentials. Create them first — the manager reads them as env # on startup. MODAL_ENVIRONMENT is optional; see docs/deploy.md for the other providers. kubectl create namespace nebula-system kubectl create secret generic nebula-modal-credentials -n nebula-system \ --from-literal=MODAL_TOKEN_ID=ak-... \ --from-literal=MODAL_TOKEN_SECRET=as-... # 2. CRDs, manager and webhook. Server-side apply, because the CRDs are too large for the # last-applied annotation. The manager provisions its own webhook cert — no cert-manager. kubectl apply --server-side -f https://github.com/InftyAI/Nebula/releases/download/v0.1.0/install.yaml
One virtual node appears per provider whose credentials are present:
kubectl get nodes -l nebula.inftyai.com/provider
See docs for more detailed instructions.
Apache-2.0 — see LICENSE.