GPUs

Loomplane finds GPUs, schedules work onto them and shows what they cost. It never installs a GPU driver, and it never installs the NVIDIA device plugin on your cluster. Whatever the target is, the driver side is yours to set up (DigitalOcean GPU Droplets are the one exception, covered below). Everything on this page is about getting that side right so Loomplane can see the GPUs.

Prices are estimates. When Loomplane doesn't know a price it says "price unknown" and never shows $0. And GPU workloads on your own servers and clusters are never stopped or scaled down for you, since nothing here can tell an idle GPU from a busy one.

KubernetesCopy Anchor Link

Install the NVIDIA device plugin or the NVIDIA GPU Operator first. Without one, the nodes have GPUs but nothing advertises nvidia.com/gpu, and the connection says so and asks you to install one.

Then let the connection list nodes. Loomplane reads the cluster's nodes to find GPUs, and a namespace-scoped credential can't do that. Without the permission, the connection reads "Loomplane can't list this cluster's nodes, so it can't see GPUs". Fix it with a ClusterRole and a binding to the service account behind your kubeconfig token (swap in your own names):

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: loomplane-nodes
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: loomplane-nodes
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: loomplane-nodes
subjects:
  - kind: ServiceAccount
    name: loomplane
    namespace: loomplane

Check the connection again afterwards.

Each GPU node also needs a label that says what kind of GPU it has. Loomplane tries these in order and uses the first one a node carries:

1. cloud.google.com/gke-accelerator
2. nvidia.com/gpu.product (GPU feature discovery adds this one)
3. k8s.amazonaws.com/accelerator
4. doks.digitalocean.com/gpu-model

If none is there, it falls back to the cloud's node.kubernetes.io/instance-type label, which only works when Loomplane knows that instance size. A cluster with GPUs but no label that names the kind shows a message asking you to add nvidia.com/gpu.product. A cluster with no GPU nodes shows No GPU nodes.

You don't have to touch taints. Loomplane turns the taints that every matching GPU node carries into tolerations on the pod. A taint on only some of those nodes isn't tolerated, and neither are the ones Kubernetes sets for a node's state (cordoned, not ready, unreachable), so a GPU pod never lands on a cordoned node or stays on a dead one. A GPU pod gets an nvidia.com/gpu limit and a node selector built from the label above, so it only lands on the kind you asked for. Wherever Loomplane lists GPU targets, a cluster's count reads as GPUs per node, not GPUs free, because Loomplane doesn't count what the cluster's other pods are using.

Anything persistent that Loomplane runs on the cluster needs a storage class with a Retain reclaim policy and a declared binding mode, and that covers catalog databases and any other volume, not only a model weight cache. Pick the class on the connection and Loomplane checks it. If the cluster has no such class, the plan is refused until you add one.

Your own serversCopy Anchor Link

A server needs the NVIDIA driver and the NVIDIA Container Toolkit. Once both are in, tell Docker to use the toolkit and restart it:

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Loomplane runs nvidia-smi over SSH when you connect the server and again on every check, so it learns what GPUs the server has and whether Docker can use them. Open the server and choose Check again after any change.

When a server has GPUs it can't use yet, it shows Can't take GPU work with the reason. There are two common ones. If nvidia-smi doesn't answer on a machine with an NVIDIA card, you need the driver. If it answers but Docker can't reach the GPUs, you need the toolkit steps above.

Each container gets its own GPUs, picked when the service is deployed. Two workloads never share a device, so a server with two GPUs runs at most two single-GPU services.

DigitalOcean GPU DropletsCopy Anchor Link

DigitalOcean is the only cloud where Loomplane creates GPU servers today. Open your Cloud, go to Servers and choose Create server with a DigitalOcean account that can provision. The size list has a GPU group, and each GPU size shows its GPUs, the memory on each one and the hourly price. Pick one and the image list narrows to GPU-ready images, which ship the driver. Review server shows the estimated hourly and monthly cost before you approve anything, and the price stays on the server afterwards.

At first boot Loomplane installs Docker only if the image doesn't have it, then installs the NVIDIA Container Toolkit and points Docker at it.

If the toolkit setup fails, the server says so, takes no GPU work and keeps costing money. Nothing retries by itself. The message gives you the commands to run (the nvidia-ctk steps above) and then Check again, or you can delete the server.

You can put a GPU server in a pool from Pools and reservations. Give the pool a GPU kind and each server a GPU budget, which is how many of its GPUs the pool may hand out. A GPU pool refuses CPU work, and a CPU pool can't hold GPU budgets.

Pools can also grow by creating GPU Droplets themselves, inside a GPU ceiling for each kind. You configure that growth and those ceilings through the Cloud's capacity policy, which has no screen in the dashboard yet. None of it happens until the operator sets LOOMPLANE_AUTOMATIC_CAPACITY_ENABLED=true (see Configuration), which is off by default while it's being qualified. When they are set, a few rules apply:

• A kind with no Cloud ceiling allows no GPUs at all.
• Each GPU pool has its own ceiling for its kind, and it can't go above the Cloud's.
• If one server's GPUs can't be read, growth of GPU capacity pauses until a check on that server succeeds. Nothing buys another GPU on a guess.

ModelsCopy Anchor Link

With the AI section on, you don't have to write a GPU request yourself to run an open model. The Models page lists models with where each one fits, and deploying one picks the GPU kind and count, writes the resource into your manifest and plans it. It can run on any server or cluster above that Loomplane can see GPUs on, or on a DigitalOcean GPU Droplet it creates for the model after an admin reviews the price. See Models and What models cost.

AWS, GCP and AzureCopy Anchor Link

GPU nodes on EKS, GKE and AKS clusters are recognized by their instance type and priced from reference prices, which are estimates. Loomplane doesn't create GPU virtual machines on these clouds yet. You bring the cluster and Loomplane places work on it, using the Kubernetes rules above.

In a manifestCopy Anchor Link

Ask for GPUs on a service with size.gpu. Add size.shm (in MiB) if your framework wants a bigger /dev/shm, which PyTorch data loaders often do. Shared memory counts toward memory, so it has to be smaller.

services:
  inference:
    source: services/inference
    provider: kubernetes
    target: conn_cluster
    size:
      cpu: 4
      memory: 32768
      shm: 8192
      gpu: { kind: nvidia-l4, count: 1 }

Catalog resources take the same gpu field under resources.<name>, with kind and count, and only NVIDIA kinds. The model kinds (model-vllm and model-llamacpp-gpu) are the ones that use it today. Models shows what a model resource looks like.

count runs from 1 to 8. kind is one of these IDs:

Planning checks the kind against the target, so a request the target can't meet shows up as a problem on the plan before anything runs. Arm64 targets can't run GPU work, and a gateway can't ask for GPUs.