Practice: GPU compute on Kubernetes with the NVIDIA stack
Generated by Claude through GigAI 0.1.11.10 on 2026-10-11.
Main tool: Kubernetes · Every piece of the GPU stack plugs into Kubernetes primitives (the kubelet device plugin API, extended resources, labels, taints and DRA claims), so practising those primitives on a local cluster teaches what the vendor stack automates even without a GPU. · Alternatives: NVIDIA GPU Operator installs driver, container toolkit, device plugin, feature discovery and DCGM Exporter in one Helm release instead of by hand; the standalone k8s-device-plugin covers only advertising and time-slicing or MPS config; a DRA GPU driver replaces counted resources with claims; managed cloud node pools hide the whole stack; Slurm or Ray schedule GPUs outside Kubernetes.
Every step links into the tool's own documentation; links were checked on 2026-10-11 · version 1
Setup (one time)
1. Install kubectl and start a local multi-node cluster with kind or minikube, with one control plane and two workers.
Every later step needs a cluster with more than one node, so that scheduling choices between a device node and a plain node are visible. A local cluster costs nothing and needs no GPU.
Explained in: Assign Pods to Nodes | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: kubectl get nodes lists three nodes in Ready state.
2. Check that the Dynamic Resource Allocation API groups are served by your cluster version, and enable them if they are not.
The sharing and partitioning steps later use DRA objects, which only exist when the API is enabled. Finding this out now avoids a confusing failure halfway through.
Explained in: Set Up DRA in a Cluster | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: Listing DeviceClasses returns an empty list rather than an unknown resource error.
3. Install an example DRA driver that publishes simulated devices, as the local substitute for a real GPU driver.
A laptop has no GPU, so a simulated driver stands in for the vendor one. It publishes devices through the same API a real GPU driver uses.
Explained in: Set Up DRA in a Cluster | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: Listing ResourceSlices shows simulated devices published for at least one worker node.
Steps
1. Read how a device plugin registers with the kubelet and advertises devices as an extended resource.
This is the layer that makes Kubernetes aware of GPUs at all. Driver, container toolkit and runtime sit below it, and the scheduler only sees what the plugin reports.
Explained in: Device Plugins | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: You can draw the chain from pod to scheduler to kubelet to device plugin to runtime to driver, and say what each layer adds.
2. Run a check of which container runtime each node uses and confirm it is containerd and healthy.
The container toolkit works by reconfiguring the node runtime so containers can see the device. You need to know which runtime the kubelet talks to before reasoning about that layer.
Explained in: Changing the Container Runtime on a Node from Docker Engine to containerd | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: The wide node listing shows containerd as the runtime on every node and all nodes are Ready.
3. Deploy a sample device plugin as a DaemonSet on one worker; with no GPU, use one of the listed example plugins that advertises a fake device.
Running the plugin as a DaemonSet is how the real GPU plugin is deployed. Watching a new resource appear on the node shows the advertising mechanism directly.
Explained in: Device Plugins | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: Describing the worker node shows the new extended resource under Capacity and Allocatable with a non-zero count.
4. Create a pod that requests one unit of the advertised extended resource in its container limits.
This is how a workload asks for a GPU: a whole-number limit on a vendor resource name. The scheduler then places the pod only on a node that advertises it.
Explained in: Device Plugins | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: The pod is Running on the worker that has the plugin, and the node description shows one unit allocated.
5. Create a second pod that asks for more devices than exist, save both pod objects to files, and compare their status fields.
Saved pod objects are the record you compare when a GPU workload will not start. The status conditions say whether the cause is scheduling, admission or the container itself.
Explained in: Pod | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: You can point to the Unschedulable condition and its message in the pending pod and to the Ready condition in the running one.
6. Read the vendor implementations section and map each GPU Operator component onto a piece you deployed by hand; installing the operator needs a real GPU node, so the sample plugin stays as the local substitute.
The operator bundles driver, toolkit, device plugin, feature discovery and monitoring into one install. Knowing the manual equivalent of each makes operator failures much easier to debug.
Explained in: Schedule GPUs | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: You have a short table listing each operator component, the layer it covers, and what you did manually in its place.
7. Label the device node with a made-up accelerator model and schedule a pod onto it with a node selector, copying what automatic node labelling does.
Feature discovery labels nodes with GPU model and memory so workloads can pick the right hardware. Doing it by hand shows what those labels are for.
Explained in: Schedule GPUs | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: The pod lands on the labelled node, and changing the selector to a model that does not exist leaves a new pod Pending.
8. Configure a NoSchedule taint on the device node and add a matching toleration only to the pods that request the device.
GPU nodes are costly, so ordinary pods must be kept off them. A taint plus toleration is the standard way to dedicate nodes with special hardware.
Explained in: Taints and Tolerations | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: A plain pod with no toleration never schedules onto the tainted node, while the device pod with the toleration still runs there.
9. Create a DeviceClass and a ResourceClaimTemplate for the simulated devices, then run a workload that references the claim.
DRA is the newer way to request GPUs. A claim describes the device you need by attributes rather than by count, which is what makes flexible sharing possible.
Explained in: Allocate Devices to Workloads with DRA | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: A ResourceClaim generated for the pod shows as allocated and reserved, and the pod is Running.
10. Read how DRA models partitionable devices, then note which sharing mode each maps to: MIG as partitions, time-slicing and MPS as shared access to one device; these modes need real GPU hardware, so the simulated driver is the local substitute.
Sharing a GPU is a trade between isolation and density. Seeing how partitions are modelled in the API explains why MIG gives hard isolation while time-slicing and MPS do not.
Explained in: DRA Features | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: You can state for time-slicing, MPS, MIG and DRA what is isolated, what is shared, and how a pod asks for it.
11. Reproduce the whole flow on a freshly created cluster from saved manifests only, in the order the DRA user workflow describes.
A platform engineer delivers GPU setup as versioned manifests, not as a sequence of typed commands. Rebuilding from files proves nothing depended on manual state.
Explained in: How DRA Works | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: After deleting and recreating the cluster, applying the manifest directory alone brings the device pod and the claim-based pod to Running.
12. Configure least-privilege RBAC for the DRA driver service account, following the node-local driver pattern, and re-run the claim workload.
On a shared production cluster a device driver runs on every node with access to claim status. Restricting its permissions limits the damage from a compromised node.
Explained in: Hardening Guide - Dynamic Resource Allocation | Kubernetes (kubernetes.io), checked 2026-10-11
Done when: The claim-based pod still reaches Running, and an access check shows the driver account cannot update claim status for other nodes.