The ML platform: clusters, GPUs and training

What MLOps adds to software delivery; Kubernetes and EKS as the substrate; experiment tracking; GPUs as cluster resources; multi-node training; scheduling and tenancy of a shared cluster.

This module covers the substrate of an ML platform and its training side. It starts with what MLOps adds to ordinary software delivery and the Kubernetes and AWS (EKS) foundations the rest runs on. It then covers training infrastructure: how runs are recorded with experiment tracking, how GPUs are exposed to Kubernetes, how one job spans many nodes, and how a shared cluster decides who runs when. It closes with the system design needed to share the platform safely across many tenants.

Lessons (8)

• MLOps and the ML lifecycle: from experiment to production system

  MLOps applies DevOps discipline to systems whose behavior depends on data and trained artifacts as well as code.

• Kubernetes for ML workloads: containers, deployments and jobs · Practice

  Kubernetes is the default substrate for ML platforms because it schedules containers across a shared pool of machines and keeps declared state true.

• AWS and EKS foundations for an ML platform · Practice

  Amazon EKS is the managed Kubernetes service many ML platforms are built on: AWS runs the control plane and the team owns the workloads, the node strategy and the integrations.

• Experiment tracking with MLflow · Practice

  Experiment tracking records what was run, with which inputs, and what came out, so any model can be traced back to its code, data and hyperparameters.

• GPU compute on Kubernetes with the NVIDIA stack · Practice

  Kubernetes does not understand GPUs natively; it learns about them through a device plugin that advertises them as an extended resource.

• Distributed training across multiple GPU nodes · Practice

  When a model or dataset is too large for one GPU, training is spread across several GPUs and several nodes that must start together, find each other and stay in sync.

• Workload scheduling, capacity management and GPU cluster operations · Practice

  A shared GPU cluster has more demand than capacity, so something must decide which jobs run, in what order, and on whose budget.

• System design for multi-tenant ML infrastructure: isolation, quotas and backpressure · Practice

  A shared ML platform runs training, batch and serving workloads from many teams on common clusters and accelerators.

Changes

• Version 1, 2026-10-14: re-cut from the role pathway groups into a topic module (site 2026.10.2); no lesson changed