Networking for private, air-gapped and classified customer environments

Generated by Claude through GigAI 0.1.11.10 on 2026-10-12.

Concept: Networking · also covers Classified and denied environments, Defense and national security · Step 28 of 34 in AI forward deployed engineer · version 1

Many enterprise and government customers run workloads on networks with no public ingress, tightly filtered egress, or no internet connection at all. This lesson covers private connectivity patterns, network controls inside Kubernetes, and what changes when the environment is air-gapped or in a government region. The recurring theme is that every dependency you normally fetch at run time must be either reachable privately or carried in.

How it is done in practice

A team keeps a connectivity matrix for each deployment: every source, destination, port and protocol, with the reason. They test the release in an internal environment that has no internet route and default-deny network policies, which catches most surprises before the customer sees them. For disconnected customers the pipeline emits an offline bundle with checksums alongside the normal release, and the installer reads every registry and endpoint address from configuration. A support bundle command and a written runbook stand in for the remote access the team will not have.

Technologies

Diagram

flowchart LR
  subgraph CustVPC[Customer private network]
    APP[Application pods in private subnets]
    EP1[Interface endpoint: model service]
    EP2[Interface endpoint: registry and storage]
    FW[Egress proxy with allowlist]
  end
  APP --> EP1 --> MODEL[Hosted model service]
  APP --> EP2 --> SVC[Registry and object storage]
  APP --> FW --> CP[Vendor control plane endpoint]
  APP -->|VPN or dedicated circuit| DC[Customer data center systems]

flowchart LR
  subgraph CustVPC[Customer private network]
    APP[Application pods in private subnets]
    EP1[Interface endpoint: model service]
    EP2[Interface endpoint: registry and storage]
    FW[Egress proxy with allowlist]
  end
  APP --> EP1 --> MODEL[Hosted model service]
  APP --> EP2 --> SVC[Registry and object storage]
  APP --> FW --> CP[Vendor control plane endpoint]
  APP -->|VPN or dedicated circuit| DC[Customer data center systems]

Code

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: app-egress
  namespace: assistant
spec:
  podSelector: {matchLabels: {app: assistant-api}}
  policyTypes: [Egress]
  egress:
    - to: [{namespaceSelector: {matchLabels: {kubernetes.io/metadata.name: kube-system}}}]
      ports: [{protocol: UDP, port: 53}]
    - to: [{podSelector: {matchLabels: {app: postgres}}}]
      ports: [{protocol: TCP, port: 5432}]
    - to: [{ipBlock: {cidr: 10.20.0.0/24}}]   # private model endpoint subnet
      ports: [{protocol: TCP, port: 443}]

Subtopics

Private network layout for a customer deployment

Reading the customer's network before you deploy

A virtual private cloud is a logically isolated network with subnets, route tables, gateways and security groups. In a customer environment you rarely design it; you are handed subnets and must discover the rules. Ask which subnets have a route to the internet, whether egress passes through a proxy or firewall with an allowlist, how DNS resolves internal names, and how much address space is free, since pod networking can exhaust small ranges quickly.

Private connectivity: endpoints, peering and VPN

Reaching services without the public internet

AWS PrivateLink lets resources in a VPC connect privately to supported AWS services, to services hosted in other accounts, and to partner services, as if they were inside the VPC, without an internet gateway, NAT device or public address. The consumer creates an interface endpoint, which appears as network interfaces with private addresses in their subnets; the provider exposes an endpoint service, typically fronted by a network load balancer. Traffic is one-directional by design: the consumer initiates, and the provider cannot reach back into the consumer network. That property makes it easier to get approved than peering, which joins two networks at the routing level and requires non-overlapping address ranges. Google Cloud Private Service Connect and Azure Private Link follow the same model.

Choosing a pattern

Use private endpoints for cloud services the workload calls, including storage, the container registry, the secret store and the hosted model service, so the deployment works in subnets with no internet route. Use an endpoint service when the vendor control plane must be reachable from the customer network or the reverse. Use site-to-site VPN or a dedicated circuit when the workload must reach systems in a customer data center. List every outbound destination in the design document, with protocol and port, because that list becomes the firewall change request.

Kubernetes networking and egress control

Services, ingress and policy

The Kubernetes network model gives every pod its own cluster-wide address, and pods can reach each other without address translation unless something restricts them. Services provide a stable address and DNS name in front of a changing set of pods; Ingress and the newer Gateway API expose HTTP routes to clients outside the cluster. NetworkPolicy objects restrict traffic by pod labels, namespaces and address ranges, but only take effect if the cluster's network plugin enforces them. Restricted customers often apply a default-deny policy per namespace, so your chart must ship explicit policies for every flow it needs: application to database, application to model endpoint, DNS, and metrics scraping. A missing DNS egress rule is the classic first failure, and it shows up as timeouts rather than refusals.

Air-gapped delivery

Everything arrives as a bundle

An air-gapped environment has no route to the internet, so nothing can be pulled at install or run time. The release becomes a self-contained bundle: all container images exported as archives, charts, operating system packages, language dependencies, model weights and tokenizer files, plus checksums and signatures. The bundle is transferred through an approved process, scanned, and loaded into an internal registry and artifact store. Hunt down hidden network calls before shipping: telemetry, license checks, libraries that download model files on first use, and fonts or scripts loaded by a web front end from a public network.

Operating without reaching in

Hosted model APIs are unavailable, so inference runs on local GPUs with an open-weight model, and evaluation results from a hosted model do not transfer; rerun the evaluation suite on the model that will actually serve. You cannot attach a debugger or read dashboards remotely, so build a diagnostics command that collects logs, configuration and versions into a single redacted file the customer can review and hand back. Upgrades are infrequent and expensive, which raises the bar on installer idempotence, rollback and written runbooks.

Government regions and classified environments

What a government region changes

AWS GovCloud (US) consists of isolated regions designed for United States government agencies and customers with sensitive workloads and regulatory requirements. Accounts and credentials are separate from standard regions, resource names use a different partition, endpoints differ, and not every service or feature is available or arrives at the same time. Practically, hard-coded partitions, region names and endpoints in infrastructure code break, and a managed service you depend on may be missing, including specific hosted models. Parameterize partition and endpoints, and check service availability during scoping rather than at install.

Classified networks

Classified environments add physical and personnel constraints on top of air-gapping: work happens on site in accredited facilities by cleared staff, no personal devices or outside connectivity are allowed, and software enters through a formal review and transfer process. The system needs an authorization to operate, which rests on documented security controls. Expect slow iteration loops, so invest in a faithful unclassified replica environment where the installer and upgrade path are rehearsed end to end before each visit.

Sources

• docWhat is Amazon VPC? (docs.aws.amazon.com)
• docWhat is AWS PrivateLink? (docs.aws.amazon.com)
• docKubernetes: Services, Load Balancing, and Networking (kubernetes.io)
• docWhat is AWS GovCloud (US)? (docs.aws.amazon.com)

Related lessons

• Deploying into customer clouds: AWS, Google Cloud and Azure
• Containers, CI/CD and serverless delivery
• GPU infrastructure and model serving for inference workloads
• Security reviews, governance and compliance in regulated industries