Practice: Load balancing and service discovery
Generated by Claude through GigAI (karthik446/gigai-learning-center 4c0cb514) on 2026-10-13.
Main tool: Envoy · Envoy exposes every concept in this lesson (balancing policy, active and passive health checking, discovery type, subsets, zone awareness and draining) as an explicit, documented setting you can change and observe on a laptop with containers. · Alternatives: NGINX and HAProxy are standalone proxies configured mainly through files rather than discovery APIs; Kubernetes Services give simple built-in virtual IP balancing; Istio and Linkerd are service meshes that manage sidecar proxies for you (Istio uses Envoy underneath); Consul and ZooKeeper are registries that supply discovery data but do not balance traffic; gRPC client load balancing puts the balancer inside the caller with no proxy hop; a managed cloud load balancer is operated by the provider and cannot run on a laptop.
Every step links into the tool's own documentation; links were checked on 2026-10-13 · version 1
Setup (one time)
1. Install a container runtime with a compose tool, pull a pinned Envoy image, and read the load balancing introduction to learn the terms cluster, host and endpoint.
Everything in this path runs as local containers, and the vocabulary in the introduction is used by every later configuration section.
Explained in: Load Balancing — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: The Envoy container prints its version and you can explain in one sentence each what a cluster and a host are.
2. Create three tiny HTTP backend containers that each return their own name, plus a way to make one of them slow or return errors on demand.
You can only see balancing, ejection and draining when responses are attributable to a backend and you can break one deliberately.
Explained in: Overview — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: Calling each backend directly returns a different name, and you can switch one into a slow or failing mode.
3. Start Envoy from a bootstrap file with one listener, one route and one cluster that lists the three backends as static endpoints, with the admin interface enabled.
A static cluster is the simplest discovery type and gives a known baseline before anything dynamic is added.
Explained in: Service discovery — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: The admin clusters view lists three hosts and a request to the listener returns a backend name.
Steps
1. Run a few hundred requests through the Envoy listener and count how many each backend answered, then change the endpoint weights and repeat.
This is the first run: it shows the default round robin behavior and how weights shift the split.
Explained in: Supported load balancers — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: Counts are near equal with equal weights and follow the ratio you set after changing weights.
2. Read the distributed and global load balancing sections and write down where your Envoy sits (edge proxy or sidecar, connection level or request level) and which decisions it makes locally.
Placement and layer decide what the balancer can see and which failures it can hide, so this covers the lesson's layers and placement subtopic before you tune anything.
Explained in: Overview — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: You have a short note naming the layer and position of your proxy and one decision that a global control plane would make instead.
3. Configure the cluster to use least request, make one backend slow, and compare the per-backend counts with the round robin run.
Least request reacts to in-flight load, so it steers away from a slow host where round robin keeps feeding it.
Explained in: Supported load balancers — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: The slow backend receives clearly fewer requests than under round robin for the same test.
4. Configure ring hash with a hash policy on a request header, send requests with a fixed set of header values, then remove one backend and send them again.
Consistent hashing fits caches and sticky workloads, and you should see for yourself how few keys move when the host set changes.
Explained in: Supported load balancers — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: Each header value always lands on the same backend, and only the keys of the removed backend move.
5. Configure an active HTTP health check on the cluster, stop one backend, and watch its health flags in the admin clusters view while traffic continues.
Active checks remove a dead host without spending user requests on it, and the interval and thresholds set how long that takes.
Explained in: Health checking — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: The stopped backend is marked as failing its active check and no client request returns an error after the unhealthy threshold is reached.
6. Enable outlier detection with consecutive 5xx ejection, make one backend answer its health check normally but fail real requests, and run traffic.
Passive detection catches a host that passes its probe but fails real traffic, which active checks alone miss.
Explained in: Outlier detection — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: The failing backend is ejected after the configured run of errors and returns to rotation after the ejection time.
7. Configure the cluster as strict DNS pointing at the compose service name, scale the backend service up and down, and watch the host list change.
This replaces the hand-written endpoint list with real discovery, so membership now changes without touching the Envoy configuration.
Explained in: Service discovery — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: The admin clusters view gains and loses hosts as you scale, with no Envoy restart.
8. Reproduce stale discovery data: stop a backend that DNS still lists, then remove a healthy backend from DNS, and record how Envoy treats each case.
Discovery is eventually consistent, so the proxy must combine it with health data rather than trust it as truth.
Explained in: Service discovery — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: You can state from your own run which combinations of discovered or absent and healthy or unhealthy still receive traffic.
9. Configure load balancer subsets by adding version metadata to the endpoints and a metadata match on a route, then send requests for each version.
Subsetting limits which hosts a request may reach, which is the basis for canaries and for shrinking connection fan-out.
Explained in: Load Balancer Subsets — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: Requests on the matched route reach only backends carrying the matching metadata value.
10. Configure zone aware routing by giving the Envoy node and the backends locality labels for two simulated zones on the laptop, then send traffic and check the zone routing statistics.
Keeping traffic in the local zone cuts latency and cross-zone cost, but it only engages when the documented preconditions hold.
Explained in: Zone aware routing — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: Most requests go to same-zone backends, and removing local hosts causes spillover to the other zone.
11. Enable the HTTP health checking filter on a backend-side Envoy, put it into failing mode through its admin interface while traffic runs, then stop the backend.
Failing the health check first lets upstream balancers stop sending new requests before the process exits, which is graceful draining.
Explained in: Health checking — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: The drained backend stops receiving new requests and the load test shows zero errors when the backend is stopped.
12. Configure a production-like cluster: explicit outlier detection intervals, base ejection time, a maximum ejection percent and success rate thresholds, combined with the active health check, kept in a version-pinned compose file, then rerun every failure test.
Defaults are rarely right for real traffic, and capping ejections stops a bad deploy from ejecting the whole fleet.
Explained in: Outlier detection (proto) — envoy 1.40.0-dev-b8182b documentation (www.envoyproxy.io), checked 2026-10-13
Done when: From a clean start the stack survives a dead host, an erroring host and a drain with no client-visible errors, and never ejects more hosts than the limit you set.