Zone Aware Routing

EnvoyGateway makes use of Envoy Zone Aware Routing to support Kubernetes Traffic Distribution and Topology Aware Routing which are useful for keeping network traffic in the originating zone. Preferring same-zone traffic between Pods in your cluster can help with reliability, performance (network latency and throughput), or cost.

Prerequisites

• The Kubernetes cluster’s nodes must indicate topology information via the topology.kubernetes.io/zone well-known label.
• There must be at least two valid topology zones for scheduling.

• Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

  Expand for instructions

  1. Install the Gateway API CRDs and Envoy Gateway using Helm:

     helm install eg oci://docker.io/envoyproxy/gateway-helm --version v0.0.0-latest -n envoy-gateway-system --create-namespace
     

  2. Install the GatewayClass, Gateway, HTTPRoute and example app:

     kubectl apply -f https://github.com/envoyproxy/gateway/releases/download/latest/quickstart.yaml -n default
     

  3. Verify Connectivity:

     • With External LoadBalancer Support
     • Without LoadBalancer Support

     Get the External IP of the Gateway:

     export GATEWAY_HOST=$(kubectl get gateway/eg -o jsonpath='{.status.addresses[0].value}')
        

     Curl the example app through Envoy proxy:

     curl --verbose --header "Host: www.example.com" http://$GATEWAY_HOST/get
        

     The above command should succeed with status code 200.

     Get the name of the Envoy service created the by the example Gateway:

     export ENVOY_SERVICE=$(kubectl get svc -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-namespace=default,gateway.envoyproxy.io/owning-gateway-name=eg -o jsonpath='{.items[0].metadata.name}')
        

     Get the deployment of the Envoy service created the by the example Gateway:

     export ENVOY_DEPLOYMENT=$(kubectl get deploy -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-namespace=default,gateway.envoyproxy.io/owning-gateway-name=eg -o jsonpath='{.items[0].metadata.name}')
        

     Port forward to the Envoy service:

     kubectl -n envoy-gateway-system port-forward service/${ENVOY_SERVICE} 8888:80 &
        

     Curl the example app through Envoy proxy:

     curl --verbose --header "Host: www.example.com" http://localhost:8888/get
        

     The above command should succeed with status code 200.

Configuration

Kubernetes Service

Create the example Kubernetes Service with either topology aware routing or traffic distribution enabled.

Topology Aware Routing

• Apply from stdin
• Apply from file

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.kubernetes.io/topology-mode: Auto
  name: zone-aware-routing-backend
  labels:
    app: zone-aware-routing-backend
    service: zone-aware-routing-backend
spec:
  ports:
    - name: http
      port: 3000
      targetPort: 3000
  selector:
    app: zone-aware-routing-backend
EOF

---
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.kubernetes.io/topology-mode: Auto
  name: zone-aware-routing-backend
  labels:
    app: zone-aware-routing-backend
    service: zone-aware-routing-backend
spec:
  ports:
    - name: http
      port: 3000
      targetPort: 3000
  selector:
    app: zone-aware-routing-backend

Traffic Distribution

• Apply from stdin
• Apply from file

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: zone-aware-routing-backend
  labels:
    app: zone-aware-routing-backend
    service: zone-aware-routing-backend
spec:
  ports:
    - name: http
      port: 3000
      targetPort: 3000
  selector:
    app: zone-aware-routing-backend
  trafficDistribution: PreferClose
EOF

---
apiVersion: v1
kind: Service
metadata:
  name: zone-aware-routing-backend
  labels:
    app: zone-aware-routing-backend
    service: zone-aware-routing-backend
spec:
  ports:
    - name: http
      port: 3000
      targetPort: 3000
  selector:
    app: zone-aware-routing-backend
  trafficDistribution: PreferClose

Example deployments and HTTPRoute

Next apply the example manifests to create two Deployments and an HTTPRoute. For the test configuration one Deployment (zone-aware-routing-backend-local) includes affinity for EnvoyProxy Pods to ensure its Pods are scheduled in the same zone and the second Deployment (zone-aware-routing-backend-nonlocal) uses anti-affinity to ensure its Pods don’t schedule to the same zone in order to demonstrate functionality.

kubectl apply -f https://raw.githubusercontent.com/envoyproxy/gateway/latest/examples/kubernetes/zone-aware-routing.yaml -n default

Ensure that both example Deployments are marked as ready and produces the following output:

kubectl get deployment/zone-aware-routing-backend-local deployment/zone-aware-routing-backend-nonlocal -n default
NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
zone-aware-routing-backend-local      1/1     1            1           9m1s
zone-aware-routing-backend-nonlocal   1/1     1            1           9m1s

An HTTPRoute resource is created for the /zone-aware-routing path prefix along with two example Deployment resources that are respectively configured with pod affinity/anti-affinity targeting the Envoy Proxy Pods for testing.

Verify the HTTPRoute configuration and status:

kubectl get httproute/zone-aware-routing -o yaml

Testing

Ensure the GATEWAY_HOST environment variable from the Quickstart is set. If not, follow the Quickstart instructions to set the variable.

echo $GATEWAY_HOST

HTTPRoute

We will now send a request and expect to be routed to backend in the same local zone as the Envoy Proxy Pods.

curl -H "Host: www.example.com" "http://${GATEWAY_HOST}/zone-aware-routing" -XPOST -d '{}'

We will see the following output. The pod header identifies which backend received the request and should have a name prefix of zone-aware-routing-backend-local-. We should see this behavior every time and the nonlocal backend should not receive requests.

{
  "path": "/zone-aware-routing",
  "host": "www.example.com",
  "method": "GET",
  "proto": "HTTP/1.1",
  "headers": {
    "Accept": [
      "*/*"
    ],
    "User-Agent": [
      "curl/8.7.1"
    ],
    "X-Envoy-External-Address": [
      "127.0.0.1"
    ],
    "X-Forwarded-For": [
      "10.244.1.5"
    ],
    "X-Forwarded-Proto": [
      "http"
    ],
    "X-Request-Id": [
      "0f4c5d28-52d0-4727-881f-abf2ac12b3b7"
    ]
  },
  "namespace": "default",
  "ingress": "",
  "service": "",
  "pod": "zone-aware-routing-backend-local-5586cd668d-md8sf"
}

Clean-Up

Follow the steps from the Quickstart to uninstall Envoy Gateway and the example manifest.

Delete the zone-aware-routing example resources and HTTPRoute:

kubectl delete service/zone-aware-routing-backend 
kubectl delete deployment/zone-aware-routing-backend-local
kubectl delete deployment/zone-aware-routing-backend-nonlocal
kubectl delete httproute/zone-aware-routing