Gateway API Metrics

  4 minute read  

Prerequisites

Install Envoy Gateway

Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

Expand for instructions

  1. Install the Gateway API CRDs and Envoy Gateway using Helm:

    helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.3.2 -n envoy-gateway-system --create-namespace

  2. Install the GatewayClass, Gateway, HTTPRoute and example app:

    kubectl apply -f https://github.com/envoyproxy/gateway/releases/download/v1.3.2/quickstart.yaml -n default

  3. Verify Connectivity:

    You can also test the same functionality by sending traffic to the External IP. To get the external IP of the Envoy service, run:

    export GATEWAY_HOST=$(kubectl get gateway/eg -o jsonpath='{.status.addresses[0].value}')

    Note: In certain environments, the load balancer may be exposed using a hostname, instead of an IP address. If so, replace ip in the above command with hostname.

    Curl the example app through Envoy proxy:

    curl --verbose --header "Host: www.example.com" http://$GATEWAY_HOST/get

    Get the name of the Envoy service created by the example Gateway:

    export ENVOY_SERVICE=$(kubectl get svc -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-namespace=default,gateway.envoyproxy.io/owning-gateway-name=eg -o jsonpath='{.items[0].metadata.name}')

    Port forward to the Envoy service:

    kubectl -n envoy-gateway-system port-forward service/${ENVOY_SERVICE} 8888:80 &

    Curl the example app through Envoy proxy:

    curl --verbose --header "Host: www.example.com" http://localhost:8888/get

Install Add-ons

Envoy Gateway provides an add-ons Helm chart to simplify the installation of observability components.
The documentation for the add-ons chart can be found here.

Follow the instructions below to install the add-ons Helm chart.

helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version v1.3.2 -n monitoring --create-namespace

By default, the OpenTelemetry Collector is disabled. To install add-ons with OpenTelemetry Collector enabled, use the following command.

helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version v1.3.2 --set opentelemetry-collector.enabled=true -n monitoring --create-namespace

Enable kube-state-metrics

The kube-state-metrics service is required to collect metrics from the Kubernetes API server. Use the following command to enable it:

helm upgrade eg-addons oci://docker.io/envoyproxy/gateway-addons-helm \
--version v1.3.2 \
--reuse-values \
--set prometheus.kube-state-metrics.enabled=true \
-n monitoring

Metrics

To query metrics using Prometheus API, follow the steps below. Make sure to wait for the statefulset to be ready before port-forwarding.

export PROMETHEUS_PORT=$(kubectl get service prometheus -n monitoring -o jsonpath='{.spec.ports[0].port}')
kubectl port-forward service/prometheus -n monitoring 9090:$PROMETHEUS_PORT

The example query below fetches the gatewayapi_gateway_created metric. Alternatively, access the Prometheus UI at http://localhost:9090.

curl -s 'http://localhost:9090/api/v1/query?query=gatewayapi_gateway_created' | jq .

Alerts

A set of example alert rules are available in config/examples/rules. To create alert use the following command:

cat <<EOF | helm upgrade eg-addons oci://docker.io/envoyproxy/gateway-addons-helm \
  --version v0.0.0-latest \
  -n monitoring --reuse-values -f -
prometheus:
  serverFiles:
    alerting_rules.yml:
      groups:
        - name: gateway-api.rules
          rules:
            - alert: UnhealthyGateway
              expr: (gatewayapi_gateway_status{type="Accepted"} == 0) or (gatewayapi_gateway_status{type="Programmed"} == 0)
              for: 10m
              labels:
                severity: critical
              annotations:
                summary: "Either the Accepted or Programmed status is not True"
                description: "Gateway {{ \$labels.namespace }}/{{ \$labels.name }} has an unhealthy status"
            - alert: InsecureHTTPListener
              expr: gatewayapi_gateway_listener_info{protocol="HTTP"}
              for: 10m
              labels:
                severity: critical
              annotations:
                summary: "Listeners must use HTTPS"
                description: "Gateway {{ \$labels.namespace }}/{{ \$labels.name }} has an insecure listener {{ \$labels.protocol }}/{{ \$labels.port }}"
EOF

To view the alerts, navigate to the Alerts tab at http://localhost:9090/alerts.

Alternatively, you can use the following command to view the alerts via the Prometheus API:

curl -s http://localhost:9090/api/v1/alerts | jq '.data.alerts[]'

Dashboards

To access the Grafana dashboards, follow these steps:

  1. Wait for the deployment to complete, then set up port forwarding using the following commands:

    export GRAFANA_PORT=$(kubectl get service grafana -n monitoring -o jsonpath='{.spec.ports[0].port}')
kubectl port-forward service/grafana -n monitoring 3000:$GRAFANA_PORT

  2. Access Grafana by navigating to http://localhost:3000 in your web browser

  3. Log in using the default credentials:

    • Username: admin
    • Password: admin

A set of Grafana dashboards is provided by Gateway API State Metrics. These dashboards are available in ./config/examples/dashboards and on grafana.com. To import them manually navigate to the Grafana UI and select Dashboards > New > Import.

Alternatively, use the following command to import dashboards using the Grafana API:

export GRAFANA_API_KEY="your-api-key"

urls=(
  "https://grafana.com/api/dashboards/19433/revisions/1/download"
  "https://grafana.com/api/dashboards/19432/revisions/1/download"
  "https://grafana.com/api/dashboards/19434/revisions/1/download"
  "https://grafana.com/api/dashboards/19570/revisions/1/download"
)

for url in "${urls[@]}"; do
  dashboard_data=$(curl -s "$url")
  curl -X POST \
    -H "Authorization: Bearer $GRAFANA_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"dashboard\": $dashboard_data, \"overwrite\": true}" \
    "http://localhost:3000/api/dashboards/db"
done

Next Steps

Check out the Gateway Exported Metrics section to learn more about the metrics exported by the Envoy Gateway.