This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Installation

This section includes installation related contents of Envoy Gateway.

1 - Install with Helm

Helm is a package manager for Kubernetes that automates the release and management of software on Kubernetes.

Envoy Gateway can be installed via a Helm chart with a few simple steps, depending on if you are deploying for the first time, upgrading Envoy Gateway from an existing installation, or migrating from Envoy Gateway.

Before you begin

The Envoy Gateway Helm chart is hosted by DockerHub.

It is published at oci://docker.io/envoyproxy/gateway-helm.

Install with Helm

Envoy Gateway is typically deployed to Kubernetes from the command line. If you don’t have Kubernetes, you should use kind to create one.

Install the Gateway API CRDs and Envoy Gateway:

helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.1.4 -n envoy-gateway-system --create-namespace

Wait for Envoy Gateway to become available:

kubectl wait --timeout=5m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available

Install the GatewayClass, Gateway, HTTPRoute and example app:

kubectl apply -f https://github.com/envoyproxy/gateway/releases/download/v1.1.4/quickstart.yaml -n default

Note: quickstart.yaml defines that Envoy Gateway will listen for traffic on port 80 on its globally-routable IP address, to make it easy to use browsers to test Envoy Gateway. When Envoy Gateway sees that its Listener is using a privileged port (<1024), it will map this internally to an unprivileged port, so that Envoy Gateway doesn’t need additional privileges. It’s important to be aware of this mapping, since you may need to take it into consideration when debugging.

Helm chart customizations

Some of the quick ways of using the helm install command for envoy gateway installation are below.

Increase the replicas

helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.1.4 -n envoy-gateway-system --create-namespace --set deployment.replicas=2

Change the kubernetesClusterDomain name

If you have installed your cluster with different domain name you can use below command.

helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.1.4 -n envoy-gateway-system --create-namespace --set kubernetesClusterDomain=<domain name>

Note: Above are some of the ways we can directly use for customization of our installation. But if you are looking for more complex changes values.yaml comes to rescue.

Using values.yaml file for complex installation

deployment:
  envoyGateway:
    resources:
      limits:
        cpu: 700m
        memory: 128Mi
      requests:
        cpu: 10m
        memory: 64Mi
  ports:
    - name: grpc
      port: 18005
      targetPort: 18000
    - name: ratelimit
      port: 18006
      targetPort: 18001

config:
  envoyGateway:
    logging:
      level:
        default: debug

Here we have made three changes to our values.yaml file. Increase the resources limit for cpu to 700m, changed the port for grpc to 18005 and for ratelimit to 18006 and also updated the logging level to debug.

You can use the below command to install the envoy gateway using values.yaml file.

helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.1.4 -n envoy-gateway-system --create-namespace -f values.yaml

Open Ports

These are the ports used by Envoy Gateway and the managed Envoy Proxy.

Envoy Gateway

Envoy GatewayAddressPortConfigurable
Xds EnvoyProxy Server0.0.0.018000No
Xds RateLimit Server0.0.0.018001No
Admin Server127.0.0.119000Yes
Metrics Server0.0.0.019001No
Health Check127.0.0.18081No

EnvoyProxy

Envoy ProxyAddressPort
Admin Server127.0.0.119000
Heath Check0.0.0.019001

2 - Install with Kubernetes YAML

This task walks you through installing Envoy Gateway in your Kubernetes cluster.

The manual install process does not allow for as much control over configuration as the Helm install method, so if you need more control over your Envoy Gateway installation, it is recommended that you use helm.

Before you begin

Envoy Gateway is designed to run in Kubernetes for production. The most essential requirements are:

  • Kubernetes 1.27 or later
  • The kubectl command-line tool

Install with YAML

Envoy Gateway is typically deployed to Kubernetes from the command line. If you don’t have Kubernetes, you should use kind to create one.

  1. In your terminal, run the following command:

    kubectl apply --server-side -f https://github.com/envoyproxy/gateway/releases/download/v1.1.4/install.yaml
    
  2. Next Steps

    Envoy Gateway should now be successfully installed and running, but in order to experience more abilities of Envoy Gateway, you can refer to Tasks.

Upgrading from v1.0

Due to breaking changes in Gateway API v1.1, some manual migration steps are required to upgrade Envoy Gateway to v1.1.

  1. Delete BackendTLSPolicy CRD (and resources):
kubectl delete crd backendtlspolicies.gateway.networking.k8s.io
  1. Update Gateway-API and Envoy Gateway CRDs:
helm pull oci://docker.io/envoyproxy/gateway-helm --version v1.1.4 --untar
kubectl apply --force-conflicts --server-side -f ./gateway-helm/crds/gatewayapi-crds.yaml
kubectl apply --force-conflicts --server-side -f ./gateway-helm/crds/generated
  1. Update your BackendTLSPolicy and GRPCRoute resources according to Gateway-API v1.1 Upgrade Notes

  2. Update your Envoy Gateway xPolicy resources: remove the namespace section from targetRef.

  3. Install Envoy Gateway v1.1.4:

helm upgrade eg oci://docker.io/envoyproxy/gateway-helm --version v1.1.4 -n envoy-gateway-system

3 - Install egctl

This task shows how to install the egctl CLI. egctl can be installed either from source, or from pre-built binary releases.

From The Envoy Gateway Project

The Envoy Gateway project provides two ways to fetch and install egctl. These are the official methods to get egctl releases. Installation through those methods can be found below the official methods.

Every release of egctl provides binary releases for a variety of OSes. These binary versions can be manually downloaded and installed.

  1. Download your desired version
  2. Unpack it (tar -zxvf egctl_latest_linux_amd64.tar.gz)
  3. Find the egctl binary in the unpacked directory, and move it to its desired destination (mv bin/linux/amd64/egctl /usr/local/bin/egctl)

From there, you should be able to run: egctl help.

egctl now has an installer script that will automatically grab the latest release version of egctl and install it locally.

You can fetch that script, and then execute it locally. It’s well documented so that you can read through it and understand what it is doing before you run it.

curl -fsSL -o get-egctl.sh https://gateway.envoyproxy.io/get-egctl.sh

chmod +x get-egctl.sh

# get help info of the 
bash get-egctl.sh --help

# install the latest development version of egctl
bash VERSION=latest get-egctl.sh

Yes, you can just use the below command if you want to live on the edge.

curl -fsSL https://gateway.envoyproxy.io/get-egctl.sh | VERSION=latest bash 

You can also install egctl using homebrew:

brew install egctl

4 - Control Plane Authentication using custom certs

Envoy Gateway establishes a secure TLS connection for control plane communication between Envoy Gateway pods and the Envoy Proxy fleet. The TLS Certificates used here are self signed and generated using a job that runs before envoy gateway is created, and these certs and mounted on to the envoy gateway and envoy proxy pods.

This task will walk you through configuring custom certs for control plane auth.

Before you begin

We use Cert-Manager to manage the certificates. You can install it by following the official guide.

Configure custom certs for control plane

  1. First you need to set up the CA issuer, in this task, we use the selfsigned-issuer as an example.

    You should not use the self-signed issuer in production, you should use a real CA issuer.

    cat <<EOF | kubectl apply -f -
    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
      labels:
        app.kubernetes.io/name: envoy-gateway
      name: selfsigned-issuer
      namespace: envoy-gateway-system
    spec:
      selfSigned: {}
    ---
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
      name: envoy-gateway-ca
      namespace: envoy-gateway-system
    spec:
      isCA: true
      commonName: envoy-gateway
      secretName: envoy-gateway-ca
      privateKey:
        algorithm: RSA
        size: 2048
      issuerRef:
        name: selfsigned-issuer
        kind: Issuer
        group: cert-manager.io
    ---
    apiVersion: cert-manager.io/v1
    kind: Issuer
    metadata:
      labels:
        app.kubernetes.io/name: envoy-gateway
      name: eg-issuer
      namespace: envoy-gateway-system
    spec:
      ca:
        secretName: envoy-gateway-ca
    EOF
    
  2. Create a cert for envoy gateway controller, the cert will be stored in secret envoy-gatewy.

    cat<<EOF | kubectl apply -f -
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
      labels:
        app.kubernetes.io/name: envoy-gateway
      name: envoy-gateway
      namespace: envoy-gateway-system
    spec:
      commonName: envoy-gateway
      dnsNames:
      - "envoy-gateway"
      - "envoy-gateway.envoy-gateway-system"
      - "envoy-gateway.envoy-gateway-system.svc"
      - "envoy-gateway.envoy-gateway-system.svc.cluster.local"
      issuerRef:
        kind: Issuer
        name: eg-issuer
      usages:
      - "digital signature"
      - "data encipherment"
      - "key encipherment"
      - "content commitment"
      secretName: envoy-gateway
    EOF
    
  3. Create a cert for envoy proxy, the cert will be stored in secret envoy.

    cat<<EOF | kubectl apply -f -
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
      labels:
        app.kubernetes.io/name: envoy-gateway
      name: envoy
      namespace: envoy-gateway-system
    spec:
      commonName: "*"
      dnsNames:
      - "*.envoy-gateway-system"
      issuerRef:
        kind: Issuer
        name: eg-issuer
      usages:
      - "digital signature"
      - "data encipherment"
      - "key encipherment"
      - "content commitment"
      secretName: envoy
    EOF
    
  4. Create a cert for rate limit, the cert will be stored in secret envoy-rate-limit.

    cat<<EOF | kubectl apply -f -
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
      labels:
        app.kubernetes.io/name: envoy-gateway
      name: envoy-rate-limit
      namespace: envoy-gateway-system
    spec:
      commonName: "*"
      dnsNames:
      - "*.envoy-gateway-system"
      issuerRef:
        kind: Issuer
        name: eg-issuer
      usages:
      - "digital signature"
      - "data encipherment"
      - "key encipherment"
      - "content commitment"
      secretName: envoy-rate-limit
    EOF
    
  5. Now you can follow the helm chart installation guide to install envoy gateway with custom certs.

5 - Gateway Addons Helm Chart

Version: v0.0.0-latest Type: application AppVersion: latest

An Add-ons Helm chart for Envoy Gateway

Homepage: https://gateway.envoyproxy.io/

Maintainers

NameEmailUrl
envoy-gateway-steering-committeehttps://github.com/envoyproxy/gateway/blob/main/GOVERNANCE.md
envoy-gateway-maintainershttps://github.com/envoyproxy/gateway/blob/main/CODEOWNERS

Source Code

Requirements

RepositoryNameVersion
https://fluent.github.io/helm-chartsfluent-bit0.30.4
https://grafana.github.io/helm-chartsgrafana8.0.0
https://grafana.github.io/helm-chartsloki4.8.0
https://grafana.github.io/helm-chartstempo1.3.1
https://open-telemetry.github.io/opentelemetry-helm-chartsopentelemetry-collector0.73.1
https://prometheus-community.github.io/helm-chartsprometheus25.21.0

Values

KeyTypeDefaultDescription
fluent-bit.config.filtersstring"[FILTER]\n Name kubernetes\n Match kube.*\n Merge_Log On\n Keep_Log Off\n K8S-Logging.Parser On\n K8S-Logging.Exclude On\n\n[FILTER]\n Name grep\n Match kube.*\n Regex $kubernetes['container_name'] ^envoy$\n\n[FILTER]\n Name parser\n Match kube.*\n Key_Name log\n Parser envoy\n Reserve_Data True\n"
fluent-bit.config.inputsstring"[INPUT]\n Name tail\n Path /var/log/containers/*.log\n multiline.parser docker, cri\n Tag kube.*\n Mem_Buf_Limit 5MB\n Skip_Long_Lines On\n"
fluent-bit.config.outputsstring"[OUTPUT]\n Name loki\n Match kube.*\n Host loki.monitoring.svc.cluster.local\n Port 3100\n Labels job=fluentbit, app=$kubernetes['labels']['app'], k8s_namespace_name=$kubernetes['namespace_name'], k8s_pod_name=$kubernetes['pod_name'], k8s_container_name=$kubernetes['container_name']\n"
fluent-bit.config.servicestring"[SERVICE]\n Daemon Off\n Flush {{ .Values.flush }}\n Log_Level {{ .Values.logLevel }}\n Parsers_File parsers.conf\n Parsers_File custom_parsers.conf\n HTTP_Server On\n HTTP_Listen 0.0.0.0\n HTTP_Port {{ .Values.metricsPort }}\n Health_Check On\n"
fluent-bit.enabledbooltrue
fluent-bit.fullnameOverridestring"fluent-bit"
fluent-bit.image.repositorystring"fluent/fluent-bit"
fluent-bit.podAnnotations.“fluentbit.io/exclude”string"true"
fluent-bit.podAnnotations.“prometheus.io/path”string"/api/v1/metrics/prometheus"
fluent-bit.podAnnotations.“prometheus.io/port”string"2020"
fluent-bit.podAnnotations.“prometheus.io/scrape”string"true"
fluent-bit.testFramework.enabledboolfalse
grafana.adminPasswordstring"admin"
grafana.dashboardProviders.“dashboardproviders.yaml”.apiVersionint1
grafana.dashboardProviders.“dashboardproviders.yaml”.providers[0].disableDeletionboolfalse
grafana.dashboardProviders.“dashboardproviders.yaml”.providers[0].editablebooltrue
grafana.dashboardProviders.“dashboardproviders.yaml”.providers[0].folderstring"envoy-gateway"
grafana.dashboardProviders.“dashboardproviders.yaml”.providers[0].namestring"envoy-gateway"
grafana.dashboardProviders.“dashboardproviders.yaml”.providers[0].options.pathstring"/var/lib/grafana/dashboards/envoy-gateway"
grafana.dashboardProviders.“dashboardproviders.yaml”.providers[0].orgIdint1
grafana.dashboardProviders.“dashboardproviders.yaml”.providers[0].typestring"file"
grafana.dashboardsConfigMaps.envoy-gatewaystring"grafana-dashboards"
grafana.datasources.“datasources.yaml”.apiVersionint1
grafana.datasources.“datasources.yaml”.datasources[0].namestring"Prometheus"
grafana.datasources.“datasources.yaml”.datasources[0].typestring"prometheus"
grafana.datasources.“datasources.yaml”.datasources[0].urlstring"http://prometheus"
grafana.enabledbooltrue
grafana.fullnameOverridestring"grafana"
grafana.service.typestring"LoadBalancer"
loki.backend.replicasint0
loki.deploymentModestring"SingleBinary"
loki.enabledbooltrue
loki.fullnameOverridestring"loki"
loki.gateway.enabledboolfalse
loki.loki.auth_enabledboolfalse
loki.loki.commonConfig.replication_factorint1
loki.loki.compactorAddressstring"loki"
loki.loki.memberliststring"loki-memberlist"
loki.loki.rulerConfig.storage.typestring"local"
loki.loki.storage.typestring"filesystem"
loki.monitoring.lokiCanary.enabledboolfalse
loki.monitoring.selfMonitoring.enabledboolfalse
loki.monitoring.selfMonitoring.grafanaAgent.installOperatorboolfalse
loki.read.replicasint0
loki.singleBinary.replicasint1
loki.test.enabledboolfalse
loki.write.replicasint0
opentelemetry-collector.config.exporters.logging.verbositystring"detailed"
opentelemetry-collector.config.exporters.loki.endpointstring"http://loki.monitoring.svc:3100/loki/api/v1/push"
opentelemetry-collector.config.exporters.otlp.endpointstring"tempo.monitoring.svc:4317"
opentelemetry-collector.config.exporters.otlp.tls.insecurebooltrue
opentelemetry-collector.config.exporters.prometheus.endpointstring"0.0.0.0:19001"
opentelemetry-collector.config.extensions.health_checkobject{}
opentelemetry-collector.config.processors.attributes.actions[0].actionstring"insert"
opentelemetry-collector.config.processors.attributes.actions[0].keystring"loki.attribute.labels"
opentelemetry-collector.config.processors.attributes.actions[0].valuestring"k8s.pod.name, k8s.namespace.name"
opentelemetry-collector.config.receivers.otlp.protocols.grpc.endpointstring"${env:MY_POD_IP}:4317"
opentelemetry-collector.config.receivers.otlp.protocols.http.endpointstring"${env:MY_POD_IP}:4318"
opentelemetry-collector.config.receivers.zipkin.endpointstring"${env:MY_POD_IP}:9411"
opentelemetry-collector.config.service.extensions[0]string"health_check"
opentelemetry-collector.config.service.pipelines.logs.exporters[0]string"loki"
opentelemetry-collector.config.service.pipelines.logs.processors[0]string"attributes"
opentelemetry-collector.config.service.pipelines.logs.receivers[0]string"otlp"
opentelemetry-collector.config.service.pipelines.metrics.exporters[0]string"prometheus"
opentelemetry-collector.config.service.pipelines.metrics.receivers[0]string"otlp"
opentelemetry-collector.config.service.pipelines.traces.exporters[0]string"otlp"
opentelemetry-collector.config.service.pipelines.traces.receivers[0]string"otlp"
opentelemetry-collector.config.service.pipelines.traces.receivers[1]string"zipkin"
opentelemetry-collector.enabledboolfalse
opentelemetry-collector.fullnameOverridestring"otel-collector"
opentelemetry-collector.modestring"deployment"
prometheus.alertmanager.enabledboolfalse
prometheus.enabledbooltrue
prometheus.kube-state-metrics.enabledboolfalse
prometheus.prometheus-node-exporter.enabledboolfalse
prometheus.prometheus-pushgateway.enabledboolfalse
prometheus.server.fullnameOverridestring"prometheus"
prometheus.server.global.scrape_intervalstring"15s"
prometheus.server.image.repositorystring"prom/prometheus"
prometheus.server.persistentVolume.enabledboolfalse
prometheus.server.readinessProbeInitialDelayint0
prometheus.server.securityContextobject{}
prometheus.server.service.typestring"LoadBalancer"
tempo.enabledbooltrue
tempo.fullnameOverridestring"tempo"
tempo.service.typestring"LoadBalancer"

6 - Gateway Helm Chart

Version: v0.0.0-latest Type: application AppVersion: latest

The Helm chart for Envoy Gateway

Homepage: https://gateway.envoyproxy.io/

Maintainers

NameEmailUrl
envoy-gateway-steering-committeehttps://github.com/envoyproxy/gateway/blob/main/GOVERNANCE.md
envoy-gateway-maintainershttps://github.com/envoyproxy/gateway/blob/main/CODEOWNERS

Source Code

Values

KeyTypeDefaultDescription
certgenobject{"job":{"annotations":{},"resources":{},"ttlSecondsAfterFinished":30},"rbac":{"annotations":{},"labels":{}}}Certgen is used to generate the certificates required by EnvoyGateway. If you want to construct a custom certificate, you can generate a custom certificate through Cert-Manager before installing EnvoyGateway. Certgen will not overwrite the custom certificate. Please do not manually modify values.yaml to disable certgen, it may cause EnvoyGateway OIDC,OAuth2,etc. to not work as expected.
config.envoyGateway.gateway.controllerNamestring"gateway.envoyproxy.io/gatewayclass-controller"
config.envoyGateway.logging.level.defaultstring"info"
config.envoyGateway.provider.typestring"Kubernetes"
createNamespaceboolfalse
deployment.envoyGateway.image.repositorystring""
deployment.envoyGateway.image.tagstring""
deployment.envoyGateway.imagePullPolicystring""
deployment.envoyGateway.imagePullSecretslist[]
deployment.envoyGateway.resources.limits.cpustring"500m"
deployment.envoyGateway.resources.limits.memorystring"1024Mi"
deployment.envoyGateway.resources.requests.cpustring"100m"
deployment.envoyGateway.resources.requests.memorystring"256Mi"
deployment.pod.affinityobject{}
deployment.pod.annotations.“prometheus.io/port”string"19001"
deployment.pod.annotations.“prometheus.io/scrape”string"true"
deployment.pod.labelsobject{}
deployment.pod.tolerationslist[]
deployment.pod.topologySpreadConstraintslist[]
deployment.ports[0].namestring"grpc"
deployment.ports[0].portint18000
deployment.ports[0].targetPortint18000
deployment.ports[1].namestring"ratelimit"
deployment.ports[1].portint18001
deployment.ports[1].targetPortint18001
deployment.ports[2].namestring"wasm"
deployment.ports[2].portint18002
deployment.ports[2].targetPortint18002
deployment.ports[3].namestring"metrics"
deployment.ports[3].portint19001
deployment.ports[3].targetPortint19001
deployment.replicasint1
global.images.envoyGateway.imagestringnil
global.images.envoyGateway.pullPolicystringnil
global.images.envoyGateway.pullSecretslist[]
global.images.ratelimit.imagestring"docker.io/envoyproxy/ratelimit:master"
global.images.ratelimit.pullPolicystring"IfNotPresent"
global.images.ratelimit.pullSecretslist[]
kubernetesClusterDomainstring"cluster.local"
podDisruptionBudget.minAvailableint0