Envoy Gateway Extension Server
4 minute read
This task explains how to extend Envoy Gateway using an Extension Server. Envoy Gateway can be configured to call an external server over gRPC with the xDS configuration before it is sent to Envoy Proxy. The external server can modify the provided configuration programmatically using any semantics supported by the xDS API.
Using an extension server allows vendors to add xDS configuration that Envoy Gateway itself doesn’t support with a very high level of control over the generated xDS configuration.
Note: Modifying the xDS configuration generated by Envoy Gateway may break functionality configured by native Envoy Gateway means. Like other cases where the xDS configuration is modified outside of Envoy Gateway’s control, this is risky and should be tested thoroughly, especially when using the same extension server across different Envoy Gateway versions.
Introduction
One of the Envoy Gateway project goals is to “provide a common foundation for vendors to build value-added products without having to re-engineer fundamental interactions”. The Envoy Gateway Extension Server provides a mechanism where Envoy Gateway tracks all provider resources and then calls a set of hooks that allow the generated xDS configuration to be modified before it is sent to Envoy Proxy. See the design documentation for full details.
This task sets up an example extension server that adds the Envoy Proxy Basic Authentication HTTP filter to all the listeners generated by Envoy Gateway. The example extension server includes its own CRD which allows defining username/password pairs that will be accepted by the Envoy Proxy.
Note: Envoy Gateway supports adding Basic Authentication to routes using a SecurityPolicy. See this task for the preferred way to configure Basic Authentication.
Quickstart
Prerequisites
- Follow the steps from the Quickstart task to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.
Build and run the example Extension Server
Build and deploy the example extension server in the examples/extension-server
folder into the cluster
running Envoy Gateway.
Build the extension server image
Note: The provided
Makefile
builds an image with the nameextension-server:latest
. You may need to create a different tag for it in order to allow Kubernetes to pull it correctly.make image
Publish the extension server image in your docker repository
kind load docker-image --name envoy-gateway extension-server:latest
docker tag extension-server:latest $YOUR_DOCKER_REPO docker push $YOUR_DOCKER_REPO
Deploy the extension server in your cluster
If you are using your own docker image repository, make sure to update the
values.yaml
with the correct image name and tag.helm install -n envoy-gateway-system extension-server ./examples/extension-server/charts/extension-server
Configure Envoy Gateway
Grant Envoy Gateway’s
ServiceAccount
permission to access the extension server’s CRDkubectl create clusterrole listener-context-example-viewer \ --verb=get,list,watch \ --resource=ListenerContextExample kubectl create clusterrolebinding envoy-gateway-listener-context \ --clusterrole=listener-context-example-viewer \ --serviceaccount=envoy-gateway-system:envoy-gateway
Configure Envoy Gateway to use the Extension Server
Add the following fragment to Envoy Gateway’s configuration file:
extensionManager: # Envoy Gateway will watch these resource kinds and use them as extension policies # which can be attached to Gateway resources. policyResources: - group: example.extensions.io version: v1alpha1 kind: ListenerContextExample hooks: # The type of hooks that should be invoked xdsTranslator: post: - HTTPListener service: # The service that is hosting the extension server fqdn: hostname: extension-server.envoy-gateway-system.svc.cluster.local port: 5005
After updating Envoy Gateway’s configuration file, restart Envoy Gateway.
Testing
Get the Gateway’s address:
export GATEWAY_HOST=$(kubectl get gateway/eg -o jsonpath='{.status.addresses[0].value}')
The extension server adds the Basic Authentication HTTP filter to all listeners configured by Envoy Gateway. Initially there are no valid user/password combinations available. Accessing the example backend should fail with a 401 status:
$ curl -v --header "Host: www.example.com" "http://${GATEWAY_HOST}/example"
...
> GET /example HTTP/1.1
> Host: www.example.com
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 Unauthorized
< www-authenticate: Basic realm="http://www.example.com/example"
< content-length: 58
< content-type: text/plain
< date: Mon, 08 Jul 2024 10:53:11 GMT
<
...
User authentication failed. Missing username and password.
...
Add a new Username/Password combination using the example extension server’s CRD:
kubectl apply -f - << EOF
apiVersion: example.extensions.io/v1alpha1
kind: ListenerContextExample
metadata:
name: listeneruser
spec:
targetRefs:
- kind: Gateway
name: eg
group: gateway.networking.k8s.io
username: user
password: p@ssw0rd
EOF
Authenticating with this user/password combination will now work.
$ curl -v http://${GATEWAY_HOST}/example -H "Host: www.example.com" --user 'user:p@ssw0rd'
...
> GET /example HTTP/1.1
> Host: www.example.com
> Authorization: Basic dXNlcm5hbWU6cEBzc3cwcmQ=
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-type: application/json
< x-content-type-options: nosniff
< date: Mon, 08 Jul 2024 10:56:17 GMT
< content-length: 559
<
...
"headers": {
"Authorization": [
"Basic dXNlcm5hbWU6cEBzc3cwcmQ="
],
"X-Example-Ext": [
"user"
],
...
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.