--- title: Configuring Envoy proxy deployments summary: null url: >- https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/reverse-proxy-deployment/configuring-envoy-proxy-deployments --- Support is available for the [Envoy Proxy](https://envoyproxy.io) via builtin Envoy gRPC APIs implemented in the `sigsci-agent` running as a gRPC server. Envoy v1.11.0 or later is recommended, however, Envoy v1.8.0 or later is supported with limited functionality as noted in the documentation below. Envoy (as of v1.11) does not support a bidirectional gRPC API for inspecting traffic. There are instead two separate gRPC APIs available to inspect traffic. The [External Authorization HTTP filter (`envoy.filters.http.ext_authz`)](https://www.envoyproxy.io/docs/envoy/v1.33.0/configuration/http/http_filters/ext_authz_filter) gRPC API allows the request to be held while waiting inbound request inspection, which allows for a request to be blocked if required. An additional [gRPC AccessLog Service](https://www.envoyproxy.io/docs/envoy/v1.33.0/api-v3/extensions/access_loggers/grpc/v3/als.proto) gRPC API can then be used to inspect the outbound request data. Using these two APIs together with the `sigsci-agent` running as a gRPC server allows for inspection in both directions using only Envoy builtin APIs. This allows web application inspection without installing a module for every upstream application. In this case the `sigsci-agent` is acting as the module. ## Request allowed (normal) processing ![Diagram showing requests allowed through Envoy and Next-Gen WAF as described below.](/img/ngwaf/Envoy_SigSci_notBlocked.png) This is the flow for normal requests that the `sigsci-agent` allows through Envoy. 1. Client request received by Envoy and routed to the Envoy `External Authorization (ext_authz)` HTTP filter where request metadata is extracted for processing via the `sigsci-agent`. 2. Request metadata is sent to the `sigsci-agent` via gRPC `ext_authz` API 3. The `sigsci-agent` sends back an 'allow request' response allowing the request through the `ext_authz` HTTP filter to continue normal Envoy request processing. 4. Request makes it through any additional HTTP filters to the Handler, which processes the request and generates the response. 5. Request/Response metadata is extracted via the Envoy `gRPC AccessLog Service (als)` asynchronously for processing via the `sigsci-agent`. 6. In parallel, additional metadata, such as response headers and the HTTP status code, is sent to the `sigsci-agent` via gRPC `als` API for further processing while the response data is sent back to the originating client. ## Request blocked processing ![Diagram showing requests flowing through Envoy and blocked by Next-Gen WAF as described below.](/img/ngwaf/Envoy_SigSci_Blocked.png) This is the flow if the `sigsci-agent` blocks a request from being processed through Envoy. 1. Client request received by Envoy and routed to the Envoy `External Authorization (ext_authz)` HTTP filter where request metadata is extracted for processing via the `sigsci-agent`. 2. Request metadata is sent to the `sigsci-agent` via gRPC `ext_authz` API 3. The `sigsci-agent` sends back a 'block request' response, disallowing the request to continue being processed by the HTTP filter chain. 4. This triggers the `ext_authz` filter to generate a HTTP 406 response, blocking the request from any further processing. ## Next-Gen WAF agent configuration The `sigsci-agent` is normally [installed as a sidecar via Kubernetes](https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/kubernetes/kubernetes-intro/) with a slightly different configuration than a normal install. The `sigsci-agent` must be configured to run with an Envoy gRPC listener instead of the normal RPC listener. To do this, configure the Envoy gRPC listener via the `envoy-grpc-address` [agent configuration](https://www.fastly.com/documentation/reference/ngwaf/agent-config#agentcfg_envoy-grpc-address) option, which will then start instead of the default RPC listener. Setting the configuration value in the `sigsci-agent` config file: ```text envoy-grpc-address = "0.0.0.0:8000" ``` Or setting the configuration value in the `sigsci-agent` environment: ```text SIGSCI_ENVOY_GRPC_ADDRESS=0.0.0.0:8000 ``` Optionally, the `sigsci-agent` can be configured with TLS enabled. To do this, set the certificate and key files in the `sigsci-agent` configuration. ```text envoy-grpc-cert = "/path/to/cert.pem" envoy-grpc-key = "/path/to/key.pem" ``` Or ```text SIGSCI_ENVOY_GRPC_CERT=/path/to/cert.pem SIGSCI_ENVOY_GRPC_KEY=/path/to/key.pem ``` Additionally, it is recommended to enable response data processing. To do this, the `sigsci-agent` must be configured to expect response data from Envoy by setting the `envoy-expect-response-data` [agent configuration](https://www.fastly.com/documentation/reference/ngwaf/agent-config#agentcfg_envoy-expect-response-data) option. By default, response data is ignored in the `sigsci-agent` as this is an optional Envoy configuration option in order to better support older versions of Envoy. If you are running Envoy v1.10 or higher, then you should enable this option. Setting the configuration value in the `sigsci-agent` config file: ```text envoy-expect-response-data = 1 ``` Or setting the configuration value in the `sigsci-agent` environment: ```text SIGSCI_ENVOY_EXPECT_RESPONSE_DATA=1 ``` Some aspects of inspection in the `sigsci-agent` can be configured but generally should be left as the default. Check out `inspection-*` [agent configuration](https://www.fastly.com/documentation/reference/ngwaf/agent-config#agentcfg_inspection-anomaly-duration) for more details. ## Envoy configuration Envoy must to be configured with an [External Authorization HTTP filter (`envoy.filters.http.ext_authz`)](https://www.envoyproxy.io/docs/envoy/v1.33.0/configuration/http/http_filters/ext_authz_filter) before the main handler filter to process request data and (optionally, though recommended) a [gRPC AccessLog Service](https://www.envoyproxy.io/docs/envoy/v1.33.0/api-v3/extensions/access_loggers/grpc/v3/als.proto) to process response data. To do this, multiple configuration items must to be added to the Envoy configuration: a cluster to handle the gRPC calls via the `sigsci-agent`, the `envoy.filters.http.ext_authz` HTTP filter before the main handler, and the `envoy.http_grpc_access_log` service added to the `access_log` section of the HTTP listener filter if response data is to be enabled. ### Adding the Next-Gen WAF agent cluster A [cluster](https://www.envoyproxy.io/docs/envoy/v1.33.0/api-v3/clusters/clusters) must be added which is configured with the Envoy gRPC address used in the `sigsci-agent` configuration. Currently load balancing will not work correctly if response data is enabled as there is not a way to enable consistent hashing for gRPC services in Envoy (yet), so it is recommended not to configure load balancing at this time unless only the `envoy.filters.http.ext_authz` API is being used without response data inspection. ```yaml clusters: - name: sigsci-agent-grpc connect_timeout: 0.2s type: strict_dns #lb_policy: LEAST_REQUEST http2_protocol_options: {} #tls_context: {} ### You can also use 'hosts' below, but this is deprecated load_assignment: cluster_name: sigsci-agent-grpc endpoints: - lb_endpoints: - endpoint: address: socket_address: address: sigsci-agent port_value: 8000 ``` The `address` is a resolvable hostname or IP for the `sigsci-agent` and the `port_value` must match that configured in the `sigsci-agent` configuration for the `envoy-grpc-address` option. > **HINT:** The `connect_timeout` is the timeout to connect to the `sigsci-agent` (but not to process the data) and can be adjusted if required. The `tls_context` option must be defined if TLS is to be used. TLS can be configured in the `sigsci-agent` config via `envoy-grpc-cert` and `envoy-grpc-key`. If TLS is configured in the `sigsci-agent`, then just the empty `tls_context` must be configured (e.g., `tls_context: {}`) to let Envoy know to connect via TLS. If certificate validation is desired, then `validation_context` must be configured in the `tls_context` to specify a `trusted_ca` filename to use for validation. As gRPC services are HTTP/2 based, the `http2_protocol_options: {}` option is required so that traffic is sent to the `sigsci-agent` cluster as HTTP/2. ### Adding the Envoy External Authorization HTTP filter The listener must have an [External Authorization HTTP filter (`envoy.filters.http.ext_authz`)](https://www.envoyproxy.io/docs/envoy/v1.33.0/configuration/http/http_filters/ext_authz_filter) added before the main handler which points at the `sigsci-agent` cluster. ```yaml http_filters: - name: envoy.filters.http.ext_authz typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz transport_api_version: V3 grpc_service: envoy_grpc: cluster_name: sigsci-agent-grpc timeout: 0.2s failure_mode_allow: true with_request_body: # Maximum request body bytes buffered and sent to the sigsci-agent max_request_bytes: 8192 # NOTE: If allow_partial_message is set false, then any request over # the above max bytes will fail with an HTTP "413 Payload Too Large" # so it is recommended to set this to true. allow_partial_message: true # NOTE: By default, envoy carries the HTTP request body as a UTF-8 string # and it fills the body @ # field. To pack the request body as raw bytes, # set pack_as_bytes to true. pack_as_bytes: true - name: envoy.filters.http.router typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router ``` > **HINT:** `failure_mode_allow: true` is so that this will fail open, which is recommended. And `timeout` allows failing _with the defined failure mode_ (true for fail open, false for fail closed) after a given time duration. Once this is done, all HTTP requests will be first sent to the `envoy.filters.http.ext_authz` filter handled by the `sigsci-agent` cluster. The `sigsci-agent` will then process requests and deny auth with a 406 HTTP status code if the request is to be blocked or allow the request through to the next HTTP filter if it is allowed. Any additional [HTTP request headers](https://www.fastly.com/documentation/guides/next-gen-waf/developer/x-sigsci-headers/) are also added to the request as they are in other modules. ### Adding the Envoy gRPC AccessLog Service This is a recommended, but optional step. If it is configured in Envoy, then the agent _must_ also be configured to expect response data by setting the `envoy-expect-response-data` [agent configuration](https://www.fastly.com/documentation/reference/ngwaf/agent-config#agentcfg_envoy-expect-response-data) option as noted in the [Next-Gen WAF agent configuration](https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/reverse-proxy-deployment/configuring-envoy-proxy-deployments#next-gen-waf-agent-configuration) section. The Envoy External Authorization (`envoy.filters.http.ext_authz`) HTTP Filter can only process request data. As the `sigsci-agent` needs the response data for full functionality, a [gRPC AccessLog Service](https://www.envoyproxy.io/docs/envoy/v1.33.0/api-v3/extensions/access_loggers/grpc/v3/als.proto) must be set up to send the response data to the `sigsci-agent`. To do this an `access_log` section must be added to the Envoy configuration under the listener filter (typically under the `envoy.filters.network.http_connection_manager` filter) if it does not already exist. If it does exist, then it must be appended to. Refer to the `access_log` configuration option of the [HTTP Connection Manager](https://www.envoyproxy.io/docs/envoy/v1.33.0/api-v3/extensions/filters/network/http_connection_manager/v3/http_connection_manager.proto.html) for more details. An `envoy.http_grpc_access_log` entry must be added here (in addition to any other existing access log entries). Recommended Configuration (see [Limitations and considerations](https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/reverse-proxy-deployment/configuring-envoy-proxy-deployments#limitations-and-considerations) for further customizations to minimize limitations): ```yaml access_log: - name: envoy.http_grpc_access_log typed_config: "@type": type.googleapis.com/envoy.extensions.access_loggers.grpc.v3.HttpGrpcAccessLogConfig common_config: log_name: "sigsci-agent-grpc" transport_api_version: V3 grpc_service: envoy_grpc: cluster_name: sigsci-agent-grpc timeout: 0.2s additional_request_headers_to_log: # These sigsci-agent headers are required for correct processing: - "x-sigsci-request-id" - "x-sigsci-waf-response" # Optionally, additional headers can be added that should be recorded: - "accept" - "content-type" - "content-length" additional_response_headers_to_log: - "date" - "server" - "content-type" - "content-length" ``` ## Limitations and considerations Here are the current limitations when using the `sigsci-agent` with Envoy Proxy. As support for Envoy Proxy improves in the future, these limitations will be addressed and should be reduced. ### No request bodies are processed by default Prior to Envoy v1.10.0, the Envoy External Authorization did not send the request body. In all versions of Envoy, the request body is not included in the `ext_authz` call by default and it will not be inspected by the `sigsci-agent` unless configured. For Envoy v1.10.0 or higher, support to include the request body is built in to the `envoy.filters.http.ext_authz` configuration and it is now possible to configure the `with_request_body` in this section of the Envoy configuration as noted above. For Envoy v1.11.0 or higher, support was extended to be able to detect partial bodies more accurately. For HTTP/2 (and gRPC) support Envoy must be running a version later than v1.12.1. In Envoy v1.10.0 - v1.12.1 Envoy is not properly sending the request body using `with_request_body`. However, it is possible to work around this Envoy limitation using Lua until an Envoy upgrade is possible. The following is an example Lua filter that can be used to pass on gRPC based bodies to the `sigsci-agent` for inspection: To do this, the [Lua HTTP filter (`envoy.lua`)](https://www.envoyproxy.io/docs/envoy/v1.11.0/configuration/http_filters/lua_filter) HTTP filter can be configured before the `envoy.ext_authz` filter to add an internal `x-sigsci-encoded-body` header with this data. A small snippet of Lua code must be added to extract the body and add it to the request as follows: ```yaml http_filters: - name: envoy.lua config: inline_code: | -- Add a special header to pass the encoded body function envoy_on_request(req) local len = 0 local reqbody -- Determine the body length local cl = req:headers():get("content-length") if cl ~= nil then len = tonumber(cl) end -- gRPC does not have a content-length header to limit the body before buffering if len == 0 and req:headers():get("content-type") == "application/grpc" then -- Triggers buffering len = req:body():length() end -- Limit body length sent to the agent (adjust as needed) if len > 0 and len <= 8192 then -- Triggers buffering reqbody = req:body():getBytes(0, len) -- Encode the body for use in a header value local enc, t = string.gsub(reqbody, "[^%w]", function(chr) return string.format("%%%02X",string.byte(chr)) end) req:headers():add("x-sigsci-encoded-body", enc) end end - name: envoy.ext_authz config: grpc_service: envoy_grpc: cluster_name: sigsci-agent-grpc timeout: 0.2s failure_mode_allow: true # with_request_body: # max_request_bytes: 8192 # allow_partial_message: true - name: envoy.router config: {} ``` ### No TLS handshake metadata is extracted There is not currently a means for the `sigsci-agent` to see the TLS handshake metadata (e.g., cipher and protocol version) used in the originating request as this is not (yet) available in Envoy. Any TLS handshake metadata based signals will not be seen in the product for this site (also known as a workspace). The following [system signals](https://www.fastly.com/documentation/guides/next-gen-waf/signals/using-system-signals/) are currently not supported due to this limitation: - WEAKTLS ### Only minimal request headers are recorded by default if there were only response-based signals If the request was inspected by the `envoy.filters.http.ext_authz` filter and no signals were issued, then the response will be processed by the `envoy.http_grpc_access_log` service. If a signal is found in the response data, then only minimal request headers will be recorded with the signal due to the API not being sent all request headers by default. However, if additional request headers are desired to be recorded, then these should be added via the `additional_request_headers_to_log` option of the `access_log` configuration in Envoy. Currently these headers will automatically be added: - `Host` - `User-Agent` - `Referer` - `X-Forwarded-For` Two `sigsci-agent` specific headers must be added. Additionally any additional request headers can be added explicitly via `additional_request_headers_to_log`: ```yaml additional_request_headers_to_log: # These sigsci-agent headers are required for correct processing: - "x-sigsci-request-id" - "x-sigsci-waf-response" # Optionally, additional headers can be added that should be recorded: - "accept" - "content-type" - "content-length" - "x-real-ip" ``` ### No response headers are processed by default Similar to above with minimal request headers not being processed by the `envoy.http_grpc_access_log` service, there are no response headers sent to this API by default. Any headers that are desired to be recorded must be explicitly listed in the `additional_response_headers_to_log` option of the `access_log` configuration in Envoy as there is not currently any means to wildcard this. The following are recommended. ```yaml additional_response_headers_to_log: - "date" - "server" - "content-type" - "content-length" ``` ## Sample deployment The [Sample deployment configuration](https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/reverse-proxy-deployment/configuring-envoy-proxy-deployments#sample-deployment-configuration) section contains a complete Envoy proxy deployment configuration, which you can use as a starting point for your own deployment. For the sample configuration to work, complete the following steps: 1. Update the [agent keys](https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/agent-management/accessing-agent-keys) for your site (also known as workspace) by running the following command: ```term copy nolinenums $ kubectl create secret generic sigsci. \ --from-literal=accesskeyid= \ --from-literal=secretaccesskey= ``` Be sure to replace `` with the name of your site (workspace) and `` and `` with your agent keys. 2. Save the [Sample deployment configuration](https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/reverse-proxy-deployment/configuring-envoy-proxy-deployments#sample-deployment-configuration) as `envoy-deployment.yaml`. 3. Deploy `envoy-deployment.yaml` by running the following command: ```term copy nolinenums $ kubectl apply -f envoy-deployment.yaml ``` 4. _(Optional)_ Set up port forwarding and test the deployment locally by running the following command: ```term copy nolinenums $ kubectl port-forward svc/envoy 10000:10000 ``` ### Sample deployment configuration ```yaml # ConfigMap: Contains the Envoy configuration with ext_authz integration. apiVersion: v1 kind: ConfigMap metadata: name: envoy-config data: envoy.yaml: | node: id: proxy-node cluster: proxy-cluster static_resources: listeners: - name: listener_0 address: socket_address: address: 0.0.0.0 port_value: 10000 filter_chains: - filters: - name: envoy.filters.network.http_connection_manager typed_config: "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager stat_prefix: ingress_http route_config: name: local_route virtual_hosts: - name: backend domains: ["*"] routes: - match: prefix: "/" route: cluster: edge_cluster host_rewrite_literal: "http.edgecompute.app" cors: allow_origin_string_match: - prefix: "*" allow_methods: GET, PUT, DELETE, POST, OPTIONS allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout max_age: "1728000" expose_headers: custom-header-1,grpc-status,grpc-message access_log: - name: envoy.access_loggers.stdout typed_config: "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog - name: envoy.http_grpc_access_log typed_config: "@type": type.googleapis.com/envoy.extensions.access_loggers.grpc.v3.HttpGrpcAccessLogConfig common_config: log_name: "sigsci-agent-grpc" transport_api_version: V3 grpc_service: envoy_grpc: cluster_name: sigsci-agent-grpc timeout: 0.2s additional_request_headers_to_log: # These sigsci-agent headers are required for correct processing: - "x-sigsci-request-id" - "x-sigsci-waf-response" # Optionally, additional headers can be added that should be recorded: - "accept" - "content-type" - "content-length" additional_response_headers_to_log: - "date" - "server" - "content-type" - "content-length" http_filters: # ext_authz filter calls the NGWAF agent via gRPC. - name: envoy.filters.http.ext_authz typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz transport_api_version: V3 grpc_service: envoy_grpc: cluster_name: sigsci-agent-grpc failure_mode_allow: false - name: envoy.filters.http.router typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router clusters: # Cluster for forwarding requests to https://http.edgecompute.app. - name: edge_cluster connect_timeout: 0.25s type: LOGICAL_DNS lb_policy: ROUND_ROBIN load_assignment: cluster_name: edge_cluster endpoints: - lb_endpoints: - endpoint: address: socket_address: address: http.edgecompute.app port_value: 443 transport_socket: name: envoy.transport_sockets.tls typed_config: "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext sni: "http.edgecompute.app" # Cluster used by the ext_authz filter to contact the NGWAF agent. - name: sigsci-agent-grpc connect_timeout: 0.25s type: STATIC lb_policy: ROUND_ROBIN http2_protocol_options: {} # <--- This tells Envoy to use HTTP/2 for gRPC load_assignment: cluster_name: sigsci-agent-grpc endpoints: - lb_endpoints: - endpoint: address: socket_address: # Both containers share the same network namespace. address: 127.0.0.1 port_value: 9999 --- # Deployment: Combines the Envoy proxy and the Next‑Gen WAF Agent sidecar. apiVersion: apps/v1 kind: Deployment metadata: name: envoy-waf-deployment spec: replicas: 1 selector: matchLabels: app: envoy-waf template: metadata: labels: app: envoy-waf spec: containers: # Envoy container – set to use debug logging for detailed output. - name: envoy image: envoyproxy/envoy:v1.22-latest args: - "-c" - "/etc/envoy/envoy.yaml" - "-l" - "debug" # Change to "trace" for even more verbosity. ports: - containerPort: 10000 volumeMounts: - name: envoy-config mountPath: /etc/envoy # Next‑Gen WAF Agent container (sigsci-agent) configured for ext_authz (gRPC mode). - name: sigsci-agent image: signalsciences/sigsci-agent:latest imagePullPolicy: IfNotPresent env: - name: SIGSCI_ACCESSKEYID valueFrom: secretKeyRef: name: sigsci.my-site-name-here key: accesskeyid - name: SIGSCI_SECRETACCESSKEY valueFrom: secretKeyRef: name: sigsci.my-site-name-here key: secretaccesskey # Instruct the agent to expect response data and configure its gRPC address. - name: SIGSCI_ENVOY_EXPECT_RESPONSE_DATA value: "1" - name: SIGSCI_ENVOY_GRPC_ADDRESS value: "localhost:9999" - name: SIGSCI_DEBUG_ALL_THE_THINGS value: "true" - name: SIGSCI_WAF_DATA_LOG value: "/dev/stdout" ports: - containerPort: 9999 securityContext: readOnlyRootFilesystem: true volumes: - name: envoy-config configMap: name: envoy-config --- # Service: Exposes the Envoy listener on port 10000. apiVersion: v1 kind: Service metadata: name: envoy spec: selector: app: envoy-waf ports: - protocol: TCP port: 10000 targetPort: 10000 ``` ## Next steps [Verify the agent and module installation](https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/module-agent-deployment/about-module-agent-deployment#verifying-your-installation) and [explore module options](https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/module-agent-deployment/about-module-agent-deployment/). ## Related content - [About reverse proxy deployment](https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/reverse-proxy-deployment/about-reverse-proxy-deployment) - [Using an API with the Next-Gen WAF](https://www.fastly.com/documentation/guides/next-gen-waf/developer/using-an-api-with-the-next-gen-waf)