Kubernetes

• 1: Deploying on Kubernetes
• 2: Kubernetes Gateway API
• 3: Kubernetes Ingress v1
• 4: Kubernetes Cache Policy
• 5: Kubernetes Controller RBAC

1 - Deploying on Kubernetes

deploy/kube carries two raw-YAML deployments. configmap.yaml, deployment.yaml and service.yaml run Trickster as a caching proxy with the exhaustive example configuration; the gateway-*.yaml manifests run it as the Kubernetes Gateway API and Ingress controller described in kubernetes-gateway.md and kubernetes-ingress.md. The Helm chart at https://github.com/trickstercache/helm-charts is built from these manifests, so what they do is what the chart does with its values applied.

What is in deploy/kube

Installing the controller

The Gateway API is an add-on, installed from its own release. Install the version the controller is built against (sigs.k8s.io/gateway-api in go.mod); the standard channel carries the v1 kinds, and the experimental channel additionally carries HTTPRoute retries and the TCPRoute, TLSRoute and UDPRoute kinds. The TricksterCachePolicy definition ships in deploy/kube/crds. Both go in before the controller starts, since it probes for them once at startup:

make -C deploy/kube install-gateway-crds                                    # standard channel
make -C deploy/kube install-gateway-crds GATEWAY_API_CHANNEL=experimental

A cluster that will never serve the Gateway API skips the first apply and the GatewayClass in gateway-classes.yaml; the controller then serves Ingress objects alone. Then the controller itself, in this order:

make -C deploy/kube bootstrap-trickster-gateway

which applies the namespace, RBAC, classes, ConfigMap, Deployment, Service and PodDisruptionBudget and waits for the rollout. A pod is ready only once its controller has translated the cluster’s routing objects and the data plane serves them, so the rollout completing means the controller is serving. Verify what it claimed:

kubectl get gatewayclass trickster            # ACCEPTED True
kubectl get ingressclass trickster
kubectl -n trickster get service trickster-gateway   # the published address
kubectl get gateway,httproute,ingress -A      # PROGRAMMED, the address, and Accepted parents

A Gateway of the class opens the ports it declares; an Ingress naming the class is served on the listeners the ConfigMap configures:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: edge
  namespace: shop
spec:
  gatewayClassName: trickster
  listeners:
    - name: http
      port: 80
      protocol: HTTP

Configuration

gateway-configmap.yaml is the primary configuration file, mounted at /etc/trickster/trickster.yaml. It carries what the cluster’s routing objects cannot: the listeners claimed Ingresses are served on, the listener the probes use, logging, the kubernetes section, and the operator tier (kubernetes.defaults) no route may change. Everything the controller generates from Gateways, routes and Ingresses is merged after it, through the same validation, and may only add objects: it can never change main, logging, mgmt or a listener the file defines. Every option is described in configmap.yaml, which is the example configuration wrapped in a ConfigMap.

Fragments in conf.d

A second ConfigMap, trickster-gateway-conf-d, is mounted at /etc/trickster/conf.d and is optional. Each of its keys becomes a file there, and every .yaml, .yml or .conf file in the directory is merged into the primary file in name order, as described under Multiple Configuration Files. It is where the operator tier is provisioned without touching the primary file: the caches, negative caches, tracers, request rewriters and authenticators that kubernetes.defaults, a GatewayClass’s parameters, a TricksterCachePolicy or an Ingress annotation may then select by name.

apiVersion: v1
kind: ConfigMap
metadata:
  name: trickster-gateway-conf-d
  namespace: trickster
data:
  caches.yaml: |
    caches:
      objects:
        provider: memory
        index:
          max_size_bytes: 536870912
  authenticators.yaml: |
    authenticators:
      gateway-auth:
        provider: basic
        users:
          api: ${API_PASSWORD_HASH}

A chart renders its values into exactly these fragments; a hand-run deployment edits them in place.

How a change reaches the pod

Two patterns, which may be combined:

In-place reload. The manifests set mgmt.auto_reload_interval, so the running process polls its files and reloads when one changes, through the same validation and graceful path a SIGHUP uses. The kubelet refreshes a mounted ConfigMap within about a minute of an edit (its sync period), by an atomic symlink swap the poll sees and filesystem notifications do not, which is why polling is the mechanism. A reload that fails validation leaves the previous configuration serving, logged as a failed reload; caches and listeners whose settings did not change are kept, and a listener whose settings did change is restarted with a drain. The volumes are whole-volume mounts and never subPath mounts: a subPath mount is a copy the kubelet does not refresh, so the poll would see nothing.

Rollout on a checksum. A chart that wants every configuration change to pass through a rollout, with the Deployment’s history as its rollback, puts a hash of the rendered ConfigMap in a pod template annotation (checksum/config in gateway-deployment.yaml); a changed hash rolls the pods. Readiness holds each new pod out of the Service until its controller has programmed the routes, and the old pod drains only after it has been removed, so the rollout is invisible to clients. This is the pattern for a change the in-place reload cannot make, and for a Secret consumed through the environment.

Secrets

Credentials never go in a ConfigMap. The configuration expands ${VARIABLE} in its credential-bearing fields, listed under Configuring Secrets: authenticator users, a Redis password, a user router’s to_credential, and header values in path, CORS, health check and discovery blocks. Mount a Secret into the environment (envFrom.secretRef in gateway-deployment.yaml) and reference its keys. A container’s environment is fixed at start, so a rotated Secret reaches the pod through a rollout, not a reload. TLS certificates for HTTPS Gateway listeners and Ingress TLS sections are read from the kubernetes.io/tls Secrets the objects reference and pushed into the listeners at runtime, rotated without a reload; no certificate is mounted.

Ports and addresses

A Gateway listener binds the port it declares, in the pod, and gateway-service.yaml forwards by port number, so a Gateway declaring 80 is reached at the load balancer’s port 80. The pod runs as an unprivileged user and binds 80 and 443 through the net.ipv4.ip_unprivileged_port_start sysctl the Deployment sets, which every current Kubernetes release treats as safe. The alternative, adding NET_BIND_SERVICE to the container, does not reach a non-root process: the capability is dropped at execve unless the binary carries file capabilities, which the image does not set. A Gateway declaring some other port needs a matching Service entry.

A port is bound once, so a port belongs either to a Gateway or to a listener configured in the ConfigMap. The manifests give 80 and 443 to Gateways and serve Ingresses on 8080 and 8443, which the Service also forwards. To serve Ingresses on 80 and 443 instead, move the web and websecure listeners to those ports in the ConfigMap and have Gateways declare others; the Service forwards by number and needs no change.

The addresses published into Gateway and Ingress status are those of the Service named by kubernetes.published_service, a LoadBalancer here. On a cluster with no load balancer implementation, give the Service externalIPs or make it a NodePort, or install one such as MetalLB; the status address follows whatever the Service is assigned.

Every Gateway of the class is served by every replica at the one published address, and two Gateways declaring one port merge on it as described under Listeners. A deployment that needs Gateways at distinct addresses runs one controller Deployment per GatewayClass: a copy of the manifests with its own namespace or names, its own Service, and a distinct gateway_class_controller_name (and ingress_class) that only that class names.

Scaling and availability

The Deployment starts two replicas spread across nodes, with a PodDisruptionBudget keeping one through drains. Every replica watches the cluster, translates and serves every route; one, elected over the Lease kubernetes.leader_election names, writes status and Events, so adding replicas adds data-plane capacity and nothing else, which is what the optional HorizontalPodAutoscaler scales on. The resource requests are starting points.

A rolling update surges one pod in before one goes (maxSurge: 1, maxUnavailable: 0), and terminationGracePeriodSeconds covers the preStop sleep (which lets the endpoint removal propagate before SIGTERM) plus mgmt.shutdown_drain_timeout, with margin; see Graceful shutdown.

RBAC

gateway-rbac.yaml grants exactly the verbs kubernetes-rbac.md lists, cluster-wide, because the shipped configuration watches every namespace. A controller narrowed with kubernetes.watch_namespaces binds the namespaced kinds through a Role in each watched namespace and keeps only gatewayclasses, ingressclasses and namespaces cluster-wide; read_only: true drops every write. The endpoint routing mode adds endpointslices, left commented in the file.

The image

trickstercache/trickster (also ghcr.io/trickstercache/trickster) is a statically linked binary built with CGO_ENABLED=0 on a distroless static base: no shell, no package manager, no libc. It runs as nobody (65534) and the manifests pin that with runAsNonRoot and a read-only root filesystem. The image carries the CA bundle, the license notices of every linked dependency under /licenses, and the example configuration at /etc/trickster/trickster.yaml, which the mounted ConfigMap replaces. Images are signed; the README shows the cosign verify invocation. The Kubernetes client and Gateway API libraries the controller is built on are a fixed part of the binary whether or not the kubernetes section is configured, and a Trickster without that section holds no watches and opens no API connection.

The plain caching proxy

configmap.yaml is examples/conf/example.full.yaml wrapped in a ConfigMap, generated by make kube-configmap and checked in CI, so the two never drift; edit the example, not the ConfigMap. deployment.yaml mounts it and service.yaml exposes the proxy and metrics ports. graphite-deployment-patch.yaml and mysql-deployment-patch.yaml layer an in-cluster Graphite and a credential-bearing MySQL configuration on top of it; each file describes how it is applied.

2 - Kubernetes Gateway API

Trickster serves the gateway.networking.k8s.io/v1 objects it claims — GatewayClass, Gateway, HTTPRoute and ReferenceGrant — translating them into its own configuration and reloading onto it. Enable the controller with the top-level kubernetes section (configuring.md) and grant it the permissions in kubernetes-rbac.md. Ingress v1 is served alongside; see kubernetes-ingress.md. Caching behavior — a cache, TTLs, a time series provider, the cache key — is attached to Gateways, routes and Services with the TricksterCachePolicy resource; see kubernetes-cache-policy.md.

The Gateway API is optional at runtime. Its CRDs are an add-on that most clusters do not have, so the controller probes for the group at startup and, where it is absent, watches none of its kinds and serves Ingress alone.

What is claimed

A GatewayClass belongs to this controller when its spec.controllerName equals kubernetes.gateway_class_controller_name. A Gateway belongs to it when its spec.gatewayClassName names a claimed class, and a route of any served kind (HTTPRoute, GRPCRoute, TCPRoute, TLSRoute, UDPRoute) attaches through a parentRef naming a claimed Gateway. Anything else is ignored entirely: not translated, not counted, and never statused. A parentRef naming a Gateway this controller does not claim is another controller’s business and draws no complaint.

GatewayClass parameters

A GatewayClass may carry a parametersRef to a ConfigMap. Its data keys are the kubernetes.defaults fields, spelled the same way, and override those defaults for every route served through a Gateway of the class:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: trickster-cached
spec:
  controllerName: trickstercache.org/gateway-controller
  parametersRef:
    group: ""
    kind: ConfigMap
    name: cached-gateway-params
    namespace: trickster
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: cached-gateway-params
  namespace: trickster
data:
  cache_name: objects
  negative_cache_name: api-errors
  timeout: 45s
  authenticator_name: gateway-auth

Unlike an Ingress annotation, a GatewayClass may set the operator-tier names, because a GatewayClass is cluster-scoped infrastructure and whoever can write its ConfigMap decides the defaults for the class. A TricksterCachePolicy on a Gateway, an HTTPRoute, one of its rules, or a backend’s Service is written over the class’s parameters, most specific last; see kubernetes-cache-policy.md.

Because those parameters carry operator controls, a class whose parameters cannot be honored is not served: none of its Gateways open a port and none of the routes attached to them are emitted, until the parameters are fixed. That covers a parametersRef that cannot be read (wrong kind, no namespace, ConfigMap absent or outside the watched namespaces) and any key that is unknown, does not parse, or names something the configuration does not define. Every affected Gateway is reported with the reason. A ConfigMap with no keys is a class with no overrides, and is served.

Listeners

Each Gateway listener becomes a Trickster listener bound to the declared port. HTTP opens a plaintext port; HTTPS opens a TLS port with tls_runtime_certs, whose certificates arrive from the referenced Secrets at runtime; TCP, TLS and UDP open a stream listener of the matching protocol, which relays what it receives without reading it. Listeners on one port — within a Gateway or across Gateways — merge onto one Trickster listener, because a port can be bound once; a UDP listener binds in its own port space, so it may share a port number with a TCP, TLS, HTTP or HTTPS listener. Two listeners on one port must serve one protocol; where they do not, the listener of the older Gateway (then the earlier listener) keeps the port and the other is reported and not served. Listeners of different Gateways on one port may share a hostname, or have none: each admits its own routes and the port serves the union, with overlapping routes resolved by the ordinary precedence (the older route wins). On HTTPS the merged listeners’ certificates share one store, which answers for a name with the first certificate carrying it, so two listeners on one port, of one Gateway or two, may not bring different certificates carrying one name (a wildcard being a name of its own); the older Gateway’s listener, then the earlier one, keeps the name and the other is reported with HostnameConflict and not served until a rotation ends the overlap. A Secret rotated to unusable material withdraws nothing, so on a port whose store holds a certificate under it that certificate keeps its names until a usable rotation replaces it, while on a port holding nothing under it the Secret claims nothing. Two Gateways naming one hostname on one port must also name the same certificate. Hostless HTTPS listeners merge whatever their certificates, each selected by the names it carries, so long as those names are disjoint. A Gateway may not repeat a hostname on a port within itself, which the API refuses before it reaches the controller.

hostname may be precise or a single leading wildcard label (*.example.com). It restricts the routes the listener admits, as described below. A TCP or UDP listener cannot carry one, since nothing in the stream names a host; a listener that declares one is refused with UnsupportedValue. A TLS listener’s hostname is matched against the server name a client offers.

On an HTTPS listener tls.mode must be Terminate (the default). Every certificateRefs entry must name a kubernetes.io/tls Secret, in the Gateway’s namespace or in one whose ReferenceGrant permits it. A listener that cannot terminate TLS at all — no tls block, no certificateRefs, or Passthrough — is not served. A reference that merely does not resolve yet is reported and the listener still opens, so that creating the Secret later heals it without any further change. Certificates never travel in configuration: they are supplied to the running listener and re-supplied when the Secret changes, so a rotation costs no reload. See tls.md for how a listener holds them and for the certificate inventory that lists them.

On a TLS listener tls.mode must be Passthrough: the listener relays the client’s handshake to the backend the server name selects and terminates nothing, so it holds no certificate and certificateRefs are ignored. The mode defaults to Terminate, which is refused with UnsupportedValue rather than assumed; a TLS listener that would terminate and forward plain TCP is not served in this release.

allowedRoutes.namespaces.from is honored: Same (the default), All, and Selector against the labels of the route’s Namespace. allowedRoutes.kinds may list the kinds the listener’s protocol carries: HTTPRoute and GRPCRoute on HTTP and HTTPS, TCPRoute on TCP, TLSRoute on TLS and UDPRoute on UDP. Any other kind is reported as unsupported on the listener, and a listener admitting no supported kind admits no route.

Deviation: allowedRoutes decides which routes attach to a listener; it does not fence requests. Listeners on one port share one router, which resolves a request by hostname across every route served on the port, so a request for shop.example.com whose path no route on the shop.example.com listener claims may be answered by a route attached to a *.example.com listener on the same port — including a route from a namespace the precise listener would not have admitted. The Gateway API’s GatewayHTTPListenerIsolation feature, under which the most specific listener alone answers its hostname, is not supported.

A listener no route has attached to is bound all the same: a generated backend of its own answers 404 for whatever arrives, so the port is open and an HTTPS listener already holds its certificates before its first route, and the listener is Programmed as soon as it is declared.

Not supported in this release, reported and ignored: spec.addresses (addresses come from kubernetes.published_service), spec.tls, and spec.allowedListeners (ListenerSets). A Gateway naming spec.infrastructure.parametersRef is refused with InvalidParameters and not served, since nothing reads Gateway parameters and serving it without them would misrepresent what was asked.

Binding a port below 1024 inside a container requires running as root, granting NET_BIND_SERVICE, or the net.ipv4.ip_unprivileged_port_start sysctl the deployment in deploy/kube sets (kubernetes-deploy.md); a Gateway declaring port 80 behind a Service that maps it from a high container port is not yet expressible, so declare the port the pod may bind.

Route attachment and hostnames

A parentRef selects a claimed Gateway, optionally narrowed by sectionName or port. The route attaches to each selected listener whose allowedRoutes admit it and whose hostname intersects the route’s:

• a listener with no hostname admits whatever the route names, or every hostname when the route names none;
• a route naming no hostname takes the listener’s;
• otherwise each route hostname is kept when it equals the listener’s or falls within its wildcard (*.example.com admits shop.example.com and deep.shop.example.com), and a wildcard route hostname is narrowed to a precise listener it covers.

When both sides name hostnames and none agree, the route is not accepted on that listener, and a parentRef none of whose listeners accept the route is reported with the reason.

The route is then served once per port and hostname it was accepted on. Listeners that merged onto one port serve it for the union of the hostnames they admitted.

A wildcard hostname spans any number of labels, as the Gateway API defines: a route served on *.example.com answers shop.example.com and deep.shop.example.com, but not example.com. See paths.md for the order hosts are tried in.

Matches and precedence

A Kubernetes PathPrefix matches whole path elements: /foo matches /foo and /foo/bar but never /foobar, and a trailing slash means nothing.

Requests are resolved by host first — the exact hostname, then wildcards covering it from the nearest label boundary outward, then hostless routes — then by path, exact before longest prefix before regular expression, then by method, and finally by the match’s header and query parameter conditions, most specific first. The Gateway API’s own precedence decides among matches that meet on one path and method: a match naming a method first, then more header matches, then more query parameter matches, then the older route.

• A header or query value is compared whole, and a RegularExpression one must match the whole value. The field must be present to match at all: a pattern the empty string satisfies, such as .*, matches a field that is present and empty but not one that is absent. A match naming one header or query parameter twice uses the first condition, whatever the case.
• A method that no match claims on a path some match did claim reaches the covering match on the same host where one exists, so a POST to a path that only names GET still reaches the prefix behind it, and answers 404 otherwise. That fill does not cross into a less specific host tier.
• Two matches testing the same path, method and predicates are a duplicate: the older route keeps it (ties on creation time break on namespace then name) and the other is reported. An Exact match outranks a PathPrefix on the same path and a match naming a method outranks one that does not, whatever their ages, which is the order the Gateway API defines.

Backend references

Each backendRefs entry must be a Service (the default kind) with a port that exists on it. A reference into another namespace needs a ReferenceGrant in that namespace permitting HTTPRoute from the route’s namespace to Service, optionally by name. weight defaults to 1; a weight of 0 sends no traffic and drops the reference.

The request reaches the Service with the Host header the client sent, and cached objects are keyed by it, so two hosts one backend serves never share an object; a URLRewrite hostname replaces it, and then keys nothing. A Service port whose appProtocol is kubernetes.io/h2c is spoken to in cleartext HTTP/2 by prior knowledge. A headless Service (clusterIP: None) resolves to its pods: under service routing it is dialed on the port’s numeric targetPort, and one naming its targetPort by name, which only its pods resolve, is refused there with UnsupportedValue; under endpoint routing the pods are discovered through the EndpointSlices, which name the port, so both shapes are served. The routing mode that decides is the one the route’s policies leave in force (class parameters, route, rule, then the Service’s own policy).

A reference that cannot be resolved — unsupported kind, no permitting grant, Service or port absent — keeps its slot and its weight, and its share of the traffic answers with a fixed 500, which is what the Gateway API requires. A rule with no usable reference answers 500 outright. Because Services are watched, creating the Service later heals the route. Several references become a weighted round-robin pool, apportioned exactly by integer weight.

BackendTLSPolicy

A BackendTLSPolicy whose targetRefs name a Service, optionally one named port of it through sectionName, makes every connection to that Service TLS, verified as the policy says:

apiVersion: gateway.networking.k8s.io/v1
kind: BackendTLSPolicy
metadata:
  name: web-tls
  namespace: shop
spec:
  targetRefs:
    - group: ""
      kind: Service
      name: web-svc
      sectionName: https
  validation:
    caCertificateRefs:
      - group: ""
        kind: ConfigMap
        name: web-ca
    hostname: web.shop.internal

validation.hostname is sent as the SNI and is the name the certificate is verified against. caCertificateRefs name ConfigMaps, or kubernetes.io/tls Secrets, in the policy’s namespace whose ca.crt key holds a PEM bundle; wellKnownCACertificates: System trusts the system store instead. The two are alternatives, as the API defines them: a bundle from caCertificateRefs is the whole of the trust for that backend, so a certificate from a public authority is refused, and only a policy selecting System trusts the system roots. The bundle travels in configuration, so a rotated bundle costs a reload; a CA certificate is public material. A port-specific policy outranks one for the whole Service, and of two policies selecting one target the older wins and the other is reported.

A policy that cannot be honored — a reference that does not resolve or holds no certificate, a wildcard hostname, subjectAltNames, an unknown well-known set — makes every backendRef it governs invalid: the reference answers 500 as an unresolvable one does, because connecting without the verification the policy asked for would be worse. options are implementation-specific and are reported and ignored. Only Service targets are supported.

Filters

Filters on a rule apply to every request it matches; filters on a backendRef apply after them, to the requests that reference receives.

RequestHeaderModifier, ResponseHeaderModifier, URLRewrite, RequestRedirect and RequestMirror are supported.

Within one header modifier a header may be named once, across set, add and remove and whatever its case; a filter naming Authorization under set and authorization under remove has no defined order and makes the route not served. Header names are case-insensitive throughout. Across filters, modifications fold into one operation per header in the order they apply, the rule’s filters before a backendRef’s: a set followed by an add of the same header sets the joined value; an add after a remove is a set; a remove after a set removes.

replacePrefixMatch replaces the declared prefix on a segment boundary, so with a PathPrefix of /api and a replacement of /v2, /api becomes /v2, /api/ becomes /v2/ and /api/orders becomes /v2/orders; an empty replacement is /. It requires the rule to have exactly one match, of type PathPrefix, and on a backendRef it also requires the rule to have exactly one backendRef: behind a weighted dispatch the member no longer knows which prefix was matched. A rule and one of its backendRefs may not both rewrite the path.

RequestRedirect answers with the status it names (301, 303, 307 or 308, or 302 by default) and a Location composed from the request as the filter leaves it: scheme, hostname, port and path replace their parts of the request URL and the rest is kept. An explicit scheme with no port sends the client to the scheme’s well-known port; neither scheme nor port sends it back to the Gateway listener’s port, whatever port the request’s Host header carried; and a port that is the well-known one for the scheme is omitted. A redirecting rule forwards nothing, not even a request carrying Connection: Upgrade, which is answered with the redirection like any other. It is still a backend the operator’s controls apply to: an authenticator from kubernetes.defaults or the GatewayClass’s parameters guards it exactly as it guards a forwarding rule, and a ResponseHeaderModifier on the same rule applies to the redirection. backendRefs written on a redirecting rule are ignored and reported, and it may not also carry a URLRewrite. Filters apply in the order declared, and a redirect ends the request: a RequestHeaderModifier ahead of it modifies the request the redirect answers, so a Host it sets is the Location’s hostname when the redirect names none, while one after it has no request left to modify and is reported and ignored. The redirect filter alone defines the Location: a ResponseHeaderModifier on the same rule that sets, adds or removes Location, in any case, makes the route not served rather than being applied in one order or the other and silently losing to the redirect. Express the target through the redirect’s own scheme, hostname, port and path.

RequestMirror copies the share of requests it names — percent, or fraction rounded to a whole percent — to the Service its backendRef resolves to, discarding their responses; see Mirroring Requests for what the data plane does with a copy. The reference resolves exactly as a forwarding backendRef does, ReferenceGrants and BackendTLSPolicy included. One that does not resolve is the one filter failure the API defines as partial: the mirror is dropped, the route’s ResolvedRefs is False, and the rest of the route is served. A mirror on a rule fires once per request, ahead of a weighted dispatch; one on a backendRef fires for the requests that reference receives, and a rule served by one backendRef carrying both fires both. A filter copying nothing (percent: 0) makes the route not served.

A filter this build cannot honor — ExtensionRef, CORS, ExternalAuth, a repeated filter type, a malformed hostname, header, scheme, port or status — makes the route not served until it is fixed, because ignoring a filter would serve the route wrongly rather than partially.

Timeouts and retry

A rule’s timeouts and retry apply to every request it serves, behind a weighted dispatch included; see Timeouts and Retries for their exact semantics. timeouts.request bounds the whole upstream exchange, retries included, and timeouts.backendRequest each attempt; a zero duration bounds nothing, and backendRequest may not exceed request. A request that runs out of either with no attempt answered is answered 504. retry repeats an idempotent request after a connection failure and after a response whose status is in codes, attempts times (once when unset, at most ten), waiting backoff between attempts. The Gateway API defines no retry budget, so every eligible request is retried. A value the data plane cannot express makes the route not served, and sessionPersistence is reported and ignored.

GRPCRoute

A GRPCRoute attaches, ranks and resolves exactly as an HTTPRoute does, and lowers onto what gRPC is on the wire: every call is a POST to /{service}/{method}. A method match with service and method is an exact path; service alone is the path prefix beneath the service; method alone matches the method under any service; and a RegularExpression type matches each segment as a pattern, an absent one matching any segment. A headers match is a header match. RequestHeaderModifier, ResponseHeaderModifier and RequestMirror are the HTTPRoute’s filters; ExtensionRef is not honored. The generated backends select the proxy handler, which relays trailers, and speak cleartext HTTP/2 by prior knowledge to a plaintext Service, or HTTP/2 over TLS to one a BackendTLSPolicy governs, since gRPC needs HTTP/2 end to end. A TricksterCachePolicy on a Gateway or a Service still governs a GRPCRoute’s backends for the operator-tier controls, but a GRPCRoute caches nothing and a policy cannot target one.

An HTTP or HTTPS listener admits both kinds by default and allowedRoutes.kinds may restrict it to one; supportedKinds says which. An HTTPRoute and a GRPCRoute attached to one listener with intersecting hostnames cannot both be served: the older is, and the newer is refused with Accepted: False and reason RouteConflict, which is the resolution the Gateway API defines. The controller watches GRPCRoutes only in a cluster whose Gateway API serves the kind; see kubernetes-rbac.md.

TCPRoute, TLSRoute and UDPRoute

The stream route kinds attach as an HTTPRoute does, through a parentRef naming a claimed Gateway, narrowed by sectionName or port and subject to the listener’s allowedRoutes; a TCPRoute attaches only to TCP listeners, a TLSRoute only to TLS listeners and a UDPRoute only to UDP listeners, and one naming a listener of another protocol is refused with NotAllowedByListeners. Each is served with exactly one rule, since nothing in a stream selects among rules; a route declaring more is refused with UnsupportedValue. The rule’s backendRefs resolve as an HTTPRoute’s do (a Service port, ReferenceGrants for another namespace), with their weights, and an unresolvable reference keeps its share of the connections and refuses them, as the Gateway API requires; a BackendTLSPolicy does not apply, since the stream is relayed unread.

A TCP or UDP listener carries one route: the oldest route naming it is served and every later one is refused with RouteConflict. A TLSRoute’s hostnames are the server names it serves, intersected with the listener’s hostname as an HTTPRoute’s are; a route naming none serves every server name the listener admits. Listeners on one port bind one socket, so each server name on a port belongs to one route, the oldest, however many listeners on that port admitted it: a route admitted by a wildcard listener and a precise one on the same port is served once there, a newer route that names only names already served on the port is refused with RouteConflict, and one that names some is served for the rest and told which it lost. A route admitted by several Gateways on one port takes the first Gateway’s policy. A connection is relayed to the route whose hostname matches the server name it offers, a precise name before a wildcard, and a connection whose server name no route serves, or that offers no ClientHello at all, is closed.

A backendRef’s port is resolved by the route’s transport: a TCPRoute or TLSRoute selects the Service port of that number carrying TCP and a UDPRoute the one carrying UDP, so a Service exposing one port number over both, with different names and targets, is reached at the right target in either mode; a Service exposing the number over the other transport alone is an unresolved reference, keeping its weight and refusing its share.

Every connection, and every UDP client’s session, is committed to one backendRef by weighted round robin, and in the endpoint routing mode to one of that Service’s ready endpoints in turn; a member that cannot be dialed refuses the connection rather than passing it to a sibling, an unresolved reference refuses its share without a lookup, and it is the Service’s readiness that takes an endpoint out of rotation. The health_mode parameter does not apply to stream members, which are always judged by readiness, since no probe speaks the protocol they carry. A stream route caches nothing, and a TricksterCachePolicy cannot target one; kubernetes.defaults and a GatewayClass’s parameters reach it only for routing_mode.

The controller watches the three kinds only in a cluster whose experimental Gateway API channel serves them (gateway.networking.k8s.io/v1alpha2); see kubernetes-rbac.md. Their status is written as an HTTPRoute’s is.

Endpoint routing mode

With routing_mode: endpoint, from kubernetes.defaults or a GatewayClass’s parameters, a backendRef is served by an ALB whose pool is the Service’s ready endpoints, discovered from its EndpointSlices, rather than a backend addressing the Service’s cluster IP. Each pool carries everything the backendRef would have carried — cache, timeout, TLS, filters. Endpoint churn reaches the pools without a configuration reload, and a rolling restart drains terminating endpoints before their pods stop. The controller’s service account needs endpointslices list and watch for it; see kubernetes-rbac.md.

health_mode decides how a discovered member is judged healthy: provider (the default) trusts the EndpointSlice’s readiness, which is what the pod’s own readiness probe already established; probe runs an active health check, configured by kubernetes.defaults.healthcheck or, when that is unset, a probe of the origin’s root every 5 seconds. See alb-autodiscovery.md for the semantics of both.

Status

Every claimed object is told what became of it, by one replica at a time (see Leader election):

• GatewayClass: Accepted is True for a claimed class, or False with reason InvalidParameters when its parametersRef cannot be honored, in which case none of its Gateways is served.
• Gateway: Accepted is True when every listener is valid, True with reason ListenersNotValid when only some are, and False when none is. Programmed follows Accepted, and is lowered to False with reason Pending when the data plane rejected the generated configuration, or AddressNotAssigned when kubernetes.published_service is configured but that Service has no address yet. status.addresses carries the published Service’s load balancer addresses, or failing those its external IPs.
• Gateway listeners: one entry per declared listener, with Accepted, Programmed, ResolvedRefs and Conflicted, supportedKinds (the kinds the listener’s protocol carries, those of them allowedRoutes.kinds admits, or nothing when it admits no kind this controller serves) and attachedRoutes, which counts only routes that are served. A refused listener keeps its entry and says why (UnsupportedProtocol, UnsupportedValue, ProtocolConflict, HostnameConflict, InvalidCertificateRef). A certificateRef that does not resolve lowers ResolvedRefs (InvalidCertificateRef, RefNotPermitted) without refusing the listener, and an unsupported route kind lowers it with InvalidRouteKinds.
• HTTPRoute (and every other route kind): one parents entry, under this controller’s name, per parentRef naming a claimed Gateway. Accepted is True when at least one listener accepted the route, or False with NoMatchingParent, NotAllowedByListeners, NoMatchingListenerHostname, RouteConflict for a stream route whose listener or server names an older route already serves, or UnsupportedValue for an unroutable hostname, a filter that cannot be honored, or a stream route with more than one rule, each of which unserves the whole route. ResolvedRefs is False with BackendNotFound, InvalidKind, RefNotPermitted or UnsupportedProtocol (a BackendTLSPolicy that cannot be honored) when any backendRef did not resolve; the route stays attached and the unresolved reference answers with an error. Entries other controllers wrote are left untouched, and a parentRef naming a Gateway this controller does not claim gets no entry.

Every condition’s observedGeneration is the generation that was translated, and a verdict is written only onto that generation of that object, so an object edited after translation, or recreated under the same name, is never labeled with a verdict about its predecessor.

A Gateway whose class stops being accepted keeps its Gateway and listener entries — the listeners Accepted on their own terms, Programmed: False because the Gateway is not served — and every HTTPRoute naming it is refused with NoMatchingParent and the class’s reason, so a class rejection reaches the routes rather than leaving them with the acceptance they had. An HTTPS listener is Programmed only while the data plane holds a usable certificate for it: one whose only material cannot be parsed is Programmed: False with reason Invalid, while a rotation to unusable material leaves the certificate already serving in place, so the listener stays Programmed with ResolvedRefs: False saying why the new material was not taken.

Conditions of other types and other controllers’ entries are kept, and an object whose status already says what the pass concluded is not written at all, so a resync that changes nothing makes no API call. A write the API server refuses is logged, counted in trickster_kgw_status_write_failures_total, and retried on the next pass. Status writing never delays the routes and certificates a pass carries. Status is not withdrawn from an object once it stops being claimed; the new owner overwrites it.

Events

Everything a pass could not do with an object is a Warning Event on that object, visible in kubectl describe: Rejected for a translation problem, InvalidAnnotation for a rejected annotation, InvalidCertificate for a TLS Secret that is missing or unusable, CertificateRejected for one a listener’s certificate store refused, InvalidParameters for a GatewayClass’s parameters, and Invalid, Conflicted or TargetNotFound for a TricksterCachePolicy. A claimed GatewayClass gets a Normal Accepted Event. An Event is published when the problem first appears, and again when the replica publishing it takes over leadership, not on every resync; client-go’s aggregation folds repeats and rate-limits per object.

Leader election

Every replica watches, translates and programs its own data plane. Status and Events are written by one replica, elected over a Lease (kubernetes.leader_election), so two replicas never fight over one object’s status. Every write is bound to the leadership term it began under, so a replica that loses the Lease stops writing at once and cannot overwrite what its successor wrote. A replica that loses the election keeps serving. leader_election.enabled: false makes every replica a writer, for a single-replica deployment that wants no Lease; read_only: true makes none of them one, and needs no write permission at all. The Lease lives in leader_election.namespace, the pod’s own namespace by default, and lease_duration must be at least one second because the Lease API stores it in whole seconds.

The leader is visible as trickster_kgw_leader; the controller’s other metrics are in metrics.md. When kubernetes.defaults.tracing_name names a tracer, every reconcile pass is traced on it: kgw.reconcile, with kgw.translate, kgw.compile, kgw.apply and kgw.certificates beneath it, and each status write as a kgw.status trace of its own, since it runs on its own worker.

Pod readiness

The readiness endpoint (/trickster/ready; see Graceful shutdown) reports 503 not programmed from before the listeners open until the controller’s first translation is serving, and 200 ready from then on. A translation the data plane rejects programs nothing, so the pod stays not ready until a later reconcile succeeds. A pod is therefore not added to its Service’s endpoints until the routes it exists to serve are in place, which is what makes a rolling update of the controller Deployment invisible to clients. A controller that cannot start, because the API server is unreachable or the service account lacks a permission, keeps the pod not ready, so the rollout waits rather than replacing a working pod with one that serves nothing. A reload that changes the kubernetes section restarts the controller without touching readiness, since the routes already programmed keep serving; a reload that enables the section after it was off holds readiness again until the new controller publishes.

Tuning for generic web ingress

The defaults suit a caching proxy in front of an API. A Gateway serving arbitrary web traffic through Trickster should consider these settings on the listeners it generates and in kubernetes.defaults:

A response the origin streams is relayed as it arrives on every handler; the caching handlers buffer only what they store. Range requests are served through the cache as described in range_request.md.

Conformance

The controller is tested against the upstream Gateway API conformance suite for the GATEWAY-HTTP profile on every change, against the experimental CRD channel. make kind-conformance (or kind-conformance-docker on macOS) runs it against a local kind cluster, and CI publishes the report as a workflow artifact.

The core result is partial rather than a core conformance claim: every core test passes but HTTPRouteMultipleGateways, which is skipped. It expects two Gateways declaring one port to answer at distinct addresses, which one process serving every Gateway of a class at one address cannot do; the Gateways merge on the port as described under Listeners. A deployment that needs Gateways at distinct addresses runs one controller Deployment per GatewayClass (kubernetes-deploy.md).

The report lists the extended features claimed. They cover query parameter, method and port matching, request and response header modification on rules and backendRefs, path, host, scheme and port rewrites and redirects, request mirroring, request and backend timeouts, retries, named route rules, and WebSocket and h2c backend protocols.

Not supported

Session affinity has no Gateway API equivalent here: a weighted rule apportions each request independently, so an origin that needs affinity should carry its own session state, or be served by a single backendRef in routing_mode: service so the Service’s own session affinity applies.

Generated configuration

Every listener, backend, ALB and request rewriter the controller generates carries the reserved kgw-- prefix and a name derived from the Kubernetes object’s identity, so the names you see in logs, metrics and the management API are stable across restarts. A route served on two listeners is served by one set of objects per port and hostname, each with its own cache keys, so an object cached through one port is not served through the other.

Generated configuration is merged onto the file configuration and then loaded, validated and applied by the same code that loads a configuration file, so it is exempt from no check. A translation the daemon rejects leaves the last good configuration serving and is not carried into later reloads.

3 - Kubernetes Ingress v1

Trickster serves Kubernetes networking.k8s.io/v1 Ingress objects it claims, translating them into its own configuration and reloading onto it. Enable the controller with the top-level kubernetes section (configuring.md) and grant it the permissions in kubernetes-rbac.md.

What is claimed

An Ingress belongs to this controller when:

• its spec.ingressClassName names an IngressClass whose spec.controller matches kubernetes.gateway_class_controller_name; or
• it names no class, and one of this controller’s IngressClasses is annotated ingressclass.kubernetes.io/is-default-class: "true"; or
• it carries the deprecated kubernetes.io/ingress.class annotation naming the configured kubernetes.ingress_class.

Setting kubernetes.ingress_class narrows ownership to that one class even when others name this controller, which is how two Trickster instances divide a cluster. Anything not claimed is ignored entirely: not translated, not counted, and never statused.

Listeners

An Ingress cannot describe the port it is served on, so it names the listeners instead — exactly as a backend names the listeners it is served on. They are ordinary listeners, configured in the listeners section, so their ports, bind addresses, body limits, timeouts and TLS settings are tuned the same way every other Trickster listener’s are:

listeners:
  web:
    port: 8080
    max_request_body_size_bytes: 10485760
  websecure:
    tls_port: 8443
    # the certificates arrive at runtime from the Secrets the claimed
    # Ingresses reference, so the port is kept open with none behind it
    tls_runtime_certs: true
kubernetes:
  ingress:
    listener_names:
      - web
      - websecure

Every claimed route is served on all of them. Naming none serves claimed Ingresses on the default frontend, which is where a backend that names no listener is served. A listener that does not exist fails validation, the same way an undefined listener named by a backend does.

A single listener may serve both ports, so listeners: {web: {port: 8080, tls_port: 8443, tls_runtime_certs: true}} named on its own is equally valid. Binding 80 and 443 inside a container requires running as root, granting NET_BIND_SERVICE, or the net.ipv4.ip_unprivileged_port_start sysctl the deployment in deploy/kube sets (kubernetes-deploy.md); an install behind a load balancer that terminates TLS simply configures no TLS port.

Rules and paths

A Kubernetes Prefix matches whole path elements: /foo matches /foo and /foo/bar but never /foobar, and a trailing slash means nothing, so /foo/ and /foo are the same rule. That is the router’s segment match type, so a Prefix path is one generated path; see paths.md.

The request reaches the Service with the Host header the client sent, so an application that reads it sees the Ingress host rather than its own Service name; cached objects are keyed by it, so a rule with no host serving several never hands one host’s object to another.

The referenced Service and port must exist in the cluster. A backend that cannot be resolved still answers, with a fixed 500, rather than silently vanishing from the routing table; the reason is logged. Because Services are watched, creating the Service later heals the route without any further action. backend.resource references are not supported.

Conflicts

Two Ingresses may claim the same host. The router resolves overlapping paths on its own — exact before prefix before regex, longest first within each tier — so the only genuine conflict is a duplicate of host and path once both have been lowered. That is broken in favor of the older object; ties on creation time are broken by namespace and name, so every replica reaches the same answer. A declaration is reported only when it loses everything it asked for, and keeps whatever else it claimed.

An Exact rule outranks a Prefix rule for the same path, whichever object declared it and whichever is older, which is the precedence Kubernetes defines. The prefix keeps everything below that path, so the two coexist and neither is reported as a conflict.

spec.defaultBackend answers whatever no rule matched, so it is emitted as a catch-all ordered after every other route, including a hostless regular expression rule. Because it is the controller’s own invention rather than something the operator wrote, any declared rule that lowers onto the same route takes it — a hostless / rule with use-regex, for instance — and the unreachable default backend is reported. Two Ingresses declaring a default backend do conflict, and the older one wins.

Regular expression paths are anchored before anything else looks at them, so /api/(.*) and ^/api/(.*) are one route rather than two that would race to register the same pattern.

TLS

Each spec.tls[].secretName must name a kubernetes.io/tls Secret in the Ingress’s own namespace. The certificate never travels in configuration: it is supplied to the TLS listener at runtime and re-supplied whenever the Secret’s contents change, so rotating a Secret costs no reload at all. See tls.md.

Certificates reach whichever of the named listeners actually serves TLS; that is a property of the listener’s own configuration, so a plaintext one is simply skipped. A listener’s certificates live only as long as the listener does, so a configuration change that removes one takes its certificates with it; they are installed again when it comes back, without the Secret having to change.

Because only kubernetes.io/tls Secrets are watched, a Secret of any other type reads as absent, and is reported that way.

Annotations

Annotations outside the trickstercache.org/ namespace are another controller’s business and are ignored. One inside it that is unknown, or whose value does not parse, is rejected: the annotation is not applied, the rest of the object still translates, and the rejection is logged. Failing the whole object would let one typo delete a live route; applying it silently would leave an operator believing a setting is in force when it is not.

Durations require a unit: 600 is rejected, 600s is not.

In a header list, a name prefixed with - deletes the header and one prefixed with + appends to it rather than replacing:

metadata:
  annotations:
    trickstercache.org/request-headers: |
      X-Forwarded-Host: shop.example.com
      -X-Internal-Token:
    trickstercache.org/response-headers: |
      +Vary: Accept-Encoding

cache-name and negative-cache-name select among things the operator has already configured; a name the configuration does not define is rejected like any other bad value, because emitting it would fail validation for the whole generated configuration and stop every other route in the cluster from reconciling. The route keeps serving, on the defaults.

That is the line the annotation set is drawn on: an annotation may select among what the operator provisioned, but never grant a capability or remove a control. What an annotation may not carry — a time series provider, the cache key components, hiding the X-Trickster-Result header — belongs on a TricksterCachePolicy targeting the Ingress or its Service, a resource with RBAC of its own; a policy on the Ingress is written over its annotations. See kubernetes-cache-policy.md. Anything on the far side of that line is an operator setting under kubernetes.defaults, where it applies to every generated backend and no Ingress author can change it:

authenticator_name in particular has no annotation and will not get one: an annotation that can name an authenticator is one that can also omit it, and whoever can create an Ingress in their own namespace would then be able to take their route out from behind authentication. A name in defaults that the configuration does not define fails startup, rather than the first reconcile, because it is the operator’s own mistake to see immediately.

max-ttl, cache-name and negative-cache-name describe caching, so they take effect only on a route that caches — one whose effective handler is proxycache, either from trickstercache.org/handler or from a configured kubernetes.defaults.cache_name. On a non-caching route they are emitted nowhere, because a non-caching backend would ignore them.

Rewriting

trickstercache.org/rewrite-target replaces the part of the path the rule matched:

• an Exact or regular expression match knows the whole path it matched, so the path is set outright;
• a Prefix match knows only its leading segments, so only those are replaced and the rest of the path is carried through.

With trickstercache.org/use-regex: "true", capture groups in the path are available to the target as ${1}, ${2}, and so on (${0} is the whole match, and named groups are available under their names). The pattern is anchored at the start for you, since the API server requires every Ingress path to begin with /:

metadata:
  annotations:
    trickstercache.org/use-regex: "true"
    trickstercache.org/rewrite-target: /v2/${1}
spec:
  rules:
    - host: shop.example.com
      http:
        paths:
          - path: /api/(.*)
            pathType: ImplementationSpecific
            backend:
              service:
                name: web-svc
                port:
                  number: 8080

A request for /api/orders reaches the Service as /v2/orders.

Endpoint routing mode

With kubernetes.defaults.routing_mode: endpoint, a rule’s backend is an ALB whose pool is the Service’s ready endpoints, discovered from its EndpointSlices, rather than a backend addressing the Service’s cluster IP: the controller generates a discovery entry over its own connection, a template backend carrying the rule’s settings, and a discovery-backed ALB whose query selects the Service’s port. Endpoint churn then reaches the pool without a configuration reload, and a rolling restart of the Deployment behind the Service drains terminating endpoints before their pods stop. The controller’s service account needs endpointslices list and watch for it; see kubernetes-rbac.md.

kubernetes.defaults.health_mode decides how a discovered member is judged healthy, and trickstercache.org/health-mode overrides it per Ingress: provider (the default) trusts the EndpointSlice’s readiness, which the pod’s own readiness probe established; probe runs an active health check from the generated template, configured by kubernetes.defaults.healthcheck or, when that is unset, a probe of the origin’s root every 5 seconds. See alb-autodiscovery.md for the semantics of both.

Status and Events

When kubernetes.published_service names the Service in front of Trickster, its load balancer addresses (or, failing those, its external IPs) are written into every claimed Ingress’s status.loadBalancer by the replica holding the leader election Lease; see kubernetes-gateway.md. Without it nothing is written there. Whatever could not be done with an Ingress — a rejected annotation, a missing TLS Secret, a path that could not be lowered, a host and path lost to an older Ingress — is a Warning Event on the Ingress, visible in kubectl describe ingress.

Beyond what an Ingress expresses

Ingress is served indefinitely. Settings an Ingress cannot express — a weighted canary, a redirect, TLS to a backend, TLS passthrough, per-rule timeouts and retries — are Gateway API features; see kubernetes-gateway.md. Both kinds are served at once, so a host may be moved to an HTTPRoute without disturbing the rest.

A TricksterCachePolicy may target an Ingress, one of its rules’ Services, or an HTTPRoute, so caching behavior can be moved off annotations independently. These are the equivalents:

A header list becomes a map with the same - and + prefixes. A rewrite-target using regular expression captures has no Gateway API equivalent: URLRewrite cannot reference captures, so a RegularExpression match paired with ReplaceFullPath rewrites to a fixed path only. A policy on an Ingress is written over its annotations; see kubernetes-cache-policy.md.

Generated configuration

Every backend, ALB and request rewriter the controller generates carries the reserved kgw-- prefix and a name derived from the Kubernetes object’s identity, so the names you see in logs, metrics and the management API are stable across restarts. Listeners are not generated for Ingress: the ones an Ingress is served on are the operator’s.

Generated configuration is merged onto the file configuration and then loaded, validated and applied by the same code that loads a configuration file, so it is exempt from no check. A translation the daemon rejects leaves the last good configuration serving and is not carried into later reloads.

4 - Kubernetes Cache Policy

TricksterCachePolicy is the custom resource that attaches Trickster’s caching behavior to the objects the Kubernetes controller serves. It carries everything the trickstercache.org/* Ingress annotations carry (kubernetes-ingress.md), and three things they never will: a time series provider, the cache key components, and whether the X-Trickster-Result header reaches the client. A custom resource has RBAC of its own, so whoever may create one is decided by the cluster rather than by whoever may edit a route.

Installing the resource

The definition lives at deploy/kube/crds/trickstercachepolicies.yaml. Install it before the controller starts:

kubectl apply -f deploy/kube/crds/trickstercachepolicies.yaml
kubectl wait --for=condition=Established crd/trickstercachepolicies.trickstercache.org

The controller probes for the resource once at startup, as it does for the Gateway API, and where the cluster does not serve it, serves every route without one. Installing the definition later takes effect at the next restart or kubernetes configuration change. The controller’s service account needs list and watch on trickstercachepolicies and update on trickstercachepolicies/status; see kubernetes-rbac.md.

Targets and precedence

A policy governs the objects its targetRefs name, in the policy’s own namespace:

group may be omitted; when given it must be the kind’s own.

Several policies may govern one rule. They are applied least specific first, each field an inner one sets written over the outer one’s, and a field none of them sets takes the configured default:

1. the GatewayClass’s parametersRef (kubernetes-gateway.md), or an Ingress’s annotations;
2. a policy on the Gateway;
3. a policy on the HTTPRoute or Ingress;
4. a policy on the route’s rule, by sectionName;
5. a policy on the backend’s Service, then one on the Service’s port.

Header updates fold in that order, one operation per header whatever its spelling: an inherited Authorization set and a more specific -Authorization leave the header removed, and a more specific +Vary appends to an inherited Vary. A map may name a header once, across spellings and operators, because a map is applied in no particular order. cacheKeyParams and cacheKeyHeaders replace as a whole: a policy that omits a list inherits it, and one that sets it empty (cacheKeyHeaders: []) clears what was inherited. A Service policy applies to one backendRef of a rule and not its siblings, so two members of a weighted rule may be served differently, resultHeader included.

Of two policies naming one target, the older one governs it (ties on creation time break on namespace then name) and the newer one is reported Conflicted on that target. A policy any field of which cannot be honored — an unknown value, a duration without a unit, a cacheName the configuration does not define — governs nothing at all and is reported Invalid, since a route governed by half a policy would look configured and be something else.

apiVersion: trickstercache.org/v1alpha1
kind: TricksterCachePolicy
metadata:
  name: metrics
  namespace: monitoring
spec:
  targetRefs:
    - kind: Service
      name: prometheus
      sectionName: http
  provider: prometheus
  cacheName: timeseries
  timeout: 60s
  resultHeader: Hide

Fields

In a header map a name prefixed with - deletes the header and one prefixed with + appends to it rather than replacing, exactly as in the annotations and in Trickster’s own request_headers. Durations require a unit. cacheName and negativeCacheName select among what the operator configured; the operator-tier names — a tracer, a request rewriter, an authenticator — have no field here either, for the reason given in the Ingress document: a policy that could name an authenticator could also omit one.

maxTTL, cacheName, negativeCacheName, cacheKeyParams and cacheKeyHeaders take effect only on a route that caches: one whose effective handler is proxycache, from handler, a configured cacheName, or a provider.

Provider-aware acceleration

provider makes the generated backend that provider — prometheus, influxdb, clickhouse or graphite — instead of a reverse proxy cache, so a request for /api/v1/query_range behind a prometheus policy reaches the Delta Proxy Cache and is served by time range rather than as an opaque object, exactly as it would from a hand-configured provider: prometheus backend. See the provider documents (prometheus.md, influxdb.md, clickhouse.md, graphite.md) for what each accelerates.

A provider never widens a route. Its accelerated paths are reachable only beneath the route’s own PathPrefix match, and carry that match’s headers, query parameters and methods: a request that does not satisfy the match, that names a method the match did not allow, or that asks for a provider path outside the prefix is served exactly as it would be without the provider — by the next route the router would have chosen, or not at all. A provider path that another route on the same host owns, by declaring it exactly or by a longer prefix, is left to that route.

Each accelerated path keeps the provider’s own cache key components (query, start, end, step for a range query), which are what make acceleration possible, and takes the policy’s cacheKeyParams and cacheKeyHeaders beside them, its header updates over the provider’s own, its CORS policy, its collapsed forwarding and its result header disposition. Behind a weighted or endpoint-mode dispatch, each member is accelerated under its own effective policy.

Three things follow:

• The route must expose the provider’s API at its native paths. The router selects the handler by the request path before any rewrite, so a PathPrefix of / or /api puts the provider’s paths in reach, while a route that serves Prometheus under /prom and rewrites the prefix away never lets a request reach /api/v1/query_range on the provider. Serve a provider at its own paths, on a hostname of its own where it has to share a listener.
• Only a PathPrefix match accelerates. An Exact match names one path, and a RegularExpression match cannot be intersected with the provider’s paths ahead of time; requests under either are served by the provider backend’s plain object cache.
• A route may not declare a path the provider predefines. An Exact match for /api/v1/query_range would take the path from the provider’s handler. The controller refuses that: the provider is withheld from that rule alone, the rest of the policy still applies, and both the route and the policy are told by Event. The root path is not a conflict, since every provider predefines it as a plain proxy catch-all.

The generated backend carries only what the policy and the configured defaults describe: an origin, a cache, timeouts, headers. Provider settings with no policy field — a Prometheus instant_round, an InfluxDB flux block, a Graphite render section — take their defaults. MySQL is served over its own wire protocol rather than HTTP and cannot be selected.

Cache key components

A cached object is stored under a key derived from the backend, the path and, for a caching path, whatever cacheKeyParams and cacheKeyHeaders name; see paths.md. The policy’s components reach every caching path the route serves, the provider’s predefined paths included, where they join the provider’s own. Name a header whose value partitions the cache — a tenant, an authorization scope — and every value gets an object of its own.

Exposing the result header

Every Trickster response carries X-Trickster-Result, which says how the request was handled (trickster-result.md). It is a debugging aid, and on a gateway in front of the public it discloses which paths are cached and how. resultHeader: Hide withholds it from the client on every path the policy governs. Metrics and the access log still record the result: the value is kept for the access log’s %{cache-status}x and %{engine}x fields, so an operator sees what the client does not; hand-written configuration does the same with the hide_result_header path option (paths.md). The disposition is the serving path’s: behind a weighted rule that hides the header, a member whose Service policy says Expose exposes it, and a response the dispatch answered itself, with no member selected, follows the rule’s.

Shared caches

The controller’s generated backends use whatever kubernetes.defaults.cache_name, a GatewayClass’s parameters, or a policy names, and the default in-memory cache when nothing does. Memory is right for one replica and for objects that are cheap to fetch again; it is per replica, so two replicas behind one Service each fetch and store their own copy, and a restart starts cold.

For several replicas, or for time series that are expensive to backfill, configure a Redis cache (caches.md) and name it:

caches:
  shared:
    provider: redis
    redis:
      endpoint: redis.trickster.svc:6379
kubernetes:
  defaults:
    routing_mode: service
    cache_name: shared

Every replica then reads and writes one store, a request served by any replica warms the cache for all of them, and a rolling restart keeps what was cached. A policy may still select a different configured cache for the routes it governs, so a shared store for time series and memory for everything else is one cacheName on one policy.

Cache keys are stable across restarts and replicas. They are derived from the Kubernetes object’s identity rather than the rule’s position, so inserting a rule does not invalidate what its neighbors cached. Renaming a route or moving it between namespaces changes the key; editing its rules does not.

Status and Events

The controller writes one status.ancestors entry per targetRefs entry, under its own controllerName, with an Accepted condition:

Status is written by the replica holding the leader election Lease, as every other status is (kubernetes-gateway.md), and by none under read_only. Entries other controllers wrote are kept. The same verdicts, and a provider withheld from a rule, are Warning Events on the policy, visible in kubectl describe trickstercachepolicy; the withheld provider is an Event on the route as well.

A policy’s target is judged to exist when the object is in a watched namespace, whether or not this controller claims it: a policy on an HTTPRoute attached to another controller’s Gateway is Accepted and has no effect.

5 - Kubernetes Controller RBAC

Every permission the Kubernetes Gateway/Ingress controller uses, and why. This is the source the Helm chart’s Role and ClusterRole are generated from; a verb that does not appear here is one the controller does not use.

The controller is entirely absent unless the top-level kubernetes configuration section is present, so a Trickster deployment that is not a gateway needs none of this.

Watched resources

The controller reads these to build its routing model. It never writes them themselves; the get verbs on the four statused kinds exist only to re-read an object whose status write was refused as a conflict, and a read-only instance does not need them.

The gateway.networking.k8s.io grants are needed only in a cluster that serves the Gateway API; its CRDs are an add-on. The controller probes for the group at startup and, where it is absent, watches none of its kinds and serves Ingress objects alone. Granting the verbs for resources the cluster does not define is harmless, so one Role covers both cases; a cluster that will never install the CRDs can leave them out.

secrets access is narrowed server-side to type=kubernetes.io/tls, so the controller never holds an application’s credentials in memory. The grant itself cannot express that narrowing, which is the strongest reason to scope the controller to named namespaces where that is possible.

namespaces is requested when namespace_selector is configured, and in any cluster that serves the Gateway API, because a Gateway listener may admit routes by the labels of their namespace (allowedRoutes.namespaces.from: Selector) and only the Namespace object carries those labels. An Ingress-only controller using watch_namespaces avoids both the cluster-wide namespace read and the cluster-scoped Role binding for it.

configmaps is requested only in a cluster that serves the Gateway API, for the ConfigMap a GatewayClass’s parametersRef may name and the CA bundle a BackendTLSPolicy’s caCertificateRefs may name. It is watched in the same namespaces as Services, so a reference into a namespace outside watch_namespaces is reported as not found.

backendtlspolicies graduated to the Gateway API’s v1 later than the core kinds, so a cluster may serve the group without it. The controller reads the served resources at startup and builds the informer only where the kind is served; a policy on a cluster without it is simply never seen.

trickstercachepolicies is this project’s own custom resource (kubernetes-cache-policy.md), installed from deploy/kube/crds. The controller probes for it at startup exactly as it probes for the Gateway API and watches it only where the cluster serves it, so the grant is harmless on a cluster without the definition.

EndpointSlices are not watched by the controller itself: endpoint churn is the autodiscovery provider’s job and reaches the data plane through pool mutation rather than a configuration reload. In the endpoint routing mode the controller generates a discoverer over its own connection, so the service account then also needs the endpointslices grant above in every namespace whose Services routes reference, exactly as a hand-configured discoverer would; see the RBAC section of alb-autodiscovery.md. In the service routing mode it is not needed.

Cluster writes

Status and Events are the only writes the controller makes; everything above is read. Setting read_only: true gives up all of them, and the grant below along with them, for an instance whose service account cannot or should not write to the cluster. It still watches, translates and serves traffic — read-only describes the relationship with the cluster, not the data plane.

Leader election

Required only when leader_election.enabled is true (the default) and the instance is not read-only: the election decides which replica writes status and Events, so an instance that writes neither contends for nothing. Every replica programs its own data plane regardless, so a replica that loses an election still serves traffic.

Address publishing

Required only when published_service is configured: the controller watches that one Service (list, watch on services, narrowed by a field selector to its name) in the Service’s own namespace, which need not be one of the watched namespaces, and publishes its assigned addresses into Gateway and Ingress status.