This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Security & Policy

Controlling who can reach Trickster and how often: IP access lists, geo ACLs and rate limiting.

1 - IP Access Lists

ip_acls defines named IPv4 and IPv6 access lists. A listener, backend, or path selects one with ip_acl_name. The exhaustive example is examples/conf/example.full.yaml.

ip_acls:
  office:
    match: longest          # longest (default) | ordered
    default: deny           # deny (default) | allow
    allow: [10.0.0.0/8, 2001:db8::/32]
    deny: [10.0.99.0/24]
    allow_file: /etc/trickster/office-allow.lst
    deny_file: /etc/trickster/blocklist.lst
    source: client_ip       # client_ip (default) | peer
    action: reject          # reject (default) | drop
    status: 403             # 400-599; default 403

listeners:
  default:
    ip_acl_name: office

backends:
  api:
    ip_acl_name: partners
    paths:
      - path: /public/
        ip_acl_name: none
      - path: /admin/
        ip_acl_name: office

Configuration and composition

Each entry under ip_acls is one named list. ip_acl_name attaches that name to a listener, a backend, or a path. none is reserved and is not a list name. On a listener or a backend it is an undefined name and the configuration fails to load. On a path it is the clear operation below.

Three attachments can apply to one HTTP request, in two decisions:

  • The listener list is its own decision. It does not inherit, replace, or clear a backend or path list.
  • The route decision is the path list when the path names one. That name replaces the backend list for that path, and the decision is counted with scope path. An empty path ip_acl_name inherits the backend list, and that decision is counted with scope backend. ip_acl_name: none clears the backend list for that path, so the route makes no access-list decision.

When both the listener and the route have a list, the request must be allowed by each. A denial from either one stops the request.

Match modes

match is longest or ordered. Empty means longest. default is the verdict when nothing matches. Empty means deny. allow is the other value.

longest uses the longest matching prefix. An equal-length allow and deny is a deny, and the load reports a warning. allow, deny, allow_file, and deny_file belong to this mode. rules in this mode fails the load.

ordered walks rules from top to bottom and uses the first match. allow, deny, allow_file, and deny_file next to rules fail the load; each rule sets exactly one of those. An empty rules list is valid. With default: deny it is the same warning as any other list that has no entries: every address is denied. A rule that an earlier rule already covers completely is unreachable and is a load warning. The commented legacy list in example.full.yaml is this mode:

ip_acls:
  legacy:
    match: ordered
    default: deny
    rules:
      - deny: 10.1.2.0/24
      - allow: 10.0.0.0/8
      - deny: all

A list with no entries and default: deny is valid and warns, because every address is denied. That warning covers an empty longest list and an empty ordered rule list alike.

Addresses and files

An entry is an address or a CIDR, in IPv4 or IPv6. Host bits in a CIDR are masked. all means 0.0.0.0/0 and ::/0. ::/0 alone is IPv6. An IPv4-mapped IPv6 address is matched as its IPv4 form. At request time an address that cannot be parsed is denied, whatever default is. A zoned IPv6 address matches no prefix, so default applies.

allow_file and deny_file are one entry per line. A blank line, or a line whose first non-space character is #, is skipped. The file is read when the configuration is loaded and on every configuration reload. A missing or unreadable file fails that load. A reload happens only when a configuration file has changed, and an access-list file is not one: after editing only the list, update a configuration file’s modification time (touch it, for example) before sending the reload, or the reload is skipped and the old list stays.

client_ip and peer

source is client_ip or peer. Empty means client_ip. peer is valid only on a listener. A backend or path that names a peer list fails validation.

client_ip is the address already resolved through the listener’s trusted_proxies and, where the listener reads one, its PROXY protocol header. See Trusted Proxies. peer is the socket peer. The two are not interchangeable across protocols.

HTTP and HTTP/3

On HTTP and HTTP/3, client_ip is judged in middleware, after trusted-proxy and PROXY resolution. A peer list on HTTP/1 or HTTP/2 is judged when the TCP connection is accepted and is not judged again in middleware. HTTP/3 has no TCP accept, so a peer list is judged in middleware from the QUIC peer address.

TCP, TLS, and UDP

On a tcp or tls listener, peer is judged at accept, before the PROXY header and before TLS. client_ip is judged later, during stream admission, from the client address on the flow. That address is the socket peer, or the address from a trusted PROXY header when the listener has read one.

UDP has no TCP accept. A UDP listener list, whichever its source, and the UDP backend list are judged during stream admission when the flow opens. Later datagrams of a flow that already opened are not judged again. A denied UDP flow stays denied for the datagrams that follow.

A backend peer list is not a valid attachment, so stream admission does not enforce one.

Stream admission enforces the list on the backend selected by the listener table, including a front ALB. Lists on stream ALB pool members, nested ALBs, or discovery templates are not supported and fail configuration validation. This includes named path lists on those members or templates, unless HTTP also serves the member: a path list then judges its HTTP requests. Attach the policy to the listener or front ALB instead. Geo ACLs follow the same rule.

Native protocols

A native listener (mysql, postgres, clickhouse, flight-sql) judges its list at accept. client_ip on that listener is the socket peer, and only when proxy_protocol is off. client_ip together with proxy_protocol is refused, because the socket peer and the address in the header are different addresses. client_ip with proxy_protocol and no trusted_proxies still loads on a listener that reads a PROXY header, including udp, and warns: an empty trusted-proxy list believes every peer’s header.

Native sessions are judged by the listener list only, so a backend that has an access list and is mapped to a native listener loads with a warning. The configuration remains valid. A ClickHouse native listener also sends each query through the backend’s routes with the session’s address, so the backend or path list judges every query: reject fails the query with exception 62 and keeps the session open, and drop closes the session. A ClickHouse backend served only over HTTP does not get the warning.

Actions

action is reject or drop. Empty means reject. status is the HTTP status used by reject, from 400 to 599. Empty means 403. Both actions count as a deny in the decision metric below.

HTTP

reject writes that status, an empty body, and Cache-Control: no-store. The response then follows the normal access-log path.

drop aborts the request with http.ErrAbortHandler. No response bytes are written, and the access log does not write its completion line. See access-logs.md.

drop is refused when the list is attached to a backend or path that an HTTP listener serves, including a backend mapped to both HTTP and a stream listener. HTTP reachability includes dispatch through ALB pools, discovery templates, rules, user routers, and mirrors. A backend mapped only to tcp, tls, or udp may use drop.

Streams

On TCP and TLS, reject resets the connection and drop closes it. Neither writes an HTTP response. On UDP, both actions are the admission verdict for that flow: the flow is not opened, and later datagrams from that client are not admitted again. An HTTP drop does not describe this path.

Readiness

The listener list wraps every handler on that listener. On a proxy listener that includes the readiness handler. On the management listener it includes that listener’s routes. Readiness is not exempt from the list. A client the list denies does not receive a ready response from a listener that list protects.

Reload

Access-list files are read again only when the configuration is loaded or reloaded, and a reload is skipped while no configuration file has changed; see Addresses and files. An access-list change does not by itself rebind the listener: the running listener swaps in the new list. Connections accepted after the reload, and stream flows admitted after the reload, are judged by the new list.

A request already in progress is not judged a second time. An established TCP or TLS flow keeps the upstream it was relayed to and is not judged again. An established UDP flow is not admitted again. A flow that was denied stays denied; a flow that was allowed is not closed because the list changed.

Metrics

trickster_ip_acl_decisions_total counts each decision an attached list actually makes. The labels are:

LabelValues
ip_aclthe list name
scopelistener, backend, or path
verdictallow or deny

reject and drop are both deny. The scope is the attachment that enforced the list: listener for the listener, path when the path named its own list, and backend when the path inherited the backend list. Each enforcement point increments the counter once. A TCP or TLS peer decision counted at accept is not counted again in HTTP middleware or in stream admission. See metrics.md.

Debug logging

A denial writes one application log line at debug: ip acl denied, with ip_acl, scope, address, and action. Allows are not logged. The default log level does not show the line. This is separate from the access log and from the decision metric.

Kubernetes

Generated backends can take a list from kubernetes.defaults.ip_acl_name, or from the GatewayClass parameter ip_acl_name when that parameter is set. Only a list with source: client_ip and action: reject is eligible. There is no Ingress annotation and no cache-policy field for the name. See kubernetes-ingress.md and kubernetes-gateway.md.

Compile time

Compiling a file of 100,000 IPv4 allow entries was measured with go test -bench=BenchmarkCompileFile100k -benchmem -count=1 ./pkg/proxy/ipacl/ on darwin/arm64 (Apple M1):

BenchmarkCompileFile100k-8   	      24	  55277351 ns/op	46640989 B/op	  600334 allocs/op

That is the measured cost of compiling that file, not of matching one address against an already compiled list.

2 - Geo ACLs

Geo ACLs restrict backends and paths by where their clients are. Other products call this geo-restriction, geo-blocking, geo-filtering or geofencing. A geo ACL allows or denies countries, first-level subdivisions (states, provinces, regions) and continents, and refuses a client in the form its protocol expects: an HTTP status and body, a native database error, or a reset stream.

Two top-level sections configure it:

  • geo_locators are named lookups that place a client address: a MaxMind DB format database, a list of RFC 8805 geofeed entries, or a location header a CDN in front of Trickster sets.
  • geo_acls are named lists of the locations that may, or may not, pass. Each names the locator it uses, so many geo ACLs share one database.

Backends and paths name a geo ACL with geo_acl_name.

Trickster ships no location database and downloads none. You provide the file and keep it current; see Where location data comes from. To add a source of location data, see the locator provider guide.

A geo ACL is a compliance and licensing control, not a security boundary. A VPN, a proxy or Tor places a client wherever it chooses.

Example

geo_locators:
  default:                             # a geo ACL uses the locator named default unless it names another
    provider: mmdb
    mmdb:
      path: /var/lib/GeoIP/GeoLite2-Country.mmdb

  city:
    provider: mmdb
    mmdb:
      path: /var/lib/GeoIP/GeoLite2-City.mmdb

  corrections:
    provider: geofeed
    geofeed:
      entries:
        - 203.0.113.0/24,DE,DE-BE
      files: [ /etc/trickster/private-relay.csv ]

geo_acls:
  north-america:
    allow: [ US, CA, MX ]
    exempt: [ private ]

  no-restricted-states:
    geo_locator_name: city             # subdivisions need a City-class database
    deny: [ US-TX, US-UT ]

  europe:
    allow: [ continent:EU ]

backends:
  video:
    provider: reverseproxycache
    origin_url: http://video.example.com
    geo_acl_name: north-america
    paths:
      - path: /trailers/
        geo_acl_name: none             # clears the backend's geo ACL for this path
      - path: /eu/
        geo_acl_name: europe           # replaces it for this path
  orders:
    provider: mysql
    geo_acl_name: no-restricted-states # a refused session gets MySQL error 1130

example.full.yaml describes every option.

Locators

Every locator has a provider and one options block named for it. A block for another provider, or an unknown provider, fails validation.

mmdb

Reads a MaxMind DB format file. That covers MaxMind GeoIP2 and GeoLite2, DB-IP, IPinfo, and IP2Location’s MMDB editions.

OptionDefaultMeaning
pathrequiredthe database file
schemaautowhere records keep the location: auto, geoip2, ipinfo or custom
fieldsthe record paths of country (required), continent and subdivision, for schema: custom
reload_interval5mhow often the file is checked for a replacement, besides change events; 10s to 24h
max_age0 (never)log a warning when the loaded file was built longer ago than this
  • Schemas. geoip2 reads country.iso_code, continent.code and subdivisions[0].iso_code; MaxMind, DB-IP and IP2Location’s MMDB editions use it. ipinfo reads country_code and continent_code, falling back to the country and continent codes of IPinfo’s older files. auto chooses by the file’s database type and confirms against the file’s own records. A file that auto cannot read fails to load, and the error names schema: custom.
  • Custom paths are lists of map keys and array indexes, such as [subdivisions, 0, iso_code].
  • Fields a file serves. A Country file serves countries and continents. A file serves subdivisions only when one of its records carries one, whatever its database type says; a load searches the whole file before deciding it serves none. A geo ACL with subdivision entries refuses a locator that serves none, when the configuration is applied.
  • In memory. The file is read into memory, not mapped, so an updater that rewrites it in place can never fault the process. It costs the file’s size, and twice that while a replacement loads. GeoLite2 Country is about 8 MB and GeoLite2 City about 65 MB; use a Country file in tight pods.
  • An IPv4-only file logs a warning, since it places no IPv6 client.

geofeed

Places addresses by RFC 8805 lines: prefix,country,region,city,postal code. City and postal code are ignored.

OptionDefaultMeaning
entriesinline lines
filesgeofeed files, watched for changes
reload_interval5mas for mmdb
  • At least one of entries and files is required. An inline entry that does not parse fails validation; a line in a file that does not parse is skipped, and a warning counts the skipped lines and shows the first.
  • The longest prefix holding an address wins. An inline entry beats a file’s for the same prefix, which makes inline entries a place for corrections.
  • A line with every location field empty places no one: an address under it has no location, even when a shorter prefix would place it.
  • region is a full ISO 3166-2 code, such as US-AL, whose country must agree with the line’s. A country’s continent is derived from the country.
  • Feeds such as Apple’s iCloud Private Relay egress ranges are published in this format.

Reads a location header that a trusted upstream set, such as Cloudflare’s CF-IPCountry or CloudFront’s CloudFront-Viewer-Country and CloudFront-Viewer-Country-Region.

OptionDefaultMeaning
countryrequiredthe header holding a country code
subdivisiona header holding a subdivision code, with or without its country
continenta header holding a continent code; otherwise the country’s continent
unknown_values[XX, T1, ZZ, '-']values that mean no location
  • Headers are believed only from the listener’s trusted_proxies. From any other peer they are not read, and the request has no location. A listener with no trusted_proxies believes no header, and a load warning says so.
  • It judges HTTP requests only. A geo ACL whose locator is header cannot gate a backend that a native protocol or stream listener serves; validation refuses it.

Replacing files

A replaced mmdb or geofeed file is loaded while serving, with no configuration reload. Write the new file and rename it over the old one: a file written in place can be read half-written. A replacement that fails to load leaves the last good file serving, counts a failed reload and logs one warning. Replacements are refused when:

  • an mmdb file does not open, or serves fewer location fields than the file it replaces;
  • a geofeed file changed while it was read, or lost every entry where the old file had some.

A locator whose options are unchanged is kept across configuration reloads, with its loaded file.

Geo ACLs

OptionDefaultMeaning
geo_locator_namedefaultthe locator that places clients
allowthe only locations allowed
denythe locations denied
unknownas an unlisted locationallow or deny: the verdict for a client with no location
exemptaddresses and prefixes allowed with no lookup; private is every non-routable range
actionrejectreject, or count, which counts a denial and allows the client
messagesee belowwhat a refused client is told, in every protocol
responsethe HTTP refusal’s status, headers and body

Entries

A geo ACL has an allow list or a deny list, never both. An entry is one of:

SpellingMatchesExample
two lettersa country (ISO 3166-1 alpha-2)US
country, hyphen, 1 to 3 letters or digitsa first-level subdivision (ISO 3166-2)US-TX, FR-75
continent: and two lettersa continent: AF AN AS EU NA OC SAcontinent:EU
  • Entries ignore case.
  • A list is a union: allow: [US, CA-QC] is all of the United States and Quebec only.
  • Country codes are checked against the assigned ISO 3166-1 codes, plus XK for Kosovo, which the databases use. An unassigned code fails with a suggestion: UK is not assigned; use GB.
  • Continents take the continent: prefix because AF, AS, NA and SA are also countries.

How a client is judged

  1. An exempt address is allowed, with no lookup.
  2. The locator places the address.
  3. A client with no location (the locator had no answer, the lookup failed, or the address did not parse) takes the unknown verdict. Unset, that is the verdict of an unlisted location: denied by an allow list, allowed by a deny list.
  4. Otherwise an allow list allows only what it lists, and a deny list denies only what it lists.
  5. A denial under action: count is counted and allowed. On HTTP the request goes on carrying an X-Trickster-Geo-Denied header naming the geo ACL, for the origin’s logs.

private stands for loopback, RFC 1918, link-local, carrier-grade NAT (100.64.0.0/10) and IPv6 unique local addresses. No database places them, so an allow list refuses them unless they are exempt; a load warning says so for an allow list with neither exempt nor unknown.

Where geo ACLs apply

  • Backends and paths. A path’s geo_acl_name replaces its backend’s, and geo_acl_name: none on a path clears it. none is valid on paths only. Provider paths a configuration does not name take the backend’s geo ACL.
  • HTTP. The geo ACL runs in the route chain just outside the authenticator: a refused request never reaches a credential check, a mirror, the cache or the origin, and is logged and counted like any response. A backend’s geo ACL is on its own route too, which ALB pools and rules dispatch into, so a pool member’s geo ACL judges what an ALB sends it. Put the geo ACL on the backend a client reaches first: a fanout ALB treats a member’s refusal as that member’s response.
  • Native protocols. A mysql, postgres, clickhouse or flight-sql listener judges each session by the geo ACL of the backend it maps to, before any credential is checked. flight-sql judges every call. clickhouse also judges every query, since it sends each one through its backend’s HTTP route for /: the backend’s geo ACL, or that of a / path that names its own, can refuse a query on an open session after a reload or a replaced file. See Refusals for how that differs from a refused session. Otherwise a session admitted before a reload is not judged again. A backend that a native listener’s ALB routes sessions to may not name a geo ACL, since the session is judged by the ALB’s before it is routed; a path-level geo ACL on such a member judges its own HTTP requests.
  • Stream listeners. A tcp or udp listener judges each connection or flow by its backend’s geo ACL; a tls listener judges by the backend the server name selects. A refusal resets a tcp or tls connection and drops a udp flow’s datagrams; no message can be sent on an opaque stream. An ALB on a stream listener dials its members without entering their routes, so a backend or discovery template in its pool may not name a geo ACL: set it on the ALB, which judges the connection before a member is dialed. A path-level geo ACL on such a member is refused too, unless HTTP also serves the member, whose requests it then judges. IP access lists follow the same rule.
  • Not gated: the readiness path and the mgmt and metrics listeners.
  • Other gates on the same route judge independently, and any refusal ends the request.

The client address

  • On HTTP it is the address the listener’s trusted_proxies resolved from the forwarding headers, or else the peer’s. A mirrored copy carries it, as does each query of a ClickHouse native session.
  • On native and stream listeners it is the peer, or the source a trusted PROXY protocol header named.
  • A wrong trusted_proxies makes every client the proxy. Behind a private address that is no location for everyone: an allow list refuses every client, which is noticed at once, and a deny list allows every client, which is not. Watch the rate of trickster_geo_locator_lookups_total{result="not_found"}; a listener with proxy_protocol and no trusted_proxies logs a load warning, since any client can name its own source.

Refusals

Every protocol carries the geo ACL’s message, which defaults to This resource is not available in your geographical area. It is one line of at most 255 bytes.

ClientShown
HTTP403 Forbidden and the message, as text/plain; charset=utf-8
mysqlERROR 1130 (HY000): <message>
psqlFATAL: <message>, SQLSTATE 28000
clickhouse-clientCode: 195. DB::Exception: <message> (IP_ADDRESS_NOT_ALLOWED)
Flight SQLgRPC status PermissionDenied with the message
tcp, tlsa reset
udpnothing; the datagrams are dropped

A clickhouse query refused on a session that was already admitted fails like any other query: Code: 62. DB::Exception: <message>, where the message is the HTTP refusal’s body, so a response.body replaces it. The session stays open, and the refusal is counted with plane="http". Only a new session gets exception 195.

HTTP refusals:

  • response.status is from 400 to 599. No status code is defined for a geographic refusal, so 403 is the default, as CDNs answer.
  • 451 is for a legal demand (RFC 7725), not a licensing or business restriction. A 451 should explain the demand in its body and carry a Link: <uri>; rel="blocked-by" header; a load warning names a 451 without one.
  • response.headers are merged over the default Cache-Control: no-store and Content-Type.
  • response.body replaces the message and its newline, for a JSON or HTML refusal.

Caches in front of Trickster

Trickster marks its own refusals no-store. A shared cache in front of Trickster may still store an allowed client’s response and serve it to anyone. Serve a restricted path behind such a cache with Cache-Control: private, or apply the same restriction at that cache.

Where location data comes from

SourceFormatTermsUpdates
MaxMind GeoIP2 (paid), GeoLite2 (free)MMDBan account and license key to download; attribution; a GeoLite file is replaced within 30 days of a releaseGeoLite twice a week
DB-IP LiteMMDBCC BY 4.0, with a link backmonthly
IPinfo LiteMMDBCC BY-SA 4.0, with attribution; a token to downloadfrequent
IP2Location LITEMMDB editions for some packagesfree with attribution; sign-upmonthly
CDN headersa request headerthe CDN’slive
GeofeedsRFC 8805 CSVthe publisher’sthe publisher’s
  • Licenses bind whoever distributes the file. Trickster ships none; a container image that bakes a GeoLite file in redistributes it.
  • On Kubernetes, MaxMind’s geoipupdate image can run as an init container and a sidecar writing to a volume that Trickster reads; see Deploying on Kubernetes.
  • Accuracy falls with precision. MaxMind cites country accuracy above 99% with VPNs left out, state or province accuracy of 55% to 80%, and city accuracy of 20% to 75%. A subdivision list refuses some clients wrongly and admits others; exempt and a geofeed of corrections are the remedies.
  • An address can move between networks daily. max_age and the build-time gauge show a stale file.
  • Whether a restriction is lawful where it applies is the operator’s question.

Metrics

MetricLabels
trickster_geo_acl_decisions_totalgeo_acl, plane (http, native, stream), verdict (allow, deny, count, exempt)
trickster_geo_locator_lookups_totalgeo_locator, result (found, not_found, error)
trickster_geo_locator_reloads_totalgeo_locator, result (success, error)
trickster_geo_locator_build_timestamp_secondsgeo_locator; mmdb only

No metric carries a country. Refusals log at debug level only, so a flood of refused clients is not a flood of log lines; a MySQL listener’s failed logins log at debug too, and are counted by trickster_mysql_connection_errors_total.

Performance

Measured on an Apple M3 Max, none allocating:

WhatTime
a geofeed lookup, 10 or 300,000 entries11 to 15 ns
a geo ACL check over a geofeedabout 25 ns
the HTTP gate, an allowed requestabout 50 ns over the route without it
an mmdb lookup, test databaseabout 105 ns

An mmdb file is decoded once per record, then answered from a bounded cache of decoded records. A route with no geo ACL is unchanged, and a native listener without one pays one atomic load per session.

Kubernetes

A geo ACL is an operator control, like an authenticator, so the controller takes it only from operator-tier settings:

  • kubernetes.defaults.geo_acl_name gates every route the controller generates. Since stream routes take it too, it must name a geo ACL whose locator places addresses.
  • A GatewayClass’s parameters can name geo_acl_name for the class’s routes. A geo ACL whose locator reads headers serves the class’s HTTP routes, and refuses its stream routes rather than serving them ungated.
  • No Ingress annotation and no TricksterCachePolicy field names one, since whoever writes those in their own namespace could name a looser geo ACL, or none.
  • The geo ACL goes on the backend a route attaches, never on a pool member, an endpoint template or a mirror target, so each request is judged once.

See Kubernetes Ingress and Kubernetes Gateway API.

3 - Rate Limiting

Trickster counts requests, connections, sessions, or datagrams against named limiters and refuses what exceeds a limit. Limiters are defined once under rate_limiters and attached by rate_limiter_name. The exhaustive example is examples/conf/example.full.yaml.

rate_limiters:
  per-client:
    keys: [client_ip]          # empty means one bucket for every event
    ipv6_prefix: 64            # leading bits of an IPv6 client address; default 64
    limit: 600                 # events per window
    window: 1m                 # 1s to 1h; default 1m
    algorithm: sliding_window  # the only algorithm
    missing_key: exempt        # exempt (default) | shared
    action: reject             # reject (default) | count | close
    policy_headers: none       # none (default) | ietf | legacy
    max_keys: 100000           # buckets tracked; default 100000
    max_keys_action: allow     # allow (default) | reject
    # unit: requests           # leave unset when the limiter is used on more than one plane
    response:
      status: 429              # 400-599; default 429
      body: "Too Many Requests\n"
      headers:
        Cache-Control: no-store

listeners:
  default:
    rate_limiter_name: per-client

backends:
  api:
    rate_limiter_name: per-client
    paths:
      - path: /healthz
        rate_limiter_name: none   # clears the backend limiter; the listener limiter remains

Where a limiter applies

rate_limiter_name attaches a limiter to a listener, a backend, or a path. none is reserved and is not a limiter name. On a listener or a backend it is an undefined name and the configuration fails to load. On a path it clears the backend limiter for that path.

Two decisions can apply to one HTTP request:

  • The listener limiter is its own decision. A path name does not replace it, and none does not clear it.
  • The route decision is the path limiter when the path names one. That name replaces the backend limiter for that path. An empty path rate_limiter_name inherits the backend limiter. none clears it, so the route makes no rate-limit decision.

When both are set, each judges the request. Naming the same limiter on the listener and on the route spends two events from that limiter’s buckets. A listener allowance can be spent by a request that a later route access list denies. A route access-list denial happens before the route limiter, so it does not spend the route allowance.

The same limiter name is one set of buckets everywhere it is attached, for as long as its keys, IPv6 prefix, window, limit, and max_keys stay the same. missing_key, max_keys_action, action, the HTTP response, and policy_headers can change on reload while those buckets stay.

unit says what an event is: requests, connections, sessions, or datagrams. Leave it unset when the limiter is attached on more than one plane. An unset unit uses requests on HTTP, connections on tcp and tls, and sessions on udp, and those events share one quota. A limiter attached on more than one plane with unit set fails to load. close is valid only on stream listeners. A native listener (mysql, postgres, clickhouse, flight-sql) cannot name a limiter. A moq listener can: it counts sessions, by client_ip or in one bucket, and a moq backend’s limiter counts its sessions’ MoQT requests; see Media over QUIC.

A cache hit is still an event: the limiter judges the request before the cache.

The estimate

algorithm is sliding_window. Empty means that. Each bucket keeps the count for the previous window and the count for the current one. At a moment frac of the way through the window, the estimate is the previous count times (1 - frac), plus the current count. An event is allowed when the estimate plus one is at most limit. The event that would pass the limit is not added, except in count mode, which records it and still lets it through.

This is an estimate, not a log of every event. A burst at the end of one window and the start of the next can weigh as less than the number of events, so a client can be allowed slightly past what an exact sliding log would allow.

A bucket that is not read for a little more than two windows is dropped. The table may drop it up to a second early, or up to a sixteenth of that idle time when that is shorter.

Keys

keys is up to five attributes, in order. Together they are one bucket. Empty keys is one bucket for every event.

KeyRead on
client_ipHTTP, HTTP/3, tcp, tls, and udp. The client address after trusted_proxies, never the port. IPv6 keeps the leading ipv6_prefix bits (default 64), so a client that rotates its privacy address stays on one key.
hostHTTP. The request host, without its port, without regard to case.
header:<name>HTTP. The first value of that request header.
cookie:<name>HTTP. The value of that request cookie.
query:<name>HTTP. The raw value of that query parameter.
queryHTTP. The query string as the client sent it.
methodHTTP. The request method.
pathHTTP. The request path, without the query string, after path normalization.
snitls only. The server name the client offered. It is known after the handshake, so the connection is counted then.
proxy_tlv:<type>tcp and tls listeners that accept a PROXY protocol header. <type> is the TLV type, as a decimal or hex integer.
userThe authenticated name on a native session. No listener that can carry a limiter reads it, so a limiter keyed on user cannot be attached.

A key the attachment cannot read fails to load. A listener limiter can use only keys that need nothing but the connection or the request itself.

missing_key is exempt or shared. Empty means exempt. exempt lets an event through without counting it when a key has no value, such as a header the request did not send. shared puts every such event in one extra bucket.

client_ip is the address Trickster already resolved. Clients behind one NAT share a key. A listener behind a proxy that is missing from trusted_proxies sees that proxy as the client, so every request shares the proxy’s key. On a stream listener that reads a PROXY header and lists no trusted_proxies, a client_ip limiter still loads and warns: an empty trusted-proxy list believes every peer’s header.

Two different identities that hash to the same 64-bit key share a bucket, and therefore one quota. A collision spends the allowance faster. It does not open a second one.

Capacity

max_keys is how many buckets the limiter tracks. Empty means 100000. The table is split into as many as 64 shards, and each shard rounds its share up, so the table can hold up to 63 buckets past max_keys. A new key can also be refused while the table is still under max_keys, once the shard it lands in is full.

max_keys_action is what a new key gets when it does not fit. Empty means allow: the event proceeds and is not counted. reject treats it as over the limit. Both record the full result.

Each tracked key is a 40-byte counter plus the table entry that holds it. On a 64-bit process that comes to about 110 bytes of heap per key, including the map slot.

HTTP

On an HTTP or HTTP/3 listener the limiter sits after the listener IP access list and path normalization, and before the router. The route limiter sits after the route’s IP and geo access lists and before the authenticator. The readiness path, after normalization, is exempt on every listener except the metrics listener. A health check is not exempt. An ACME http-01 challenge is answered before any limiter.

action is reject or count on HTTP. Empty means reject. close fails to load when the limiter is attached to anything HTTP serves.

reject writes the configured response. The status is 400–599 and defaults to 429. The body defaults to Too Many Requests\n and is omitted for HEAD. Cache-Control: no-store is always set. Retry-After is the wait until a new event could pass, in whole seconds, rounded up, and at most about two windows. A full table that rejects uses the window length. A response header cannot replace Retry-After, the quota headers below, Content-Length, Transfer-Encoding, or a hop-by-hop field.

policy_headers adds quota fields to responses the limiter allowed as well as to refusals:

  • none adds none.
  • ietf adds RateLimit-Policy ("<name>";q=<limit>;w=<window seconds>) and RateLimit ("<name>";r=<remaining>;t=<seconds until the window ages out>). Each limiter adds its own pair.
  • legacy sets X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset when those fields are still absent, so the route limiter’s values win over the listener’s.

count forwards the event after recording it, including one that is over the limit and a new key a rejecting full table would have stopped. The limiter appends its name to the request header X-Trickster-Rate-Limited. A listener that has a limiter, or that serves a backend or path which names one, removes any value the client sent before judging.

A refusal, a counted event, and a full table write one debug log line, rate limit decision, with the limiter name and the result. Allows are not logged.

Streams

A tcp or tls listener counts a connection unless unit says otherwise. A udp listener counts a session unless unit is datagrams.

  • connections are counted when the connection is accepted, from client_ip and proxy_tlv. A key of sni, or an IP or geo access list that still judges after the flow is known, waits until then, so that list can refuse the connection before it is counted. sni is always counted after the handshake.
  • sessions are counted once, when the UDP flow opens. A refused session is held until a new event could pass, and that wait does not count again. A full table that rejects holds for the window.
  • datagrams are counted on every datagram, including the first.

reject resets a TCP or TLS connection and drops a UDP datagram. close finishes a TCP or TLS connection and drops a UDP datagram. count allows the event after the same single charge. A connection that ends before the stage that counts it is not counted. That includes a TLS name that routes nowhere.

A ClickHouse query that arrives over HTTP uses the HTTP route limiter. That is separate from a limiter on a native session, which a native listener does not enforce.

Reload and restart

A reload that keeps the limiter’s name, keys, IPv6 prefix, window, limit, and max_keys keeps its buckets. Changing any of those starts that limiter empty. Removing the limiter drops its buckets after the new configuration is in place. A reload that fails to apply leaves the previous buckets alone. Restarting the process starts every limiter empty. The clock is monotonic for the life of the process, so a step of the wall clock does not roll a window backward or forward.

Metrics

trickster_ratelimit_decisions_total counts each decision. The labels are limiter, plane (http, stream or moq), and result:

resultMeaning
allowedcounted, and at or under the limit
limitedover the limit, refused, and not counted
countedover the limit in count mode, recorded, and still forwarded
exemptnot counted: a missing key under exempt, or the readiness path
fulla new key that did not fit. allow forwards it without counting; reject refuses it

trickster_ratelimit_keys is the number of buckets that limiter holds at scrape time, labeled by limiter. See metrics.md.

Kubernetes

kubernetes.defaults.rate_limiter_name puts one limiter on every HTTP and gRPC route the controller generates. A stream route does not take it. A GatewayClass parameter of the same name overrides it for that class. The limiter must use HTTP keys, a unit of requests or unset, and an action other than close. There is no Ingress annotation and no TricksterCachePolicy field, and a generated route cannot set the name to none. The name is not copied onto pool members, endpoint templates, mirror targets, or generated listeners. See kubernetes-ingress.md and kubernetes-gateway.md.