This is the multi-page printable view of this section. Click here to print.
Listeners & TLS
1 - TLS Support
Trickster supports TLS on both the frontend server and backend clients.
Basics
To enable the TLS server, specify the tls_port, and optionally, the tls_address in one of the listeners of your config file. For example:
listeners:
# default is built-in and is the default listener for backends
# it uses these values by default:
default:
port: 8480
tls_port: 8483
# listen on all interfaces
tls_address: ''
Note, Trickster will only start listening on a TLS port if at least one origin mapped to the named listener has a valid certificate and key configured, obtains its certificates through ACME, or the listener sets tls_runtime_certs.
Instead of certificate files, a backend can have Trickster obtain and renew its certificates from an ACME certificate authority such as Let’s Encrypt. See Automatic Certificates (ACME).
Each origin section of a Trickster config file can be augmented with the optional tls section to modify TLS behavior for front-end and back-end requests. For example:
backends:
example: # example backend
tls: # TLS settings for example backend
# frontend configs
full_chain_cert_path: '/path/to/my/cert.pem'
private_key_path: '/path/to/my/key.pem'
# backend configs
insecure_skip_verify: true
certificate_authority_paths: [ '/path/to/ca1.pem', '/path/to/ca2.pem' ]
certificate_authority_pem: |
-----BEGIN CERTIFICATE-----
...an inline CA bundle, for material that arrives as configuration...
-----END CERTIFICATE-----
server_name: origin.internal.example.com
exclude_system_roots: true
client_cert_path: '/path/to/client/cert.pem'
client_key_path: '/path/to/client/key.pem'
Server Configs - used when responding to clients
Each backend contributes up to 1 certificate and key pair, as configured in the TLS section of the backend config (demonstrated above). A listener serves the certificates of all backends mapped to it, selecting per-handshake by SNI (see Certificate Selection (SNI) below), so a single TLS listener can serve many hostnames.
If the path to any configured Certificate or Key file is unreachable or unparsable, Trickster will exit upon startup with an error providing reasonable context.
You may use the same TLS certificate and key for multiple backends, depending upon how your Trickster configurations are laid out. Any certificates configured by Trickster must match the hostname header of the inbound http request (exactly, or by wildcard interpolation), or clients will likely reject the certificate for security issues.
Certificate Selection (SNI)
When a listener has multiple certificates, Trickster selects the certificate for each TLS handshake in this order:
- Exact match of the client’s SNI hostname against a certificate’s Subject Alternative Names
- Wildcard match (e.g. a
*.example.comcertificate forfoo.example.com) - A linear compatibility scan (covers clients that send no SNI, and IP SANs)
- The listener’s first certificate, if nothing else matches
The exact and wildcard lookups are index-based, so per-handshake selection cost is independent of the number of certificates a listener serves.
Automatic Certificate Rotation Detection
Trickster automatically detects when a serving certificate is renewed in place on disk — same file paths, config untouched, as performed by tools like certbot or cert-manager — and hot-swaps the renewed certificate into the live listener. This is entirely independent of configuration reloads: no manual reload, auto_reload_interval, or config change is required.
- Detection is hybrid: filesystem events (via fsnotify) trigger a near-immediate check where the platform and filesystem support them, and a timer-based poll runs regardless — both as the fallback for deployments where filesystem events don’t work (e.g. some network and FUSE filesystems) and as a self-healing backstop for missed events. If event watching is unavailable, detection silently degrades to poll-only.
- Every check compares file contents (not modification times or event payloads), so detection works across platforms and through the atomic symlink swaps used by Kubernetes Secret and projected volumes.
- The certificate, key (and any associated CA bundle) files are watched and validated as one unit: if a poll observes a mid-rotation partial state (e.g. the cert file updated before the key file), the mismatched pair is never served; the last-good pair keeps serving and the change is retried on the next poll.
- Read errors and invalid content never disable detection: the watcher keeps retrying, logs a warning after sustained failures, and the last-good certificate keeps serving. Deleting the files is treated as a persistent failure, not as certificate removal.
- Startup behavior is unchanged: an unreadable or invalid configured certificate is still a fatal startup error. Only post-startup source failures are tolerated.
Rotation detection is on by default and is configured per listener. tls_watch_interval sets the fallback/backstop poll cadence (default 30s); filesystem events, where available, apply rotations within moments regardless of the interval. Setting the interval to 0 disables rotation detection entirely (events included):
listeners:
default:
tls_port: 8483
# backstop poll interval for the cert/key files of mapped backends
# (default 30s); filesystem events accelerate detection where supported.
# set to 0 to disable automatic rotation detection.
tls_watch_interval: 30s
Note: automatic rotation detection currently applies to HTTP(S) listeners. Native-protocol listeners (e.g. mysql) pick up rotated certificates on config reload.
Hot Swap and No-Close Semantics
Certificate swaps — whether from a config reload or automatic rotation detection — never restart, drain, or rebind the listener:
- The certificate is consulted only at handshake time, so established connections (including keep-alive connections and in-flight requests) are untouched by a swap; they continue on the certificate they were handshaken with until they close naturally.
- Only new handshakes see the new certificate.
Runtime Certificates
A listener normally serves TLS only when a mapped backend provides a certificate and key file. Setting tls_runtime_certs: true on a listener serves its tls_port even when no backend provides a file pair, so certificates can be supplied to the running process instead, for example by the Kubernetes Gateway/Ingress controller from TLS Secrets:
listeners:
default:
port: 8480
tls_port: 8483
tls_runtime_certs: true
Such a listener starts with an empty certificate store and fails handshakes until the first certificate is supplied. Certificates supplied at runtime are validated with the same pair-coherence checks as files, participate in SNI selection alongside any file-sourced certificates on the same listener, appear in the certificate inventory with source memory, and survive configuration reloads and file rotations: a reload replaces only the config-sourced certificates, and a rotation replaces only the file-sourced ones. tls_runtime_certs is valid only on http and moq listeners; on a moq listener it serves QUIC handshakes with the supplied certificates in place of a backend’s file pair.
Certificate Inventory (mgmt)
The mgmt listener exposes a read-only, per-listener certificate inventory at /trickster/certificates (configurable via mgmt.certificates_handler_path). Each entry reports the certificate’s id, source kind (file, memory, acme or config), common name, subject alternative names, validity window and last-load time. The inventory never includes key material.
Observability
Certificate rotation and inventory are covered by the following metrics: trickster_tls_certificate_expiration_time_seconds, trickster_tls_certificate_last_load_time_seconds, trickster_tls_certificate_swaps_total, trickster_tls_certificate_validation_failures_total, trickster_tls_watcher_errors_total, and trickster_tls_certificate_store_size. See metrics.md and the example alerting rules in examples/alerting for cert-expiry, sustained-validation-failure and watcher-staleness alerts.
Client Configs - used when proxying to an origin
Each backend’s TLS configuration can also configure the https client used for making requests against the origin as demonstrated above.
insecure_skip_verify will instruct the http client to ignore hostname verification issues with the upstream origin’s certificate, and process the request anyway. This is analogous to -k | --insecure in curl.
certificate_authority_paths will provide the http client with a list of certificate authorities (used in addition to any OS-provided root CA’s) to use when determining the trust of an upstream origin’s TLS certificate. In all cases, the Root CA’s installed to the operating system on which Trickster is running are used for trust by the client.
certificate_authority_pem adds certificate authorities the same way, from a PEM bundle written inline in the configuration rather than read from a file, for material that arrives as configuration (a Kubernetes BackendTLSPolicy’s CA bundle, for instance). A bundle holding no parsable certificate fails validation.
exclude_system_roots: true makes the configured certificate authorities (certificate_authority_paths and certificate_authority_pem) the only ones trusted for the origin, rather than additions to the operating system’s. Use it when the origin must present a certificate from one specific private authority and a certificate from any public authority would be an error. It requires at least one authority to be configured, since excluding the system roots with none would trust nothing.
server_name sets the hostname sent as SNI and verified against the origin’s certificate, when it differs from the host in origin_url — an origin reached by IP or by an internal alias whose certificate names something else. It changes nothing about where the connection goes or which Host header is sent.
To us Mutual Authentication with an upstream origin server, configure Trickster with Client Certificates using client_cert_path and client_key_path parameters, as shown above. You will likely need to also configure a custom CA in certificate_authority_paths to represent your certificate signer, unless it has been added to the underlying Operating System’s CA list.
2 - HTTP/3
Trickster can serve any HTTP listener’s routes over HTTP/3 (RFC 9114) in addition to HTTP/1.1 and HTTP/2, using QUIC over UDP.
HTTP/3 helps most where Trickster acts as an edge cache: loss recovery without head-of-line blocking benefits parallel byte-range fetches, and connection migration keeps a client’s session alive as it changes networks.
Enabling
HTTP/3 attaches to an existing http listener that already serves TLS. QUIC has
no cleartext mode, so a listener without a working TLS endpoint cannot serve it.
listeners:
default:
port: 8480
tls_port: 8483
http3:
enabled: true
That is the whole configuration for the common case. The UDP endpoint defaults to the same address and port as the TLS endpoint, so clients find HTTP/3 where they already found HTTPS.
Options
| Field | Default | Description |
|---|---|---|
enabled | false | Serves this listener’s routes over HTTP/3. |
address | the listener’s tls_address | IP for the UDP socket. |
port | the listener’s tls_port | UDP port to bind. |
advertised_port | port | Port published in Alt-Svc. Set this when a load balancer or NAT presents a different port to clients than the one bound here. |
listeners:
default:
tls_address: 0.0.0.0
tls_port: 8483
http3:
enabled: true
port: 8443 # bind UDP/8443
advertised_port: 443 # but tell clients to use 443
TLS certificates come from the backends mapped to the listener, exactly as they do for the TLS/TCP endpoint — see TLS. HTTP/3 uses the same certificates; no separate configuration is needed.
How clients discover HTTP/3
Browsers and most clients do not attempt HTTP/3 first. They connect over
TCP and look for an Alt-Svc response header naming an HTTP/3 endpoint:
Alt-Svc: h3=":8443"; ma=2592000
Trickster adds this header to every response from the listener’s TLS/TCP endpoint while HTTP/3 is enabled, so adoption is automatic and gradual. The TCP endpoint keeps serving HTTP/1.1 and HTTP/2 for clients that do not upgrade.
Protocol-upgrade requests (Connection: Upgrade) are not advertised, since such
a request hijacks its connection before response headers matter.
WebTransport
While a moq backend uses the listener, its HTTP/3 endpoint also
serves WebTransport (draft-ietf-webtrans-http3), which browsers use to reach a
MoQ relay. Each WebTransport session arrives as an Extended CONNECT request and
is routed like any other request, so the backend’s paths, hosts,
authenticator_name and ACLs apply to it. A reload that binds or unbinds a
moq backend changes how new connections are served, without restarting the
endpoint or touching the connections already open.
A WebTransport connection cannot carry an HTTP/3 GOAWAY, because WebTransport owns its control stream, so draining such an endpoint closes each connection once it carries no request. A WebTransport session counts as a request for its whole life, and whatever is still open when the drain timeout ends is closed.
Sharing the UDP port
The HTTP/3 endpoint may share its UDP address and port with a protocol: moq
listener (moq.md): each QUIC connection is handed to the listener
whose protocol it negotiated (h3 or a moqt-NN draft), and each listener
keeps its own certificates and routes. A protocol: udp listener cannot share
the port.
Limitations
- No protocol upgrades. HTTP/3 has no equivalent of the HTTP/1.1
101 Switching Protocolshandshake; RFC 9114 4.2 makes connection-specific headers malformed. WebSockets over HTTP/3 use Extended CONNECT (RFC 9220), which Trickster serves only for WebTransport sessions to amoqbackend. Send WebSocket traffic over the TCP endpoint, which supports it fully. - Inbound only. Trickster speaks HTTP/1.1 or HTTP/2 to origins regardless of
the protocol a client used to reach it, except that a
moqbackend reaches a WebTransport origin over HTTP/3. This matches Traefik; among major proxies only Envoy implements HTTP/3 to upstreams, and it needsAlt-Svccaching plus TCP fallback to do so safely. - UDP must reach the listener. Some networks block or throttle UDP/443. Clients that cannot establish QUIC silently keep using TCP, so this degrades rather than fails.
Operating notes
UDP receive buffer
QUIC moves far more data through a single socket than typical UDP workloads, and the kernel default receive buffer is often too small. On Linux, raise it:
sysctl -w net.core.rmem_max=7500000
sysctl -w net.core.wmem_max=7500000
Without this, a warning is logged at startup. In containers that cannot change
sysctls the warning is unactionable and can be silenced with
QUIC_GO_DISABLE_RECEIVE_BUFFER_WARNING=1.
Reloads
The HTTP/3 listener participates in the same reload and drain lifecycle as every other listener. Route changes are hot-swapped without rebinding the socket; changing the bound or advertised port restarts just that endpoint. A drained or removed endpoint releases its UDP port, so a later reload can bind it again; before 2.2, the port stayed bound until the process exited.
Trying it locally
Most systems ship a curl built without HTTP/3 (curl --version | grep HTTP3
to check). Trickster includes a small client so this is not a prerequisite:
make dev-certs # self-signed cert for localhost
go run ./hack/h3-client -url https://127.0.0.1:8483/ -ca docs/developer/environment/certs/trickster-dev.crt
The client prints the negotiated protocol, so HTTP/3.0 200 OK confirms the
request was served over QUIC.
3 - Apache Arrow Flight SQL Listeners
Trickster can serve Apache Arrow Flight SQL — the gRPC-based query protocol spoken by ADBC drivers, Grafana’s SQL-mode datasources, and the Arrow-native client SDKs — as a caching proxy in front of a Flight SQL-capable origin. The protocol support is vendor-neutral; each backend provider that adopts it supplies only its origin-specific wiring. InfluxDB 3.x is the first supported provider (see InfluxDB Support for its specifics and examples).
Enabling a Flight SQL listener
Define a listener with the flight-sql protocol and map exactly one
compatible backend to it through the backend’s listener_names:
listeners:
my-flight:
protocol: flight-sql # Apache Arrow Flight SQL over gRPC
port: 8485
backends:
influx3:
provider: influxdb
origin_url: 'http://influxdb3:8181/'
listener_names: [ default, my-flight ]
Flight SQL listeners share Trickster’s standard listener lifecycle:
connection limits (connections_limit), graceful drain on SIGTERM, and
config reload (SIGHUP) — a reload with an unchanged backend configuration
keeps serving on the existing socket, while a changed configuration drains
the old server (closing its upstream connection) and rebinds.
TLS
The listener serves TLS when the mapped backend’s tls block presents a
certificate and key, with in-place certificate rotation on config reload;
without one it serves plaintext gRPC. The upstream dial is TLS-capable via
provider options (for InfluxDB, influxdb.flight_upstream_tls).
Caching model
Every cache entry is scoped to the backend name plus a per-tenant namespace
derived from the request metadata the provider declares (for InfluxDB:
database, bucket-name, and a hash of authorization), mirroring the
scope the upstream itself grants — tenants never share entries, and
credentials never appear in cache keys.
Statement queries are served through three tiers:
- Delta proxy cache — statements the provider’s SQL analyzer classifies
as delta-cacheable are cached by time extent: repeat and overlapping
queries fetch only missing sub-ranges from the origin, and responses are
rebuilt into Arrow record batches conforming to the response’s original
schema (types, column order, nullability, and schema metadata are
preserved). A statement’s
ORDER BYis carried through reconstruction, so a delta hit returns rows in the order the statement asked for; ordering terms that do not resolve to a select-list output fall to the next tier. Responses whose schemas the delta model cannot represent fall automatically to the next tier as well. The partial buckets at a range’s edges are never delta-cached; the backend’s step alignment mode decides what the response shows for them:drop, which leaves them out, unless configured otherwise or chosen in a comment such as/* trickster-step-align:partial_end */. Thepartialmodes fetch each partial bucket as a statement of its own through the object cache, andoffanswers every statement from the object cache with its range as sent. - Object cache — everything else cacheable is stored as the verbatim Arrow IPC byte stream and returned byte-identically, with a short, provider-configured lifetime. Metadata RPCs use this tier.
- Proxy — nondeterministic and non-SELECT statements are never cached.
Serving-cost characteristics of the tiers are recorded in Flight SQL Cache Tier Benchmarks.
Supported RPC surface
- Statement execution (
GetFlightInfoStatement/DoGetStatement). - Result-set schema requests (
GetSchemaStatement/GetSchemaPreparedStatement), which ADBC drivers probe on connect. When the origin itself does not implement its schema RPC (InfluxDB 3 Core among them), the schema is derived by executing the statement through the object cache, so the probe’s cost folds into the execution the client performs next. - Catalog metadata (
GetTables,GetCatalogs,GetDbSchemas,GetTableTypes,GetSqlInfo), proxied and cached per tenant. - Type and key discovery (
GetXdbcTypeInfo,GetPrimaryKeys,GetImportedKeys,GetExportedKeys,GetCrossReference), which JDBC- and ODBC-shaped clients probe during metadata discovery, likewise proxied and cached per tenant and per request shape. - The prepared-statement lifecycle (create, bind, execute, close). A parameterless prepared statement is served through the same three statement tiers — sharing cache entries with ad-hoc executions of the same text — while parameterized executions are cached whole, keyed by the bound parameter hash. Statements abandoned by disconnected clients are closed upstream after 15 minutes of inactivity.
A FlightInfo whose result the origin partitions across several endpoints is
consumed in full: every endpoint’s ticket is resolved in order, and an
endpoint that advertises its own location is retrieved from that location
only when its exact host:port authority is explicitly allowlisted by the
provider. This opt-in is required because request metadata, including
credentials, is forwarded to the alternate host. A TLS primary is never
downgraded to a plaintext alternate. Alternate connection caches are bounded
(16 clients by default). Partitions that disagree on schema are refused rather
than served partially.
Writes/updates/ingest, transactions, savepoints, Substrait plans, query
cancellation, endpoint renewal, and session options return gRPC
Unimplemented — they are outside a read-through cache’s contract. Clients
requiring them should connect to the origin directly. Prepared-statement
bindings are accepted as a single record batch; a client streaming several
batches for one binding receives Unimplemented rather than a silently
truncated binding.
Response size bound
Responses are buffered whole so they can be cached, so a single very large
query would otherwise be able to exhaust the process heap. Each upstream
response is bounded at 128MiB by default; a response exceeding the bound is
refused with gRPC ResourceExhausted rather than being buffered. Providers
expose the bound as a backend option (for InfluxDB,
influxdb.flight_max_response_bytes; a negative value removes it). A separate
aggregate budget bounds response bytes concurrently assembled or streamed
(512MiB by default; influxdb.flight_max_buffered_bytes for InfluxDB).
Metrics
Statement executions record cache outcomes to the native SQL metrics under
the flightsql dialect label: trickster_sql_query_cache_total
(cache_mode x cache_status), the standard proxy request/points/duration
metrics, trickster_sql_query_rewrite_failures_total for extent-rendering
failures, and cache-error events on the shared cache metrics. See
Metrics.
Architecture (for provider implementers)
The protocol lives in vendor-neutral packages:
pkg/proxy/flightsql— the gRPC protocol server, caching statement/ metadata/prepared handling, and the upstream client. A provider supplies: the upstream address and TLS posture;ForwardMetadataKeys(which inbound gRPC metadata flows to the origin); aKeyScopercovering every forwarded key that affects results; and optionally aDeltaConfigcarrying its SQL dialect analyzer to enable the delta tier.pkg/proxy/engines/nativedelta— the transport-agnostic delta-proxy-cache engine (shared with the MySQL native listener), driven by the frozenpkg/parsing/sqlanalyzerdialect contract.pkg/timeseries/dataset/arrow— dataset ⇄ Arrow record batch conversion used by the delta tier.
pkg/backends/influxdb/native_listener.go is the reference wiring: a
provider integrates by implementing the native.Adapter interface
(declaring Protocol() == "flight-sql" and its own BackendProvider()) and
constructing the flightsql server from its backend options.
4 - Automatic Certificates (ACME)
Trickster can obtain and renew its own serving certificates from any certificate authority that speaks ACME (RFC 8555), such as Let’s Encrypt, ZeroSSL, Google Trust Services or a private step-ca. A backend opts in under its tls section, and Trickster issues a certificate for each of its hostnames, serves it from the listener’s certificate store, and renews it in the background before it expires.
ACME-managed certificates are served alongside certificates loaded from files and certificates supplied at runtime, and they use the same SNI selection, hot-swap and inventory machinery described in TLS Support. Issuance, renewal and storage all happen off the handshake path, so a certificate obtained through ACME costs nothing more per handshake than one loaded from a file.
Quick Start
listeners:
default:
port: 80
tls_port: 443
acme:
storage:
path: /var/lib/trickster/acme
issuers:
letsencrypt:
email: ops@example.com
agree_to_terms: true
backends:
site:
provider: reverseproxycache
origin_url: http://origin.internal:8080
hosts: [www.example.com, example.com]
tls:
acme:
issuer: letsencrypt
With this configuration, Trickster requests certificates for www.example.com and example.com from Let’s Encrypt, answers the CA’s validation on ports 80 and 443, and serves the certificates on the default listener as soon as they are issued.
The CA validates a name by connecting to it on port 80 (http-01) or port 443 (tls-alpn-01). When Trickster listens on other ports, such as the defaults of 8480 and 8483, forward 80 and 443 to them, or use the dns-01 challenge.
Try a new configuration against a staging directory first, such as https://acme-staging-v02.api.letsencrypt.org/directory, since production CAs rate-limit failed and duplicate orders.
Configuration
The acme Section
| Setting | Description | Default |
|---|---|---|
wait_on_startup | Holds readiness at startup until missing certificates are issued, or this long at most. See Startup Wait. | 0 (no wait) |
storage.provider | filesystem or redis. See Storage and Clusters. | filesystem |
storage.path | The storage directory, required for filesystem storage. | none |
storage.redis.cache_name | Borrows the connection settings of a named redis cache. | none |
storage.redis.connection | Redis connection settings, with the same fields as a cache’s redis section, when cache_name is not used. | none |
storage.redis.key_prefix | Namespaces every ACME key. | trickster:acme: |
storage.redis.lock_ttl | The lease of a distributed lock, refreshed while it is held. At least 5s. | 1m |
issuers | Named issuers, which backends reference. At least one is required. | none |
on_demand | Enables issuance during the first handshake for names no backend lists. See On-Demand TLS. | none |
Each issuer accepts:
| Setting | Description | Default |
|---|---|---|
directory_url | The CA’s ACME directory. | Let’s Encrypt production |
email | The account contact the CA sends expiry and policy notices to. | none |
agree_to_terms | Accepts the CA’s subscriber agreement. It must be true; Trickster never accepts terms implicitly. | false |
challenges | The challenge types the issuer may use: http-01, tls-alpn-01, dns-01. | [http-01, tls-alpn-01], or [dns-01] with a dns_provider |
key_type | ecdsa-p256, ecdsa-p384, rsa-2048, rsa-4096 or ed25519. | ecdsa-p256 |
profile | A certificate profile the CA advertises in its directory, such as Let’s Encrypt’s shortlived. | none |
external_account_binding | key_id, and hmac_key or hmac_key_file, for CAs that require an existing account, such as ZeroSSL and Google Trust Services. | none |
trusted_ca_paths | PEM files trusted, in addition to the system roots, for the CA’s own HTTPS endpoint, as a private CA needs. | none |
dns_provider | The DNS API that answers dns-01 challenges. See DNS-01. | none |
A Backend’s tls.acme
| Setting | Description | Default |
|---|---|---|
issuer | The issuer the backend’s certificates come from. | required |
domains | The names to request certificates for. | the backend’s hosts |
A backend that uses tls.acme cannot also set full_chain_cert_path or private_key_path. Its listeners must be http listeners with a tls_port. Other TLS settings, such as certificate_authority_paths for the origin, are unaffected.
Domains
Trickster requests one certificate per name. Names are lowercased, a trailing dot is dropped, and duplicates are removed. Each name must have at least two labels, and IP addresses are not supported.
- A wildcard such as
*.example.comrequires an issuer that usesdns-01. It covers exactly one label, so it matchesa.example.combut neitherexample.comnora.b.example.com. - A backend whose
hostsinclude an any-depth pattern such as**.example.commust list the names it wants certificates for intls.acme.domains, since no certificate can cover every depth. - A name belongs to exactly one issuer. Several backends may list it if they all use the same issuer.
Challenges
HTTP-01
The CA requests http://<name>/.well-known/acme-challenge/<token> on port 80. Trickster answers on the plaintext port of each listener that serves an http-01 domain, ahead of every route and middleware, so IP and geo access lists, rate limits and authenticators never block a CA’s validation. Only a pending token is answered. Any other request under /.well-known/acme-challenge/ gets an empty 404 and never reaches a backend. A listener that serves no http-01 domain does no extra work per request.
An issuer that allows http-01 but has no listener with a plaintext port cannot use it, and validation logs a warning.
TLS-ALPN-01
The CA connects to port 443 offering only the acme-tls/1 protocol (RFC 8737). Trickster answers those handshakes with the validation certificate, and every other handshake costs a single length comparison. TLS-ALPN-01 runs over TCP, so the HTTP/3 endpoint never answers it.
DNS-01
DNS-01 publishes a TXT record through a DNS provider’s API. It is the only challenge that can issue wildcards, and it works for names the CA cannot reach. An issuer using dns-01 uses it exclusively.
acme:
issuers:
letsencrypt-dns:
email: ops@example.com
agree_to_terms: true
dns_provider:
provider: cloudflare
cloudflare:
api_token_file: /etc/trickster/secrets/cloudflare-token
| Provider | Settings |
|---|---|
cloudflare | api_token or api_token_file (Zone.DNS:Edit), and optionally zone_token or zone_token_file |
route53 | Optionally region, profile, hosted_zone_id, and access_key_id with secret_access_key or secret_access_key_file; unset credentials use the AWS default credential chain |
rfc2136 | server (host:port), key_name, key_alg (such as hmac-sha256), and key or key_file; works with BIND, PowerDNS, Knot and other servers that accept dynamic updates |
Every provider also accepts propagation_delay, a wait before checking that the record is visible; propagation_timeout, which bounds that check (default 2 minutes, and a negative value skips it); and resolvers, the DNS servers (host:port) the check queries.
Secrets set directly in the configuration are redacted from the config handler and every config dump. The _file variants keep them out of the configuration altogether.
Storage and Clusters
ACME accounts, certificates, private keys and locks are kept in storage, so a restart serves the certificates it already has instead of ordering new ones.
Filesystem storage keeps everything under storage.path, in directories created with mode 0700 and files with mode 0600. Lock files make it safe for several processes on one host. It is not safe on network filesystems with weak locking semantics.
Redis storage lets every Trickster instance that shares the keyspace share certificates. Distributed locks ensure that only one instance orders or renews a given name. Any instance can answer a challenge another instance started, so a load balancer may send the CA’s validation to any of them. Every key carries one hash tag, so the keyspace stays within a single slot on Redis Cluster.
acme:
storage:
provider: redis
redis:
cache_name: shared-redis # borrow the connection settings of a redis cache
key_prefix: "trickster:acme:"
ACME storage never shares the cache’s keyspace, even when it borrows a cache’s connection settings. Eviction would destroy account keys and certificates, so the Redis server must not evict these keys: run it with maxmemory-policy noeviction, or without maxmemory.
Startup Wait
A brand-new deployment has no certificates until the CA issues them, and handshakes for those names fail in the meantime. wait_on_startup holds readiness until they are issued:
acme:
wait_on_startup: 90s
- Listeners bind immediately, because the CA’s challenges must reach them. What waits is the readiness endpoint (
/trickster/ready), which reports503with the bodycertificates pending, so load balancers and Kubernetes probes keep traffic away. - Only names without a usable certificate in storage are waited for, and an expired stored certificate counts as missing. A routine restart therefore becomes ready immediately.
- When the duration elapses, readiness is released anyway. Trickster logs an error for each name still missing, with the CA’s last error, and issuance keeps retrying in the background. Holding readiness indefinitely would let a CA outage take every instance out of rotation, including those serving backends that don’t use ACME.
- The wait applies only at startup. Names added by a configuration reload are issued in the background and never return readiness to
503.
On-Demand TLS
On-demand TLS issues a certificate during the first handshake for a name no backend lists. This suits a CDN or a SaaS platform whose customers bring their own domains. Because it lets clients cause certificate orders, it requires at least one gate:
acme:
on_demand:
issuer: letsencrypt
listeners: [default]
ask: http://127.0.0.1:9000/allowed # GET ?domain=<name>; a 200 permits issuance
allowed_domains: ["*.customers.example.com"] # names must also match one of these
decision_ttl: 5m # how long an answer or a refusal is remembered
rate_limit: 10 # most issuances started per minute
negative_cache_size: 10000
- When both
askandallowed_domainsare set, a name must pass both. - Refused names are remembered for
decision_ttl, so repeated handshakes for an unwanted name cost a table lookup, not a request toask. - The first handshake for a permitted name waits while the certificate is issued, which typically takes a few seconds. Later handshakes are answered from the listener’s certificate store.
- On-demand issuance cannot use a
dns-01issuer, and it applies only to TLS over TCP. Handshakes on the HTTP/3 endpoint are served from the store. - A handshake for a name the store already covers never reaches the on-demand gates.
Renewals and Reloads
Certificates are renewed during the last third of their lifetime, or earlier when the CA advises it through ACME Renewal Information (RFC 9773). A failed order or renewal is retried with backoff. Every renewal is hot-swapped into the listeners without dropping established connections.
A configuration reload reconciles what is managed:
- New names are ordered in the background.
- Removed names stop being served. Their certificates are never revoked, and they remain in storage, so adding a name back is immediate.
- When an issuer’s settings change, its names keep serving their current certificates until the reconfigured issuer replaces them.
- Changing
storagerestarts certificate management against the new storage.
Management API
The mgmt listener serves the ACME handler at /trickster/acme (configurable via mgmt.acme_handler_path). It is served only on the mgmt listener, never on proxy listeners.
GET /trickster/acmelists every managed name with its issuer.POST /trickster/acme?domain=<name>starts a forced renewal in the background and responds202 Accepted. An unmanaged name gets404.
ACME-managed certificates also appear in the certificate inventory with the source acme and the id acme:<issuer>:<name>.
Observability
The TLS certificate metrics, such as trickster_tls_certificate_expiration_time_seconds, cover ACME-managed certificates as well. These metrics are specific to ACME:
trickster_acme_orders_totalandtrickster_acme_renewals_total, byissuerandresult(successorfailure)trickster_acme_challenge_requests_total, bytype(http-01ortls-alpn-01) andresult(served,unknownorerror)trickster_acme_on_demand_decisions_total, byresult(allowed,refused,refused_cached,rate_limitedorask_error)trickster_acme_startup_wait_secondsandtrickster_acme_startup_wait_timeouts_total
See metrics.md for details.
Kubernetes
ACME is not supported together with the Kubernetes controller. With the controller, certificates come from TLS Secrets, and cert-manager is the established way to issue them. A Trickster Deployment that does not run the controller can use ACME, with a persistent volume for filesystem storage, or with Redis storage when it has several replicas.
Limitations
- ACME certificates are served on
httplisteners only. Native-protocol listeners (postgres,mysql,flight-sql,clickhouse) andtlsstream listeners use certificate files. - IP-address certificates and certificate revocation are not supported.