This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Release Notes

What’s new and changed in each Trickster release.

1 - Trickster 2.2

Trickster 2.2 extends many of the new features introduced in 2.1, while adding support for delta-caching several new TSDB’s that use the Postgres Query Dialect, as well as a new Global Query Evaluator that works with mixed Prom, Thanos and Mimir deployments. 2.2 adds multiple security and enforcement features, including ACME automatic cert renewal, IP ACLs, Rate Limiting and GeoACLs; and new CDN-grade enhancements including MoQ support.

Load Balancing and Scaling

L4 Load Balancing - We’ve extended the HTTP L7 ALB to support Layer 4 as well (TCP and UDP), with a number of new balancing mechanisms.

Sticky Sessions - We now support configurable sticky sessions for the Load Balancer feature.

New Acceleration-supported TSDBs

TimescaleDB - You can now accelerate TimescaleDB with the delta proxy cache! If you are tired of playing whack-a-mole with new continuous aggregates to manage performance, Trickster can stop the madness. Even better - any Postgres-compatible database can be fronted by Trickster for a SELECT result cache.

GreptimeDB - We’ve added GreptimeDB as an acceleration-supported backend time series provider.

QuestDB - And we also now support accelerating QuestDB.

VictoriaMetrics - And we also now support accelerating VictoriaMetrics on the MetricsQL and Graphite HTTP endpoints.

CloudWatch Metrics - We now support accelerating AWS CloudWatch Metrics on the new PromQL endpoints. Classic GetMetricData endpoints are not yet supported.

TSM Global Evaluation - A Time Series Merge ALB can now evaluate each PromQL query once over the raw series of every member, deduplicating replicas the way Thanos does, rather than merging the members’ answers. Members serve raw series through Prometheus remote read (the new mimir flavor) or the Thanos StoreAPI (the new thanos flavor), and complete results are cached, while the members’ raw series are delta-cached, so each member is read only for the samples the cache lacks. Merging also now counts label sets returned by several members once for count or group over an inner count or group. See Global Evaluation.

HTTP Reverse Proxy Cache, CDN and Streaming Media

Media over QUIC (MoQ) - Building on the HTTP/3 and QUIC support introduced in 2.1, Trickster is now a caching Media over QUIC Transport relay, for drafts -17 through -22. Native QUIC clients connect to a protocol: moq listener and browsers connect by WebTransport over HTTP/3. One upstream subscription per track fans out to every subscriber, the newest groups stay in memory and older ones in the cache for FETCH, and publishers can send tracks straight to Trickster. Origin pools, token authorization and the Kubernetes Gateway API are supported too. See Media over QUIC for details.

HTTP/3 on Kubernetes Gateways - HTTPS listeners the Kubernetes controller generates now also serve HTTP/3, and WebTransport, on the same port number over UDP. Turn it off with kubernetes.defaults.http3: false or a GatewayClass’s http3 parameter. See GatewayClass parameters.

HTTP/3 Port Release - A reload that removes or drains an HTTP/3 endpoint now releases its UDP port. In 2.1, the port stayed bound until the process exited, so a later reload could not bind it again.

HTTP QUERY Method - Trickster accepts and caches the QUERY method (RFC 10008), keyed on the request body. Time series query endpoints forward a QUERY to their origins as a POST and advertise Accept-Query. See The QUERY Method.

Automatic Certificates (ACME) - Trickster can now obtain and renew its own serving certificates from Let’s Encrypt or any other ACME certificate authority. See Automatic Certificates for details.

Disk Caches - The Filesystem and bbolt caches are rebuilt for large caches and large objects. See Disk Caches for details.

Chunked Caching Fixes - use_cache_chunking: true is honored from configuration again; since 2.0 it was silently reset to false at load. Chunked byterange caching no longer zeroes stored chunks on a partial hit, stores range misses, retries chunks that were evicted rather than serving their bytes as zeros, and no longer truncates bodies when chunks are read concurrently. A configuration reload that changes a memory cache’s chunking settings now rebuilds the cache. See Chunked Caching.

SRV Origin Resolution - A backend can resolve its origin host as a DNS SRV owner name, dialing a target and port from the SRV answer, with failover across targets and tiers. Combined with a rule and a hostname rewriter, one backend can route to any number of origins published in DNS, including ECS, Consul and Nomad services with dynamically assigned ports. See Origin Resolution via DNS SRV.

Idle Connection Timeouts - HTTP listeners now close a keep-alive connection that waits more than 2 minutes for its next request; until now idle connections were never closed. The new idle_timeout, read_timeout and max_header_bytes listener options tune this. See Connection Timeouts and Header Size.

Security & Policy

Path Normalization - HTTP listeners now remove . and .. path segments before routing and forward the cleaned path, closing a bypass of path-scoped controls such as authenticator_name: none; see Path Normalization for the new path_normalization options and opt-out.

Forwarded Hops From Trusted Proxies Only - Trickster now forwards the Forwarded, X-Forwarded-* and X-Real-IP values a request arrived with only when one of the listener’s trusted_proxies delivered it, appending its own hop; from any other peer the origin receives Trickster’s hop alone, so a client cannot hand the origin a forged address. Passthrough paths, which forwarded no prior hops before, now forward a trusted proxy’s. A listener behind a load balancer must list it in trusted_proxies for the origin to keep seeing client addresses. See Forwarding Headers to the Origin.

Rate Limiting - A named limiter counts requests, connections, sessions, or datagrams and refuses what exceeds its limit. It attaches at the listener, backend, and path. The listener limiter is its own layer. See Rate Limiting.

IP ACLs - We’ve also added IP Access Control Lists to restrict access to certain backend resources by IP. it can be attached at the listener, backend, and path levels, with most specific winning.

Geo ACLs - We now support Geolocation Access Control lists to restrict content to geographical areas through IP -> Location translation via industry-standard locator services (MaxMind, RFC 8805 geofeeds, Header from trusted downstreams). Bring your own licensed database.

Fixes

Log Level Names Ignore Case - log_level, TRK_LOG_LEVEL and -log-level now accept a level name in any case. Until now, an uppercase name such as DEBUG passed config validation but logged at info, and a config with no logging block warned that its own default level was unknown. See Environment Variables and Command Line Arguments.

Rule Inputs Read the Requested Host - A rule’s url, url_no_params, scheme, host, hostname and port inputs now describe the URL the client requested, taking the host from the Host header and the scheme from whether the request arrived over TLS. Until now they read only the request URL, which carries no host or scheme on a client request, so rules matching on these inputs never matched. port now also infers 80 or 443 from the scheme when the host names no port, as documented. A request rewriter’s host, hostname and port replace instructions likewise now act on the requested host. See input_source permitted values.

2 - Trickster 2.1

Trickster 2.1 adds tons of new features to put acceleration in even more places: Kubernetes routing, more supported backend providers, and modern protocols. 👌

The next release of Trickster will be v2.2, which has been added to the Roadmap.

A Brand New Trick: Mecone v1.0

Alongside Trickster v2.1, we’ve launched a new companion project, Mecone (pronounced like McConey) - a Reverse Proxy conformance tester that is extensible via YAML configs. We use Mecone to measure Trickster’s conformance to the HTTP protocol specifications, and also compare its conformance against other industry solutions via published quarterly reports. Check out the Mecone repo to see how Trickster stacks up today in our maiden State of the Industry report.

Trickster v2.1 Features

Request Routing

  • Kubernetes Gateway API and Ingress Controller - Trickster now runs as a Kubernetes controller, serving gateway.networking.k8s.io GatewayClass, Gateway, HTTPRoute, GRPCRoute, TCPRoute, TLSRoute, UDPRoute, ReferenceGrant and BackendTLSPolicy objects, and networking.k8s.io/v1 Ingress objects, translating them into its own configuration and reloading onto it in-process. Routes may be served through the Service’s cluster IP or load balanced across discovered endpoints, TLS certificates arrive from Kubernetes Secrets without a reload, and status, conditions and Events are written back to every claimed object. Enable it with the new top-level kubernetes section. See kubernetes-gateway.md, kubernetes-ingress.md, kubernetes-rbac.md and kubernetes-deploy.md.

    • TricksterCachePolicy Custom Resource - A custom resource that attaches caching behavior — a cache, TTLs, cache key components, CORS, header updates and time series acceleration — to Gateways, HTTPRoutes, Ingresses and Services, so a Prometheus or ClickHouse Service behind a route is accelerated rather than merely proxied. See kubernetes-cache-policy.md.
  • Auto-discovery - The ALB can now manage pool members through common auto-discovery mechanisms such as Kubernetes APIs, DNS A and SRV records, etc.

  • Regex Path Matching - You can now define path routes with regexes to match incoming requests, and expose their capture groups to request rewriters.

  • Wildcard Host Routing - A backend’s hosts may name a single-label wildcard (*.example.com) or an any-depth wildcard (**.example.com), resolved by specificity ahead of global routes. See Path Configuration Documentation.

Supported Backend Providers

  • We now support accelerating InfluxDB 3.x, including over Flight SQL (gRPC). Support for InfluxDB 1.x and 2.x remains and is unchanged.

  • We’ve expanded ClickHouse delta proxy cache support to include the native (TCP/binary) protocol. You can configure a ClickHouse Backend that accepts HTTP and proxies to an origin via Native, or conversely a Backend that accepts Native and proxies via HTTP. A single Backend configuration can also listen on both Native and HTTP.

  • We now support accelerating Graphite.

  • We now support accelerating Apache Druid.

  • We now support accelerating MySQL with Time Series-like queries (e.g., Grafana dashboards).

HTTP Protocol Conformance

  • HTTP/2 and HTTP/3 - Between the client and Trickster, cleartext listeners now accept prior-knowledge cleartext HTTP/2 (h2c) alongside HTTP/1.1, as gRPC and other h2c clients require; no configuration is required. Between Trickster and the origin, HTTP/2 is negotiated automatically via ALPN with https:// origins that offer it, also with no configuration. For http:// origins that serve only h2c, set the new h2c_prior_knowledge backend option; it is opt-in because there is no HTTP/1.1 fallback for such an origin. Listeners can also now serve their routes over HTTP/3 (QUIC) alongside HTTP/1.1 and HTTP/2. Enable the new http3 block on any TLS-enabled listener; responses on the TLS endpoint advertise the HTTP/3 endpoint via Alt-Svc so clients upgrade on their own. See http3.md.

  • Protocol Upgrades - WebSocket and other Connection: Upgrade requests are now tunneled end-to-end rather than rejected, including through paths that are otherwise cached.

  • Streaming - Responses with an unknown length and Server-Sent Events streams are now flushed to the client as bytes arrive rather than buffered, and HTTP trailers are passed through.

  • Stream Listeners - Listeners can now relay TCP connections, TLS connections by server name without terminating them, and UDP datagrams to a backend or a load balancer pool, without reading what passes. Set protocol: tcp, tls or udp on a listener; see configuring.md. The Kubernetes controller serves TCPRoute, TLSRoute and UDPRoute through them.

Configuration, Observability & Security

  • Logging - We’ve added support for customizable access logging and error logging per backend in NCSA format, a top-level access_log that captures every request no backend handled, and logging to stdout or stderr for containerized deployments. See access-logs.md.

  • Config Files - We now support loading multiple config files in the same subdirectory below the main config. We’ve also added automatic config reloading when the file contents change - including automatic detection and reloading when a TLS certificate is swapped out. See the Configuring Documentation for more info.

  • TLS Certificate Rotation and Runtime Certificates - Serving certificates renewed in place on disk by tools such as certbot or cert-manager are detected and hot-swapped into the live listener, with no reload and without dropping established connections; tls_watch_interval tunes the backstop poll. The new tls_runtime_certs listener option keeps a TLS port open with no certificate files behind it, so certificates can be supplied to the running process instead — which is how the Kubernetes controller serves Gateway and Ingress TLS from Secrets. The mgmt listener exposes a read-only certificate inventory at /trickster/certificates, and certificate expiry, load, swap and validation metrics are available with example alerting rules. See tls.md.

  • Graceful Shutdown and Readiness - A new readiness endpoint (/trickster/ready) reports whether the process is serving, and mgmt.shutdown_delay and mgmt.shutdown_drain_timeout hold listeners open and then drain in-flight requests on SIGTERM, so a rolling deployment is invisible to clients. /trickster/ping remains a liveness check. See configuring.md.

  • Real Client Addresses - Listeners now accept the PROXY protocol, and the new trusted_proxies listener option resolves the real client address from Forwarded, X-Forwarded-For or X-Real-IP only when the connection comes from a trusted address. The resolved address is what the access log records and what max_query_range rejections are logged against. See Trusted Proxies.

Developer Environment

  • All of the new supported backend time series providers are included in the Developer Environment Docker Compose.

  • We’ve refactored the Developer Environment Data Seeder job to use 100% locally generated data. No more big S3 file downloads. This also speeds up the GitHub CI/CD integration tests that depend on the dev env. Seed data is generated once as a TSV, and all seedable databases (MySQL, Druid, ClickHouse) mount the same file to load the same data to enable cross-provider testing and verification.

3 - Trickster 2.0

An All-New Bag of Tricks

Trickster 2.0 is a near-complete rewrite of the project with performance, durability and extensibility in mind. We’ve made major architectural improvements, added new features and improved performance. Here’s a comprehensive overview of it all:

Features

  • Application Load Balancer (ALB): We have a brand new Application Load Balancer available as a backend provider type, with unique and powerful options, including:
    • Fanout and merge data from multiple time series backends into a single response
      • currently supports Prometheus backends
    • Fanout and return the first response received from any backend pool member
    • Fanout and return the response with the newest ‘Last-Modified’ header from all backend pool members
    • Routing a request to a given backend based on Basic Auth username
    • General Round Robin
  • Enhanced Health Checking: Health checking now supports automated health check polling for any backend, and provides a global health status endpoint. Automated health checks for an ALB pool member backend determines whether the ALB will route requests to it or not.
  • Cache Object Chunking: We now support Cache Object Chunking. This optional configuration allows a Time Series dataset to be chunked into multiple cache entries based on a configurable chunk size
    • Also supported by standard Reverse Proxy Cache for cache-chunking objects by a Byte Range size
    • Only the chunked cache entries needed to span the request range are inspected, rather than the entire time series
    • Significantly improves Redis and Filesystem performance of large timeseries records
  • Time Series Backend Request Sharding: We now support Time Series Backend Request Sharding. This optional feature allows a client request destined for a Time Series Backend to be cloned into multiple concurrent requests with different time ranges constituting the full uncached range
    • Backend shard responses + already-cached data are merged into a single downstream response
    • Cache Chunking and TS Backend Request Sharding work together seamlessly and can be used in any combination (on/on, on/off, off/on, off/off, including different cache chunk and request shard sizes).
  • Authenticator: We’ve added a new Authenticator feature so you can guard backends with Basic Auth or ClickHouse Auth
  • Enhanced Rules Engine: The Rules Engine now supports rmatch operations to permit regular expression-based routing against any part of the HTTP request
  • Request Rewriters: You can now chain a collection of request rewriters for more robust request transformation possibilities
  • Cache Purging: We now support purging specific cache items by Key (on the public ports) or Path (on the mgmt port). Read more in the cache documentation
  • Simulated Latency: You can use Trickster for Simulated Latency in lab environments
  • We’ve added support for InfluxDB 2.0 and Flux Query Language, plus InfluxDB 3.x via the native v3 HTTP API (SQL and InfluxQL) and an optional Apache Arrow Flight SQL (gRPC) proxy. See the InfluxDB support doc for details.

Configuration & Security

  • YAML Configuration: Trickster 2.0 uses YAML for configuration instead of TOML
  • Environment Variable Substitution: Environment Variable substitution is now possible in configuration files where sensitive information is expected
    • Supported via the following fields:
      • caches[*].redis.password, backends[*].healthcheck.headers, backends[*].paths[*].request_headers, backends[*].paths[*].request_params, backends[*].paths[*].response_headers, authenticators[*].users
    • Usage: password: ${MY_SECRET_VAR}
  • Request Body Size Limit: A configurable Request Body Size limit has been added for POST, PUT and PATCH requests, with a default of 10MB. Requests with a body size exceeding the limit will receive a 413 Content Too Large response. See Request Body Handling Customizations for more info
  • Zipkin Deprecation: Trickster uses packages provided by the OpenTelemetry project for Distributed Tracing capabilities. These external packages recently deprecated support for exporting spans via Zipkin, therefore Zipkin support has been removed for Trickster 2.0. Trickster retains support for OTLP.

Under the Hood & Performance

  • Common Time Series Format: We now use a common time series format internally for caching all supported TSDB’s, rather than implementing delta + merge algorithms per-provider.
  • New HTTP Request Router: We’ve switched to an all-new, made-for-proxies HTTP Request Router, which is up to 10X faster than the previous one
  • SQL Parser: We’ve switched from Regular Expression matches for SQL-based Time Series Backends to an extensible lexer/parser solution, providing better performance and accuracy
    • ClickHouse backend providers now use the new SQL Parser
  • Race Condition Fixes: We’ve eliminated nearly 100 race conditions and random panics
  • Expanded Compression Support: Compression support now includes options for Brotli and Zstd

Developer & Environment Improvements

  • Package Reorganization: We’ve re-organized many packages in the codebase to be more easily importable by other projects. A future maintenance release will add documentation to the examples folder for using Trickster packages in your own projects, including caching, acceleration and load balancing
  • For Trickster contributors, we have a new Docker Compose for developer environments.
  • CI/CD Enhancements: We have added new CI tools, including better linters and race condition checkers to enforce and ensure ongoing project quality
  • Docker Automation: We’ve updated our Docker automation:
    • The trickster-docker-images repo is now retired and image publishing is handled in the trickster repo
    • All merges to main will now push an image to Docker Hub at trickstercache/trickster:main as well as to trickstercache/trickster:<COMMIT_ID>
      • We no longer push images to the legacy tricksterio and tricksterproxy orgs on DockerHub. Everything is now and only trickstercache 🎉!
    • Images are also now pushed to the GitHub Container Repository as ghcr.io/trickstercache/trickster
  • Helm Charts: The Helm Charts repository is now updated for Trickster 2.0
  • Vendor Directory: We no longer include the vendor directory in the project repository and vendor is now in .gitignore. vendor will continue to be included in Release source tarballs

Documentation & Examples

  • Example Configurations: Example configurations are relocated to the examples directory
  • Docker Compose Demo: The Trickster docker-compose demo has been relocated to the examples directory and updated to use the latest version tags. This is the easiest way to try out Trickster 2.0!

Still to Come

A future Trickster 2.0.x will include the following additional features that didn’t quite make it to the finish line:

  • incorporate ALB examples into the docker-compose demo

Installing

You can build the 2.0 binary from the main branch, download binaries from the Releases page, or use the trickstercache/trickster Docker image tag in containerized environments