Providers

• 1: Prometheus Support
• 2: InfluxDB Support
• 3: ClickHouse Support
• 4: Graphite Provider
• 5: MySQL Provider

1 - Prometheus Support

Trickster fully supports accelerating Prometheus, which we consider our First Class backend provider. They work great together, so you should give it a try!

Most configuration options that affect Prometheus reside in the main Backend config, since they generally apply to all TSDB providers alike.

Supported API Endpoints

Trickster supports the full Prometheus HTTP API (v1), including features introduced in Prometheus 3.x.

Cached Endpoints

Proxied Endpoints (not cached)

All other /api/v1/* paths are reverse-proxied to the origin without caching.

Prometheus 3.x Features

• Native histograms are fully supported in query and query_range responses, including mixed series with both float samples and histogram samples.
• UTF-8 metric and label names (e.g., {"metric.name"}) are supported in queries and cache keys.
• Query stats (stats=all parameter) are cache-key differentiated, so responses with and without stats are cached separately.

Injecting Labels

Trickster can inject labels on a per-backend basis into Prometheus responses before returning them to the caller.

Here is the basic configuration for adding labels:

backends:
  prom-1a:
    provider: prometheus
    origin_url: http://prometheus-us-east-1a:9090
    prometheus:
      labels:
        datacenter: us-east-1a

  prom-1b:
    provider: prometheus
    origin_url: http://prometheus-us-east-1b:9090
    prometheus:
      labels:
        datacenter: us-east-1b

Interaction with ALB Merge Strategy

When using label injection with an ALB configured for Time Series Merge, injected labels are automatically stripped from responses before merging. This ensures that series from different backends are aggregated correctly, and the injected labels do not appear in the final response to the caller. See the ALB Merge Strategy documentation for details.

Max Query Range Limitation

Trickster supports enforcing a max_query_range limit on Prometheus backends. For details on how to configure and use query range limits, see the Query Range Limits documentation.

2 - InfluxDB Support

Trickster provides support for accelerating InfluxDB queries that return time series data normally visualized on a dashboard. Acceleration works by using the Time Series Delta Proxy Cache to minimize the number and time range of queries to the upstream InfluxDB server.

Scope of Support

Trickster is tested with the built-in InfluxDB DataSource Plugin for Grafana v5.0.0.

Trickster uses InfluxDB-provided packages to parse and normalize queries for caching and acceleration. If you find query or response structures that are not yet supported, or providing inconsistent or unexpected results, we’d love for you to report those so we can further improve our InfluxDB support.

Trickster supports integrations with InfluxDB 1.x, 2.x, and 3.x.

InfluxDB 3.x Support

Trickster supports InfluxDB 3.x via both the native v3 API endpoints and the v1/v2 compatibility endpoints.

Supported v3 Endpoints

• GET/POST /api/v3/query_sql — SQL queries with delta-proxy caching
• GET/POST /api/v3/query_influxql — InfluxQL queries with delta-proxy caching
• POST /api/v3/write_lp — line protocol writes (proxied, not cached)

All other v3 API paths (including the /api/v3/configure/* management endpoints, with any HTTP method) are proxied through untouched.

Query requests on both query endpoints may arrive as URL parameters (GET), an application/json document ({"q": ..., "db": ..., "format": ..., "params": ...}), a form-encoded body, or a raw statement body — all four shapes are parsed and cached equivalently, and the db and params values participate in the cache identity.

InfluxQL over v3

/api/v3/query_influxql requests use the same delta-proxy caching as the v1 /query endpoint (queries with a GROUP BY time(...) interval and a time-bounded WHERE clause), but speak the v3 request/response shapes: the v3 request document above and the v3 tabular response formats, including the iox::measurement column, which Trickster treats as a series tag alongside any GROUP BY tags.

SQL Query Caching

SQL queries using date_bin() or date_trunc() for time binning are parsed for time range extraction and step detection, enabling delta-proxy caching. Trickster extracts time ranges from WHERE clauses and intervals from date_bin(INTERVAL '...', time) or date_trunc('unit', time) in SELECT. Live, unaligned time ranges (the shape dashboard tools emit) are rounded inward to complete buckets. Grouped queries (GROUP BY 1, host) are cached per tag series.

Example query that Trickster will accelerate:

SELECT date_bin(INTERVAL '1 hour', time) AS time, avg(temperature)
FROM weather
WHERE time >= '2024-01-01 00:00:00' AND time < '2024-01-02 00:00:00'
GROUP BY 1

SELECT queries that cannot be delta-cached (no fixed-cadence time bucket, joins, subqueries, window functions, compound selects such as UNION, LIMIT, variable-length buckets like '1 month', or unsafe time predicates) fall back to the object proxy cache, which caches the whole response briefly and passes results through unchanged. Non-SELECT statements and parameterized queries (a params field in the request) are proxied to the origin without delta caching.

Queries without an upper time bound run to the present; for these, Trickster’s backfill tolerance is floored at one bucket so the still-filling final bucket is always refreshed from the origin rather than cached as complete.

Response Formats

Trickster supports the following v3 response formats, controlled by the format query parameter (in the URL or the request document) or, when no format is given, the Accept header (application/json, application/jsonl, text/csv, …):

• json (default) — JSON array of objects
• jsonl — JSON Lines (one JSON object per line)
• csv — standard CSV with header row

The parquet and pretty formats are not supported for caching and will be proxied through.

v1/v2 Compatibility

InfluxDB 3.x ships with v1 and v2 compatibility endpoints. Trickster’s existing InfluxQL support works against these endpoints with no additional configuration — just point Trickster at the v3 instance and query via /query.

Flight SQL (gRPC)

InfluxDB 3.x exposes SQL via Apache Arrow Flight SQL on gRPC in addition to HTTP. Grafana’s InfluxDB datasource in SQL mode, the Python/Rust/Java SDKs, and ADBC all default to Flight SQL, so HTTP-only caching misses a significant fraction of real-world query traffic. The protocol-level (backend-agnostic) listener documentation lives in Apache Arrow Flight SQL Listeners; this section covers the InfluxDB specifics.

Trickster can expose a Flight SQL server that proxies to an upstream Flight SQL endpoint and caches the Arrow IPC byte stream. Enable it by defining a listener with the flight-sql protocol and mapping exactly one InfluxDB backend to it (alongside its usual HTTP listener):

listeners:
  influx3-flight:
    protocol: flight-sql   # Apache Arrow Flight SQL over gRPC
    port: 8485
backends:
  influx3:
    provider: influxdb
    origin_url: 'http://influxdb3:8181/'
    listener_names: [default, influx3-flight]
    influxdb:
      flight_upstream_address: 'influxdb3:8181'   # optional, defaults to origin_url host
      flight_max_response_bytes: 134217728        # optional, defaults to 128MiB
      flight_max_buffered_bytes: 536870912         # optional, aggregate in-flight budget; defaults to 512MiB
      flight_allowed_location_hosts:               # optional exact authorities for alternate endpoints
        - 'influxdb3-replica:8181'

The authorization and database headers are forwarded from the client through to the upstream, and every cache entry is keyed by the backend name, the database/bucket-name headers, and a hash of the authorization header — so tenants and databases never share cache entries. Requests that send no authorization header share one anonymous cache scope, mirroring the access the upstream would grant them.

Statement queries are served through a three-tier cache:

1. Delta proxy cache — queries the SQL analyzer classifies as delta-cacheable (the same date_bin()/date_trunc() shapes as the HTTP path) are cached by time extent: repeat and overlapping queries fetch only the missing sub-ranges from the upstream, and responses are rebuilt into Arrow record batches conforming to the response’s original schema. A query’s ORDER BY is carried through that rebuild, so a cache hit returns rows in the requested order; ordering terms that do not resolve to a select-list output fall to the next tier. Entries use the backend’s timeseries_ttl and honor backfill_tolerance; still-filling buckets are always refetched. Responses whose Arrow schemas the delta model cannot represent (nested types, non-string dictionaries, …) automatically fall to the next tier.
2. Object cache — everything else cacheable is stored as the verbatim Arrow IPC byte stream, returned byte-identically, with a lifetime of influxdb.flight_cache_ttl (default 60s). Metadata RPCs and prepared statements always use this tier.
3. Proxy — statements referencing nondeterministic functions (now(), current_timestamp, random(), …) and non-SELECT statements are never cached.

Flight SQL TLS

The Flight listener serves TLS when its mapped backend’s tls block presents a certificate and key (the same mechanism HTTP listeners use); certificate rotation applies on config reload without dropping the listener. Without a certificate the listener serves plaintext gRPC. The upstream dial uses TLS when influxdb.flight_upstream_tls is set, honoring the backend tls block’s insecure_skip_verify.

Unsupported Flight SQL RPCs

Trickster proxies queries, result-set schema requests (GetSchemaStatement/GetSchemaPreparedStatement, which ADBC drivers probe on connect), the metadata RPCs below, and the prepared-statement lifecycle. Other Flight SQL RPCs — writes/updates/ingest (ExecuteUpdate, DoPut ingestion), transactions, savepoints, Substrait plans, query cancellation, endpoint renewal, and session options — return gRPC Unimplemented; clients requiring them should connect to InfluxDB directly. A prepared-statement binding must arrive as a single record batch; a client streaming several batches for one binding receives Unimplemented rather than a binding taken from its first batch alone.

Flight SQL listeners share Trickster’s standard listener lifecycle: connection limits (connections_limit), graceful drain on SIGTERM (active streams are drained until the configured drain timeout, then closed), and config reload (SIGHUP) — a reload with an unchanged backend configuration keeps serving on the existing socket, while a changed configuration drains the old server and rebinds.

Metadata RPCs

ADBC clients (Grafana’s SQL datasource, the Python adbc_driver_flightsql, etc.) probe the following RPCs on connect to populate schema browsers and query editors. Trickster proxies these to the upstream and caches the Arrow IPC response:

• GetFlightInfoTables / DoGetTables
• GetFlightInfoCatalogs / DoGetCatalogs
• GetFlightInfoSchemas / DoGetDBSchemas
• GetFlightInfoTableTypes / DoGetTableTypes
• GetFlightInfoSqlInfo / DoGetSqlInfo
• GetFlightInfoXdbcTypeInfo / DoGetXdbcTypeInfo
• GetFlightInfoPrimaryKeys / DoGetPrimaryKeys
• GetFlightInfoImportedKeys / DoGetImportedKeys
• GetFlightInfoExportedKeys / DoGetExportedKeys
• GetFlightInfoCrossReference / DoGetCrossReference

Flight SQL response size

Flight responses are buffered whole so they can be cached. influxdb.flight_max_response_bytes bounds one upstream response (128MiB by default), while influxdb.flight_max_buffered_bytes bounds response bytes concurrently assembled or streamed (512MiB by default). Exceeding either bound returns gRPC ResourceExhausted; a negative value removes that bound.

A FlightInfo partitioned across several endpoints is consumed in full. Alternate endpoint locations are followed only when their exact host:port authority appears in influxdb.flight_allowed_location_hosts, because the request’s authorization and database metadata are forwarded to that host. A TLS upstream never follows a plaintext alternate location. influxdb.flight_max_location_clients bounds cached alternate connections and defaults to 16.

Prepared Statements

Trickster proxies the full prepared statement lifecycle: CreatePreparedStatement, DoPutPreparedStatementQuery (parameter binding), DoGetPreparedStatement, ClosePreparedStatement. A parameterless prepared statement is equivalent to executing its statement text, so it is served through the same three cache tiers as plain statement queries — delta caching included — and shares their cache entries. Executions with bound parameters are cached whole, keyed by the bound parameter hash so two clients running the same statement with different values don’t alias. Prepared statements abandoned by disconnected clients are closed upstream after 15 minutes of inactivity.

Note: InfluxDB 3 Core 3.10 reports a parameter schema at prepare time but does not resolve bound values during query planning (upstream limitation). Parameterless prepared statements work end-to-end; parameterized ones require a newer Core or Enterprise build.

Flux Language Support

Trickster supports the Flux Query Language for general/basic usage with InfluxDB 1.x and 2.x. Flux is not supported in InfluxDB 3.x.

The delta-proxy cache accepts now() as a range() bound and handles queries with aggregateWindow(every: ...) – the common Grafana shape. Multi-table Flux CSV responses (one table per series in the result set) are also read correctly.

Trickster does not support advanced union-style queries (e.g., with multiple from clauses). In this rare use case, these responses will currently provide invalid data, however, a subsequent beta will proxy unsupported requests.

Trickster currently does not properly handle schema changes within a response CSV body (e.g., multiple CSVs in the same document with their own #annotation and header rows). We will fully support this use case in a future beta.

Max Query Range Limitation

Trickster supports enforcing a max_query_range limit on InfluxDB backends. For details on how to configure and use query range limits, see the Query Range Limits documentation.

3 - ClickHouse Support

Trickster will accelerate ClickHouse queries that return time series data normally visualized on a dashboard. Acceleration works by using the Time Series Delta Proxy Cache to minimize the number and time range of queries to the upstream ClickHouse server.

Scope of Support

Trickster is tested with the official ClickHouse DataSource Plugin for Grafana v4.21.1 and supports acceleration of queries constructed by this plugin using its built-in time macros like $__fromTime and $__toTime. Delta caching with this an other plugins (e.g., Altinity/Vertamedia) depends on the supported query shapes below. Trickster also supports several other query formats that return “time series like” data.

Trickster also supports the ClickHouse Go SDK (clickhouse-go/v2) over its HTTP and Native protocols, including clickhouse.OpenDB.

Native Binary Protocol Support

Inbound and upstream protocols are configured independently:

• A named listener with protocol: clickhouse accepts Native client connections.
• A ClickHouse backend with protocol: native uses the Native protocol for its origin. An empty protocol or protocol: http uses HTTP.

listeners:
  default:
    protocol: http
    port: 8480
  clickhouse-native:
    protocol: clickhouse
    port: 8487

backends:
  click1:
    provider: clickhouse
    origin_url: http://clickhouse:8123
    listener_names: [default, clickhouse-native]
    cache_name: default

This exposes one HTTP route at /click1/ and one Native listener on port 8487, backed by the same cache and HTTP ClickHouse origin. A Native listener must map to exactly one backend; ALB and user-router multiplexing are not supported.

To use a Native origin instead, set the backend protocol and point origin_url at ClickHouse’s Native port:

backends:
  click1:
    provider: clickhouse
    origin_url: http://clickhouse:9000
    protocol: native

All four HTTP/Native ingress and HTTP/Native origin combinations are supported. Native credentials, database, query settings, parameters, and query IDs are forwarded to the origin. Native identity pools are bounded to 64 credential/database combinations per backend.

TLS

Native listener TLS uses the backend’s server certificate. Set require_tls: true and provide both certificate paths:

listeners:
  clickhouse-native-tls:
    protocol: clickhouse
    port: 9441
    tls_watch_interval: 30s

backends:
  click1-tls:
    provider: clickhouse
    origin_url: http://clickhouse:8123
    listener_names: [clickhouse-native-tls]
    require_tls: true
    tls:
      full_chain_cert_path: /etc/trickster/tls/server.crt
      private_key_path: /etc/trickster/tls/server.key

Certificate changes are hot-swapped. Since require_tls applies to every binding of a backend, use a separate backend entry when plaintext and TLS listeners must coexist.

For a TLS Native origin, use an https origin URL with the Native protocol. The common backend TLS options configure origin verification and optional mutual TLS:

backends:
  click1:
    provider: clickhouse
    origin_url: https://clickhouse:9440
    protocol: native
    tls:
      certificate_authority_paths: [/etc/trickster/tls/origin-ca.crt]
      client_cert_path: /etc/trickster/tls/client.crt
      client_key_path: /etc/trickster/tls/client.key

Native Limitations

The Native listener supports SELECT queries, ping/pong, revision 54460 framing, structured errors, and LZ4 compression. INSERT, external tables, and session-changing USE or SET statements are rejected; select a database and settings per request. ClientCancel is accepted as a no-op and does not asynchronously cancel an active upstream request.

Supported native protocol data types: all integer types (8–256 bit), Float32/64, String, FixedString(N), DateTime, DateTime64, Date, Date32, UUID, IPv4, IPv6, Enum8/16, Bool, Nullable(T), Array(T), Map(K,V), Tuple(T1,T2,…), LowCardinality(T), and Decimal.

Native wire clients receive Native blocks. HTTP clients using a Native origin may request JSON, Native, CSV, or TSV-family output; compound columns in CSV/TSV and unsupported formats return an error. The modeler recognizes Native origin responses through the X-ClickHouse-Format response header, and HTTP Native framing honors client_protocol_version.

Trickster parses incoming ClickHouse statements into a full abstract syntax tree using the AfterShip ClickHouse SQL parser, then applies its own semantic analysis to determine whether a query is eligible for time series delta caching and, if so, its timestamp column, bucket cadence, time range, grouping tags, and cache identity. The cache key is derived from a canonical form of the query in which the requested time range is replaced with placeholders, so requests for different time ranges of the same logical series share one delta cache entry.

Trickster’s analysis fails closed: a valid query whose shape cannot be proven safe for delta caching is never rewritten approximately. It is instead served through the Object Proxy Cache (OPC) or proxied directly, and the classification reason is exported through the trickster_sql_query_analysis_total metric.

If you find query or response structures that are not yet supported, or providing inconsistent or unexpected results, we’d love for you to report those. We also always welcome any contributions around this functionality.

Delta-Cacheable Queries

To be eligible for the delta cache, a query must be a single SELECT statement containing a recognized time-bucketing expression in its select list, a supported time range in its WHERE or PREWHERE clause, and a GROUP BY clause that includes the time bucket. Each requirement is described below.

Time-Bucketing Expressions

Exactly one select-list expression must match a supported bucket form:

Grafana Plugin Format

SELECT intDiv(toUInt32(time_col), 60) * 60 [* 1000] [AS alias]

This is the approach used by the Grafana plugin. The argument to the ClickHouse intDiv function is the step value in seconds, since the toUInt32 function on a datetime column returns the Unix epoch seconds. An optional output multiplier of 1000, 1000000, or 1000000000 selects millisecond, microsecond, or nanosecond output timestamps.

ClickHouse Time Grouping Functions

SELECT toStartOfInterval(time_col, INTERVAL n unit) [AS alias]

with a positive constant n and a unit of millisecond, second, minute, hour, day, or week; or one of the fixed-period functions:

toStartOfNanosecond
toStartOfMicrosecond
toStartOfMillisecond
toStartOfSecond
toStartOfMinute
toStartOfFiveMinute
toStartOfTenMinutes
toStartOfFifteenMinutes
timeSlot
toStartOfHour
toStartOfDay
toStartOfWeek
toMonday

date_trunc('unit', time_col) and dateTrunc support the same fixed units. toStartOfWeek uses ClickHouse’s Sunday phase while toMonday and weekly date_trunc use Monday. Calendar-length month, quarter, and year buckets are served through the OPC.

The time column may be a plain, qualified (table.col), or quoted identifier, optionally wrapped in toDateTime, toInt32, or toUInt32. Integer constants defined in a scalar WITH clause may be used for the step value. Timezone-parameter variants of these functions (for example toStartOfHour(time_col, 'America/Denver')) are not eligible for delta caching and are served through the OPC. The current SQL parser does not accept INTERVAL MICROSECOND or INTERVAL NANOSECOND; use the corresponding fixed toStartOf... function.

Determining the Requested Time Range

Time range predicates must appear in a top-level AND conjunction of the WHERE or PREWHERE clause. Predicates joined by OR or negated with NOT make the query ineligible for delta caching.

Two predicate targets are supported, with different rules:

• The raw time column (the column inside the bucket function): the lower bound must be inclusive (>=) and the upper bound exclusive (<), and both values must fall exactly on bucket boundaries. Other comparators — including BETWEEN — describe partial buckets whose aggregates cannot be safely cached, so those queries are served through the OPC.
• The bucket alias (the output of the bucket expression): >, >=, <, <=, and BETWEEN are all supported, because bucket outputs are discrete; Trickster normalizes each comparator to the first and last included bucket.

Bound values may be expressed as epoch integers, ClickHouse string dates in the form 2006-01-02 15:04:05 (or date-only, or RFC3339), toDateTime(n), toDateTime64(n, precision), or toDate(n) wrappers, WITH-clause constants, or now()/now64() with optional addition or subtraction of seconds. DateTime64 precision is retained. Floating epoch bounds and timezone-qualified conversions such as toDateTime(n, 'America/Denver') are not eligible.

If no upper bound is present, Trickster caches results up to the current time and inserts a safe upper bound into origin requests automatically.

Examples of delta-cacheable time range clauses (for a one-minute bucket cadence):

WHERE t >= '2020-10-15 00:00:00' AND t <= '2020-10-16 12:00:00'  -- bucket alias
WHERE t BETWEEN 1574686320 AND 1574689920                        -- bucket alias
WHERE time_col >= toDateTime(1574686320) AND time_col < toDateTime(1574689920)
WHERE t >= now() - 3600 AND t < now()                            -- bucket alias

Secondary date-range predicates whose values match the primary range — such as the Date-typed partition filters emitted by the Grafana plugin — are recognized and rewritten in step with the primary range.

Grouping and Result Shape

The GROUP BY clause must include the time bucket (by alias or by its full expression), and every non-aggregate column in the select list must also be grouped. Grouped columns become the series tags in the cached time series. Queries using GROUP BY ... WITH CUBE/ROLLUP, grouping on expressions that are not selected, or leaving a selected dimension ungrouped are served through the OPC.

Output Formats

Delta-cacheable queries may specify FORMAT JSON, CSV, CSVWithNames, TabSeparated (TSV), TabSeparatedWithNames, or TabSeparatedWithNamesAndTypes, or omit the FORMAT clause. Trickster requests TSVWithNamesAndTypes from the origin and re-marshals cached data into the client’s requested format.

Non-Time-Series Queries

Queries that are not cacheable as time series — such as LIMIT-based queries, queries with set operations (UNION, EXCEPT, INTERSECT), SELECT 1 health checks, or SDK handshake requests — are transparently proxied to the upstream ClickHouse server. These requests are cached using the Object Proxy Cache (OPC) with per-query cache keys derived from the query and database URL parameters, ensuring that different SQL statements receive distinct cache entries.

Health and Ping Endpoint

Trickster exposes a /ping endpoint that returns a health check response, matching the endpoint provided by ClickHouse itself. This enables compatibility with clients and SDKs that probe /ping during connection initialization.

Normalization and “Fast Forwarding”

Trickster will always normalize the calculated time range to fit the step size, so small variations in the time range will still result in actual queries for the entire time “bucket”. In addition, Trickster will not cache the results for the portion of the query that is still active – i.e., within the current bucket or within the configured backfill tolerance setting (whichever is greater).

Per-query behavior can be adjusted with comment directives such as trickster-backfill-tolerance; see Per-Query Instructions.

Observability

Query classification outcomes are exported through two low-cardinality metrics that never include query text:

• trickster_sql_query_analysis_total — labeled by backend, dialect, cache mode (delta, object, none), and a stable reason code such as delta_cacheable, unsafe_predicate, or unsupported_bucket.
• trickster_sql_query_rewrite_failures_total — counts failures to render an origin request from a cached query plan.

With debug logging enabled, classification decisions are also logged with the same structured reason codes.

Max Query Range Limitation

Trickster supports enforcing a max_query_range limit on ClickHouse backends. For details on how to configure and use query range limits, see the Query Range Limits documentation.

4 - Graphite Provider

Trickster accelerates Graphite’s render API with the Delta Proxy Cache: it caches each metric at the origin’s native resolution and, on subsequent requests, fetches only the time ranges it does not already hold.

Graphite makes that harder than most time series databases, because a render request does not say what resolution it will be answered at. Whisper chooses an archive based on how old the query’s from is, and no API exposes a metric’s archive ladder. Trickster therefore learns each ladder by probing the origin, caches what it learns, and accelerates only the requests whose resolution it can predict. Everything else is served correctly through ordinary object caching.

Specify graphite as the provider:

backends:
  graphite1:
    provider: graphite
    origin_url: 'http://graphite.example.com:80'
    cache_name: default

That is the whole minimum configuration. Graphite’s cache sizing differs from every other provider’s, so the provider supplies its own defaults rather than asking operators to discover them; see Sizing for what they are and when to change them.

Compatibility

Tested against graphite-web 1.1.10 with Whisper storage, and written against its source. It should work with any Graphite-protocol origin that serves /render and /metrics/expand, including go-carbon’s carbonserver, graphite-clickhouse and carbonapi, but only graphite-web is verified.

Trickster always requests format=json from the origin regardless of what the client asked for, and renders the client’s format from the cached series. JSON is the only render format that carries the tags object, which Grafana relies on, and it is what makes one cached series able to serve every output format.

Configuration

Every common backend option applies (cache_name, timeseries_ttl, timeseries_retention_factor, backfill_tolerance, max_object_size_bytes, timeout, healthcheck, paths, TLS, authenticators); two of them, max_object_size_bytes and timeseries_retention_factor, take Graphite-specific defaults, as Sizing explains.

Per-path request_headers and request_params apply to the synthetic requests resolution makes (probes, wildcard expansion) as well as to proxied traffic, so an origin behind a static API key or one that needs local=1 works for learning too. What cannot work is per-client upstream identity: resolution state (learned ladders, wildcard expansion tokens, the registry generation) is backend-wide, and synthetic requests carry only the backend’s configured identity. Trickster therefore declines acceleration — the request is served correctly through the object cache, which keys on the client’s Authorization and every declared result-affecting input — whenever a render request carries an Authorization header, a header named in the render path’s cache_key_headers, or a parameter or form field named in its cache_key_params/cache_key_form_fields (such as local, which selects the clustered origin view) that the path’s request_headers/request_params does not statically override, and whenever the render path or the backend itself has a request rewriter (req_rewriter_name) attached, since a rewriter can change the upstream host, path, headers, or parameters in ways synthetic requests do not see. The fallback is counted as trickster_graphite_fallbacks_total{reason="client_identity"}.

If every client shares one upstream identity — Grafana with a configured datasource credential is the common case — supply it in the graphite block to re-enable acceleration:

backends:
  graphite1:
    provider: graphite
    origin_url: 'http://graphite.example.com:80'
    graphite:
      origin_username: 'metrics'
      origin_password: '<the origin password>'

For origins using a non-Basic scheme, set origin_authorization to the verbatim header value (e.g., 'Bearer <token>') instead of the username/password pair.

The credential becomes the Authorization request header of every path — proxied client traffic, synthetic resolution requests (probes, wildcard expansion) and the default health check alike — so the origin sees one static identity, a client-supplied Authorization header is replaced rather than forwarded, and requests accelerate regardless of what the client sent. Cached objects are keyed on the configured credential, so rotating it invalidates previously cached responses and learned resolution state rather than serving the old tenant’s data. To authenticate Trickster’s own clients as well, attach an authenticator via authenticator_name: the validated client credential is stripped before proxying and the origin credential is applied in its place.

A path config that sets its own Authorization in request_headers (or removes it with -Authorization) overrides the origin credential for that path. Appending one with +Authorization alongside an origin credential is rejected at startup, since the effective header would be ambiguous. If per-path identities then diverge across the paths resolution uses — /render and /metrics/expand selecting different request_headers/request_params for synthetic GETs, or GET and POST /render pinned to different credentials — Trickster declines acceleration rather than mix upstream namespaces, counted as trickster_graphite_fallbacks_total{reason="resolution_identity"}.

The graphite block adds provider-specific options:

backends:
  graphite1:
    provider: graphite
    origin_url: 'http://graphite.example.com:80'
    graphite:
      time_zone: UTC
      passthrough_max_data_points: false
      find_cache_ttl: 1m
      resolution_registry:
        ttl: 24h
        negative_ttl: 30s
        max_entries: 100000
        persist: true
        probe_concurrency: 2
        probe_budget: 96
      static_retentions:
        - pattern: '^collectd\.'
          retentions: '10s:6h,1m:7d,10m:5y'

Health checks

The default health check is GET /metrics/find?query=*, which every Graphite-protocol implementation serves. /version is not used because carbonapi and graphite-clickhouse answer it differently.

A configured origin credential rides on the probe even when the healthcheck block declares custom headers. To override it, set your own Authorization value there; to send the probe unauthenticated, set Authorization: ''.

Routed paths

How resolution prediction works

Why it is necessary

Whisper picks an archive from the age of the query’s left edge, not from the width of the window:

diff = now - fromTime
for archive in header['archives']:
    if archive['retention'] >= diff:
        break
step = archive['secondsPerPoint']

So from=-7d&until=-6d and from=-7d&until=now are answered at the same step, and a query one second either side of a rung boundary is answered at different steps. The delta cache must know the step before it can compose a cache key or reason about which ranges it already holds — and the step is a property of the metric’s on-disk file, which nothing in the request reveals.

Static configuration is not sufficient on its own, because a Whisper file keeps whatever ladder it was created with. Editing storage-schemas.conf affects only newly created files, so in any long-lived installation the config and the files routinely disagree.

Probe and learn

On the first request for a metric Trickster does not know the ladder, so the request is served through the object cache — correct, just not delta-cached — and the response itself teaches the step at that age. In the background, Trickster then discovers the metric’s full ladder with a short series of synthetic /render probes: a geometric sweep outward in time to find where the step changes, then a binary search for each rung boundary, and a pair of probes to establish maxRetention. Each probe asks for a one-second window and pins the origin’s reference time with now, so it is cheap and exact.

Ladders are shared. They originate from storage-schemas.conf patterns, so a deployment with thousands of metrics typically has only a handful of distinct ladders; a new metric is first confirmed against the ladders already known (about 7 probes) and only discovered from scratch (about 40) if none matches. In the developer environment, 24 metrics across 4 ladders converge in about 380 probes total, after which probing stops entirely until the TTL expires.

Once a ladder is known, prediction is arithmetic and costs nothing.

Confidence levels

Every resolved target carries a confidence, exported as the confidence label on trickster_graphite_resolution_lookups_total:

A request with several targets reports the weakest confidence among them.

Verification, and what happens when a prediction is wrong

The step is not merely predicted; it is checked. Every response Trickster models is compared against the predicted step. A mismatch means the cached ladder was wrong — a whisper-resize.py, a metric moved between schemas — and Trickster:

1. discards the response rather than caching it under the predicted key,
2. increments trickster_graphite_step_mispredictions_total and logs a warning,
3. invalidates the registry generation, so every entry learned under the old assumption misses rather than colliding,
4. relearns the affected ladders, and
5. re-serves the request through the object cache, so the client still gets the correct answer.

One response shape cannot self-verify: JSON carries no explicit step, so a fetch that returns fewer than two points is consistent with any prediction. Trickster never trusts one. A delta fetch that would cover a single bucket — the tip fetch every steady-state refresh performs — is widened one bucket into the past, which cannot change the archive whisper selects (the pinned now keeps now - from identical) but makes the response two points, and therefore self-verifying, at no extra request cost. Any fetched response that still cannot prove its step — one point, or no series where the prediction promised data inside retention — is refused outright: nothing is cached, and the request is served through the object cache instead, while the background learner re-establishes the ladder.

trickster_graphite_step_mispredictions_total should be flat at zero. A non-zero value is not a client-visible error, but it does mean something the registry believed was wrong.

Static retentions

static_retentions mirrors storage-schemas.conf: an ordered list, first match wins, patterns matched anywhere in the metric path exactly as carbon applies them.

    graphite:
      static_retentions:
        - pattern: '^collectd\.'
          retentions: '10s:6h,1m:7d,10m:5y'
        - pattern: '\.count$'
          retentions: '1m:2d,5m:30d'
        - pattern: '.*'
          retentions: '1m:1d'

Retention syntax is Whisper’s: precision:retention pairs, where each part is a number with an optional unit (s, m for minutes, h, d, w, y), and a bare number in the second position is a point count. 1m:7d is a one-minute step kept for seven days.

This is a seed and an override, never the sole source of truth. A static match yields configured confidence and schedules a confirming probe; when the probe disagrees with the configuration, the probe wins and the ladder is relearned. That is deliberate: it means a stale static_retentions block degrades to a few extra probes rather than to incorrect data.

Use it to skip the warmup cost on a known-stable deployment, not as a substitute for learning.

Changing static_retentions invalidates the persisted registry on the next start, so a corrected block takes effect immediately rather than waiting out the TTL.

What is accelerated

A target is delta-cached only if every function in it satisfies two independent properties:

1. Its step is predictable — the function does not change the resolution in a way Trickster cannot compute.
2. It is range-decomposable — each output point is a function of the input points at that same timestamp, so the values for [t1,t2] are identical whether that window was fetched alone or as part of a wider one.

The second property is what delta caching actually requires, and it is stricter than it first appears. The v1 allowlist is deliberately small:

Cross-series aggregation — one output point per input timestamp: sumSeries (sum), averageSeries (avg), minSeries, maxSeries, diffSeries, multiplySeries, divideSeries, divideSeriesLists, stddevSeries, rangeOfSeries, countSeries, percentileOfSeries, aggregate, aggregateWithWildcards, aggregateSeriesLists, group, groupByNode, groupByNodes, asPercent (pct), weightedAverage, powSeries, unique, and the *SeriesLists and *WithWildcards variants.

Per-point transforms: scale, scaleToSeconds, offset, add, pow, exp, absolute, invert, squareRoot, sigmoid, logit, log, round, transformNull, isNonNull, removeAboveValue, removeBelowValue, consolidateBy, setXFilesFactor (xFilesFactor).

Naming, name-based filtering and cosmetics: alias, aliasSub, aliasByNode, aliasByMetric, aliasByTags, upper, lower, substr, exclude, grep, color, alpha, lineWidth, dashed, drawAsInfinite, secondYAxis, stacked, areaBetween.

Bare metric paths, wildcards (*, ?, [a-z], {a,b}) and pipe syntax are all accelerated.

What falls back, and why

Anything not on the list above is served through the object cache. The reason is recorded on trickster_graphite_fallbacks_total:

Notable functions that are not accelerated, and why:

• Time-windowed: movingAverage, movingSum, movingMin, movingMax, movingMedian, exponentialMovingAverage, derivative, nonNegativeDerivative, perSecond, integral, stdev, holtWinters*, linearRegression. Each output point depends on input points before it, so a cached range cannot be reused at its edges.
• Whole-range selecting or ranking: highest*, lowest*, sortBy*, limit, mostDeviant, *Percentile, currentAbove/Below, averageAbove/Below, maximumAbove/Below, filterSeries. Which series come back depends on the entire requested range.
• Bucketing: summarize, hitcount, smartSummarize. These look decomposable — with alignToFrom=false, summarize’s buckets align to absolute interval boundaries — but the bucket covering either edge of the requested window is summarized from only the points inside that window. Measured on graphite-web 1.1.10, the same absolute one-hour bucket reported 134885.899, 302921.880 and 307388.038 over three different windows. Caching such a value and reusing it for a different window would return data the origin never produced, so these take the object lane.
• Time-shifting: timeShift, timeStack, timeSlice, delay.
• Generators and tag queries: constantLine, threshold, timeFunction, randomWalk, sinFunction, identity, seriesByTag, groupByTags, events, and template().

The object lane is still a cache: the whole response is cached by request URL with a TTL, keyed on every render parameter. It is always correct because it makes no claim about the response’s internal structure.

Multiple targets

Grafana sends every target of a panel in one /render call. Trickster splits a multi-target request into one delta-cached fetch per target and merges the results in target order, so targets on different ladders are still accelerated. If any single target cannot be accelerated, the whole request is served unaccelerated, because consolidation at the origin spans all series in the response and must be applied consistently.

maxDataPoints and consolidation

Grafana sends maxDataPoints on every panel, sized to the panel’s pixel width. Two panels of different widths showing the same query would otherwise be two different cache entries.

Trickster strips maxDataPoints from upstream requests, caches at the origin’s native resolution, and applies consolidation when rendering the response to each client. One cached series therefore serves every panel width, every output format, and noNullPoints, jsonp, pretty and tz variations.

The consolidation reproduces graphite-web’s renderViewJson exactly, including its start “nudge”, its treatment of xFilesFactor, the consolidateBy function, and the maxDataPoints=1 special case — verified byte-for-byte against a live origin.

Set passthrough_max_data_points: true if you need the origin’s own consolidation byte-for-byte instead. Requests carrying maxDataPoints are then served unaccelerated, which for Grafana means most requests, so this largely disables acceleration. It exists for operators who must guarantee identical bytes to a pre-existing consumer.

Sizing

Two of the common backend settings default differently for Graphite, because Graphite is fetched at its native resolution: maxDataPoints is stripped upstream (see maxDataPoints), so Trickster buffers and caches every point Whisper holds for the window, not the few hundred a dashboard draws. The generic defaults are sized for origins that consolidate before responding.

The Graphite values come from pkg/backends/graphite/options/defaults.go and are applied only where the configuration is silent, so setting either one in the file always wins.

max_object_size_bytes

The delta cache must buffer an entire response to model it. At native resolution a wide window on a fine archive is large: in the developer environment, a 120-day panel over two series on a 5-minute archive is 34,560 points per series and about 1.6 MB of JSON — three times the generic 512 KB limit.

A response over the limit cannot be delta-cached. Trickster logs upstream response exceeded MaxObjectSizeBytes and serves the request through the object lane instead, which streams rather than buffers — so clients still get their data, but that panel is never accelerated. Raise the limit if you serve panels wider than the 64 MB default holds:

    max_object_size_bytes: 134217728   # 128MB

timeseries_retention_factor

The cache keeps this many points per series, cropping older ones. The generic default of 1024 points is only about 3.5 days at a 5-minute step, so a 90-day dashboard panel would have almost its entire range cropped after each fetch and refetched on the next — a partial hit that re-fetches nearly everything. The Graphite default of 524288 covers 5 years at a 5-minute step, or 60 days at a 10-second one.

Raise it if you serve a wider window at a finer step than that, and lower it to cap what one series may occupy:

    timeseries_retention_factor: 1048576

Cache storage

Trickster does not roll up between rungs: a 10-second cached chunk is never reused to answer a query that resolves to 60 seconds, because the rollup Trickster computed would not match what Whisper’s lower archive holds (which depends on the metric’s aggregationMethod and xFilesFactor). A different step is a different cache key.

The practical implication is that a metric is cached once per rung it is queried at. A dashboard with a 1-hour, a 24-hour and a 30-day view of the same metric holds three cached series for it, not one. Size the cache for the distinct (metric, step) pairs your dashboards produce, not for the number of distinct metrics.

Metrics, logs, and tracing

Graphite-specific metrics, in addition to the standard trickster_proxy_requests_total{provider="graphite"} and trickster_cache_* families:

No metric path, target expression or query text ever appears in a label; every label value comes from a closed set. See metrics.md.

Logs: a Debug line per resolution decision carrying the target, a bucketed age, the lane, confidence, source and step (or reason when declined); Warn on every misprediction and on every negative-cache entry.

Traces: GraphiteExpand (a child of the request span), GraphiteProbe (kind, result, step) and GraphiteLearnLadder (ladder, probe count).

Operations and troubleshooting

Everything is falling back

Check trickster_graphite_fallbacks_total by reason.

• unknown_step dominating right after a start is normal; it should fall as ladders are learned. If it does not, resolution is failing — look for graphite ladder learning failed; negative-cached warnings.
• function_not_allowlisted means the dashboard uses functions outside the allowlist. That is expected and correct, not a defect.
• non_series_format means the client is asking for images; those panels cannot be delta-cached.

Probing never quiets down

trickster_graphite_probes_total should approach zero after warmup and trickster_graphite_ladders should flatten. If probing continues:

• Metric names may be high-cardinality or ephemeral, so new leaves appear constantly. Raise resolution_registry.max_entries, or accept the cost.
• The registry may be evicting under max_entries pressure and relearning. Compare trickster_graphite_registry_entries{layer="leaf"} against the configured maximum.
• Learning may be failing and retrying under backoff. Check the warnings.

Step mispredictions are non-zero

Something changed a metric’s on-disk ladder — usually whisper-resize.py, a schema edit followed by new file creation, or a metric that moved between schema patterns. Trickster recovers on its own (it relearns and re-serves the request unaccelerated), so this is not a client-visible error, but the counter should return to flat once the new ladders are learned. Persistent mispredictions on one namespace suggest an origin whose resolution is not a function of now - from at all — a clustered graphite-web fronting stores with different schemas, for example.

A panel is never accelerated but should be

• Look for upstream response exceeded MaxObjectSizeBytes in the logs and raise max_object_size_bytes.
• Check whether the panel’s window is wholly beyond the metric’s maxRetention; those requests have nothing to cache.
• A single non-allowlisted target in a multi-target panel makes the whole panel unaccelerated.

Repeated partial hits on wide panels

Raise timeseries_retention_factor. A partial hit that re-fetches nearly the whole range on every request is the signature of cropping.

Verifying correctness against the origin

Because the object lane and the delta lane must both be byte-identical to the origin, the simplest check is to configure two data sources in Grafana — one direct to Graphite, one through Trickster — and compare panels. The developer environment (docs/developer/environment/) is set up this way, with a dashboard whose panels deliberately exercise archive boundaries, the retention edge, schema drift, non-allowlisted functions and mixed-ladder multi-target requests.

Known gaps

• Fast Forward is not implemented. Graphite’s coarsest-rung behavior and its until > now clamp make the “one extra instant datapoint” approach ill-defined. fast_forward_disable is effectively always on.
• No ALB / Time Series Merge support. Merging across replicas requires the replicas’ ladders to agree, an invariant not yet established. Graphite backends can still be members of non-TSM ALB mechanisms.
• Non-decomposable functions are not accelerated. A future release may evaluate them inside Trickster over cached native-resolution leaves, which would move movingAverage, derivative, summarize, highest* and friends into the accelerated path.
• No cross-resolution rollup, as described under Cache storage.
• Tag-based queries (seriesByTag, groupByTags) are not accelerated. They are proxied and object-cached.
• Non-ASCII metric names (graphite-web’s UTF8_METRICS) are not supported by the target parser; such targets fail to parse and take the object lane.

5 - MySQL Provider

Trickster can accept native MySQL client connections, proxy each authenticated session to a MySQL origin, and cache eligible text-protocol query results. This guide describes the initial supported contract. Behavior not listed here is rejected or proxied without caching; cache eligibility always fails closed.

Compatibility

The supported matrix is:

• Oracle MySQL 8.4 LTS through 9.7 LTS;
• Grafana 11.0 and above using Grafana’s built-in MySQL data source; and
• native clients that use mysql_native_password and the supported commands below.

MariaDB, Percona Server, Aurora MySQL, other MySQL-compatible products, older Grafana releases, and third-party Grafana MySQL data sources are not part of the initial compatibility claim, but should work without problems. File an issue if you discover a compatibility issue.

The developer environment pins MySQL 8.4 and Grafana 13.

Direct backend configuration

The listener accepts the MySQL wire protocol on one TCP port. MySQL negotiates TLS in-band, so do not configure an HTTP-style tls_port.

listeners:
  mysql-native:
    protocol: mysql
    address: ""
    port: 8486
    connections_limit: 200
    mysql:
      handshake_timeout: 10s
      read_timeout: 30s
      write_timeout: 30s
      idle_timeout: 5m
      max_packet_size_bytes: 16777215
      max_query_size_bytes: 1048576

authenticators:
  mysql-clients:
    provider: basic
    users:
      grafana_reader: ${GRAFANA_MYSQL_PASSWORD}

caches:
  mysql-cache:
    provider: memory
    memory:
      max_size_bytes: 536870912

backends:
  mysql-primary:
    provider: mysql
    listener_names: [mysql-native]
    authenticator_name: mysql-clients
    origin_url: mysql://trickster_ro:REDACTED@mysql.example:3306/analytics
    cache_name: mysql-cache
    timeout: 30s
    max_concurrent_conns: 200
    max_object_size_bytes: 8388608
    mysql:
      max_result_rows: 100000
      max_result_size_bytes: 67108864
    healthcheck:
      interval: 5s
      timeout: 3s
      failure_threshold: 3
      recovery_threshold: 2

Exactly one direct MySQL backend or one supported MySQL User Router ALB maps to a MySQL listener. The origin URL must use the mysql scheme and include an origin username. Percent-encode reserved username, password, and database characters. Configuration stringification and the sanitized management configuration redact an embedded origin password, but the source configuration must still be protected as a secret.

The authenticator is the only downstream credential source. Trickster terminates downstream authentication; it does not pass client credentials to the origin or silently fall back to origin credentials. Native-password verification requires the original password, so MySQL authenticator entries must be plaintext values, including environment-expanded values or a protected CSV users file. Htpasswd hashes are rejected for this listener. Do not expose the raw configuration management endpoint outside a trusted administrative network.

TLS

Downstream TLS

Add a server certificate and key to the listener-facing backend:

backends:
  mysql-primary:
    # ...direct backend settings above...
    require_tls: true
    tls:
      full_chain_cert_path: /etc/trickster/tls/server.crt
      private_key_path: /etc/trickster/tls/server.key

Without a server pair, downstream TLS is disabled. With the pair, TLS is optional unless require_tls: true rejects plaintext clients. TLS 1.2 is the minimum. Downstream client certificates are not supported in the initial release.

Upstream TLS

The upstream modes are:

• no upstream TLS fields: plaintext;
• insecure_skip_verify: true: encryption without server verification;
• one certificate_authority_paths entry: CA and hostname verification; and
• a complete client_cert_path/client_key_path pair: mutual TLS, combined with verified or explicitly unverified server mode.

backends:
  mysql-primary:
    # ...direct backend settings above...
    tls:
      certificate_authority_paths:
        - /etc/trickster/tls/origin-ca.crt
      client_cert_path: /etc/trickster/tls/origin-client.crt
      client_key_path: /etc/trickster/tls/origin-client.key

Certificate/key settings must be complete pairs. More than one upstream CA path, require_tls without a downstream server pair, or an incomplete pair is a configuration error. There is no CA-verification-without-hostname mode.

Connections, limits, and lifecycle

Each authenticated downstream connection owns exactly one upstream connection. There is no pooling or multiplexing, which prevents transaction, database, and session state from leaking between clients. Size connections_limit and max_concurrent_conns together and leave capacity for health probes. A useful starting point is one admitted origin connection per downstream connection, plus normal operational headroom.

Listener limits protect the downstream boundary:

• connections_limit limits concurrent client connections (0 is unlimited);
• handshake, command-read, response-write, and idle timeouts bound each phase;
• max_packet_size_bytes is fixed at 16 MiB minus one; and
• max_query_size_bytes limits SQL text within that packet ceiling.

Backend limits protect the origin and cache:

• timeout bounds connect and query work;
• max_concurrent_conns bounds upstream admission;
• mysql.max_result_rows and mysql.max_result_size_bytes bound results; and
• max_object_size_bytes skips caching an otherwise valid oversized object.

Handshake, packet, query, read, write, and idle failures close the client connection. Origin timeout or concurrency rejection closes the affected upstream connection and returns a bounded MySQL error. A row/byte overflow closes both sides because a partial result may already have been emitted.

On shutdown, Trickster stops accepting new sessions and drains existing ones within the configured management drain window. A reload that changes MySQL authentication, listener transport, certificates, origin transport, routing, or relevant limits restarts and drains the listener; an established session is never silently moved to new credentials, TLS policy, or a different origin.

Protocol behavior

Supported commands are:

• COM_QUERY containing one text-protocol statement;
• COM_INIT_DB;
• COM_PING;
• COM_QUIT; and
• COM_RESET_CONNECTION.

Reset discards the upstream connection and tracked session state. The next command opens a fresh origin connection.

The initial release rejects binary prepared statements; SQL PREPARE, EXECUTE, and DEALLOCATE; server cursors; binary results; compression; LOCAL INFILE and all LOAD statements; multi-statements; stored-procedure CALL; HELP, XA, HANDLER, and CACHE INDEX; executable version comments; unclassified text response shapes; COM_CHANGE_USER; replica registration; and binlog commands. Connection attributes may be syntactically accepted but are ignored for authentication, routing, cache identity, and query semantics. Malformed, oversized, timed-out, or partially streamed commands close the connection. A fully consumed unsupported command normally leaves it usable.

Successful DML and DDL are proxied and never cached. They do not invalidate existing OPC or DPC entries, so another session may see cached data until TTL expiry or eviction. Use short TTLs or purge affected cache data when immediate read-after-write visibility across sessions is required.

Cache classification

Trickster uses three outcomes:

• Delta Proxy Cache (DPC): a supported deterministic aggregate time-series SELECT with one unambiguous time output, literal positive cadence, safe result shape and ordering, and an inclusive-lower/exclusive-upper raw-time predicate;
• Object Proxy Cache (OPC): a deterministic single-result SELECT that is safe to cache as one object but is not proven reusable by time extent; and
• proxy-only: mutations, transactions, unsupported or unsafe session state, non-deterministic or rejected shapes, and any query whose safety cannot be established.

DPC supports the tested zero-phase integer/cast and FLOOR(UNIX_TIMESTAMP(column) / n) * n bucket forms when n is the same positive integer literal on both sides. Dynamic, mismatched, non-positive, overflowing, ambiguous, or non-zero-phase forms fail closed. Qualified and backtick-quoted identifiers, multiple numeric values, and string dimensions are supported only in the shapes recorded by the compatibility corpus.

The authenticated downstream username, selected terminal backend, normalized database, supported time-zone literal, and query identity participate in cache keys. Cache entries are never shared across authenticated users merely because their SQL is identical.

Grafana macros and exact SQL shapes

Only patterns represented by the versioned compatibility corpus are published as supported. The corpus covers $__time, $__timeEpoch, $__timeFilter, $__timeFrom, $__timeTo, $__timeGroup, $__timeGroupAlias, $__unixEpochFilter, $__unixEpochFrom, $__unixEpochTo, $__unixEpochGroup, and $__unixEpochGroupAlias. Its documented minimum $__interval is one minute.

Grafana’s normal inclusive $__timeFilter expansion is OPC:

SELECT
  CAST(CAST(UNIX_TIMESTAMP(ts)/(300) AS SIGNED)*300 AS SIGNED) AS time,
  COUNT(*) AS samples
FROM telemetry
WHERE ts BETWEEN FROM_UNIXTIME(1785542400)
             AND FROM_UNIXTIME(1785628800)
GROUP BY time
ORDER BY time

Compose $__timeFrom() and $__timeTo() into a half-open predicate for DPC:

SELECT
  CAST(CAST(UNIX_TIMESTAMP(ts)/(60) AS SIGNED)*60 AS SIGNED) AS time,
  AVG(value) AS mean_value
FROM telemetry
WHERE site = 'denver'
  AND ts >= FROM_UNIXTIME(1785542400)
  AND ts < FROM_UNIXTIME(1785628800)
GROUP BY time
ORDER BY time

The epoch-second equivalent is:

SELECT
  CAST(CAST(epoch_seconds/(300) AS SIGNED)*300 AS SIGNED) AS time,
  SUM(requests) AS requests
FROM service_rollups
WHERE region = 'us-west'
  AND epoch_seconds >= 1785542400
  AND epoch_seconds < 1785628800
GROUP BY time
ORDER BY time

Trickster normalizes the lower bound up and the exclusive upper bound down to the cadence and caches only complete buckets. A range with no complete bucket normalizes to an empty range. Inclusive upper bounds and Grafana’s strict-lower $__unixEpochFilter expansion remain OPC because they do not prove the same complete-bucket semantics. Native DATETIME/TIMESTAMP, epoch-second integer, and the corpus’s epoch-nanosecond adaptation are supported in their recorded shapes.

Session state

The only cache-safe tracked state is the selected default database and one session-scoped literal SET time_zone = '<value>'. USE and COM_INIT_DB update the database portion of cache identity.

Transactions and savepoints bypass cache while active. Other SET forms, character set or collation changes, sql_mode, lc_time_names, user variables, temporary objects, locks, session-local functions, mutations, and unclassified state changes make that connection cache-unsafe. Reconnect or a successful COM_RESET_CONNECTION is required to restore the configured baseline.

Protocol-aware User Router

A native MySQL User Router selects one direct terminal MySQL backend from the verified downstream username:

listeners:
  mysql-routed:
    protocol: mysql
    port: 8486
    connections_limit: 200

authenticators:
  tenant-clients:
    provider: basic
    users:
      tenant_a_reader: ${TENANT_A_CLIENT_PASSWORD}
      tenant_b_reader: ${TENANT_B_CLIENT_PASSWORD}

caches:
  tenant-a-cache:
    provider: memory
  tenant-b-cache:
    provider: filesystem

backends:
  tenant-a-mysql:
    provider: mysql
    authenticator_name: tenant-clients
    origin_url: mysql://tenant_a_ro:REDACTED@mysql-a.example:3306/analytics
    cache_name: tenant-a-cache
    healthcheck:
      interval: 5s
      timeout: 3s

  tenant-b-mysql:
    provider: mysql
    authenticator_name: tenant-clients
    origin_url: mysql://tenant_b_ro:REDACTED@mysql-b.example:3306/analytics
    cache_name: tenant-b-cache
    healthcheck:
      interval: 5s
      timeout: 3s

  mysql-by-tenant:
    provider: alb
    listener_names: [mysql-routed]
    authenticator_name: tenant-clients
    alb:
      mechanism: ur
      user_router:
        default_backend: tenant-a-mysql
        users:
          tenant_a_reader:
            to_backend: tenant-a-mysql
          tenant_b_reader:
            to_backend: tenant-b-mysql

The listener-facing router owns the one downstream authentication exchange, admission, and TLS. Direct terminal MySQL backend entries must reference the same named authenticator to satisfy backend validation, but do not perform a second client exchange. The terminal owns its origin credentials, upstream TLS, cache, health, limits, and query policy. Routing occurs once before opening the upstream connection and remains sticky for the whole downstream session. Transactions, database changes, and session state never trigger rerouting.

Only direct MySQL terminal backends are supported. Nested ALBs, Rules, cycles, mixed terminal providers, empty routes, and MySQL to_user/to_credential remapping are configuration errors. A router requires an authenticator. An unmapped user uses default_backend when configured; without one, Trickster returns a bounded native MySQL no-route error. HTTP no_route_status_code is not sent on the native connection. A selected unhealthy or unavailable terminal fails as a MySQL availability error; the session is not silently routed elsewhere.

The verified username and selected terminal remain in cache identity. Route metrics use configured router/backend names and bounded outcomes, never the username.

Metrics, logs, and health

Important metrics include:

• trickster_sql_query_analysis_total{backend_name,dialect,cache_mode,reason};
• trickster_sql_query_rewrite_failures_total{backend_name,dialect,reason};
• trickster_sql_query_cache_total{backend_name,dialect,cache_mode,cache_status};
• trickster_proxy_request_duration_seconds and trickster_proxy_points_total with provider="mysql";
• trickster_mysql_connections_total, trickster_mysql_active_connections, trickster_mysql_errors_total, trickster_mysql_route_selections_total, and trickster_mysql_command_duration_seconds; and
• cache operation, usage, limit, and event metrics labeled by cache_name.

Example PromQL, replacing names to match the deployment:

sum by (cache_mode, reason) (rate(trickster_sql_query_analysis_total{backend_name="mysql1"}[5m]))
sum by (reason) (rate(trickster_sql_query_rewrite_failures_total{backend_name="mysql1"}[5m]))
sum by (cache_mode, cache_status) (rate(trickster_sql_query_cache_total{backend_name="mysql1"}[5m]))
histogram_quantile(0.95, sum by (le) (rate(trickster_proxy_request_duration_seconds_bucket{backend_name="mysql1",provider="mysql"}[5m])))
sum by (cache_status) (rate(trickster_proxy_points_total{backend_name="mysql1",provider="mysql"}[5m]))
sum by (operation, status) (rate(trickster_cache_operation_objects_total{cache_name="mysql-cache"}[5m]))
trickster_cache_usage_bytes{cache_name="mysql-cache"}
sum by (reason) (rate(trickster_cache_events_total{cache_name="mysql-cache",event="eviction"}[5m]))

Analysis logs contain bounded backend, cache mode, reason, and statement-type fields. Rewrite and protocol failures use bounded categories; SQL text, passwords, and usernames are not metric labels. Keep debug logging temporary in production because classification logs can be high volume.

Native health checks open a bounded connection with the backend’s configured origin credentials and TLS policy, then issue COM_PING. They use the common interval, timeout, failure, and recovery thresholds. HTTP health request fields do not apply. Health output reports sanitized authentication, TLS, timeout, refused-connection, and server-error categories. See Health Checks for management endpoint details.

Kubernetes deployment

The base examples expose the optional mysql Service/container port on 8486. The safe routed example keeps the complete credential-bearing configuration in deploy/kube/mysql-user-router-secret.yaml and replaces the base ConfigMap volume with deploy/kube/mysql-deployment-patch.yaml. Replace every placeholder and use an encrypted/external secret controller in production. Do not put an origin DSN, downstream passwords, private keys, or client keys in deploy/kube/configmap.yaml.

For direct TLS or mTLS, add the server, CA, and client certificate material to a Kubernetes TLS or opaque Secret, mount it read-only, and reference the mount paths from the backend tls block. Rotating that Secret requires the configuration/certificate reload path described above; established sessions are drained rather than changing transport identity in place.

Operations and troubleshooting

Kubernetes readiness

The native health scheduler does not delay listener startup or make the general health endpoint return a non-200 status. /trickster/ping proves only that the process is running, and a TCP readiness probe on port 8486 proves only that the native listener is accepting connections. The companion manifest uses that TCP probe. Treat the scheduled MySQL status published in /trickster/health?json as the origin-readiness signal in an external controller or monitoring rule; do not assume the endpoint’s HTTP status reflects origin health. Allow at least interval * failure_threshold or interval * recovery_threshold for a stable transition. User Router terminals report separately, and a failing selected terminal returns a native availability error rather than rerouting the session.

Capacity planning

Start from peak concurrent dashboard/client sessions. Reserve approximately one origin connection per admitted client, then add health and rollout headroom. Bound result rows/bytes below the pod or process memory budget and set max_object_size_bytes below both the result-byte limit and a practical fraction of cache capacity. For an in-memory cache, budget retained objects, index overhead, concurrent result encoding, and normal process memory. Use finite query, command, and idle timeouts so abandoned clients cannot retain connections indefinitely.

Repeated misses or proxy-only outcomes

1. Group trickster_sql_query_analysis_total by cache_mode and reason.
2. Confirm the query is deterministic, single-result, and outside a transaction or unsafe session.
3. For DPC, inspect the expanded SQL: require a literal cadence and >= lower, < upper raw-time predicates.
4. Confirm the requested interval contains at least one complete cadence bucket and that cache TTL, backfill tolerance, and retention are suitable.
5. Check username, selected backend, database, and time zone; these intentionally isolate keys.
6. Inspect cache operation status and eviction metrics for admission failures or churn.

Authentication and TLS

• Authentication failure: verify the named authenticator, plaintext/CSV credential source, exact username, supported native-password plugin, and filesystem permissions on a mounted users file.
• Downstream TLS failure: verify the server pair, require_tls, client TLS mode, hostname, and TLS 1.2+ support. Do not use tls_port.
• Upstream TLS failure: verify the single CA path, hostname in the origin URL, complete client pair when using mTLS, file permissions, and whether insecure_skip_verify was intentionally selected.

Origin and cache failures

During an origin outage, health transitions and MySQL error metrics should rise; existing safe cache hits may continue, while misses fail. Restore origin health before increasing connection limits. A cache-provider failure should surface in cache operation/event metrics and logs; query handling fails closed to the provider’s normal proxy behavior rather than treating an unknown cache state as a hit.

Certificate rotation and relevant configuration reloads drain affected native listeners. Monitor active connections, authentication/TLS errors, route selection outcomes, health state, rewrite failures, cache status, and p95 latency until the old sessions have drained.

Rollout and rollback

Canary the listener/backend configuration with a limited client population. Compare against the pre-release baseline and roll back for a sustained increase of 5 percentage points in proxy/origin/protocol errors, 10 percentage points in proxy-only classifications, 1 percentage point in rewrite failures, or 20% in p95 proxy latency. Roll back by restoring the previous configuration and binary/image on the release branch, reloading or restarting Trickster, and allowing the old MySQL listener to drain. Do not move an established session between versions. Preserve metrics and sanitized logs for the incident record.

Environment variables and reload behavior

The global TRK_ORIGIN_URL and TRK_ORIGIN_TYPE variables map only to the default backend and are suitable only for a basic single-backend deployment. There are no one-to-one TRK_* variables for MySQL listener limits, backend limits, TLS, health, or User Router options; configure those in YAML. Authenticator users values support ${VARIABLE} expansion, and supported users files and secret volume mounts are preferred for production. The origin_url field does not expand ${VARIABLE} in structured YAML; store a configuration containing an embedded DSN as a protected secret, not a public ConfigMap.

Validate before rollout:

trickster -validate-config -config /etc/trickster/trickster.yaml

SIGHUP and the management reload endpoint reload a changed file. Authentication, TLS, transport, routing, and terminal runtime changes restart and drain the affected MySQL listener. Cache policy changes apply to new work; existing cache objects remain subject to their stored TTL and normal eviction.

Known compatibility gaps

The initial release does not claim support for:

• MySQL-compatible server products outside Oracle MySQL 8.4-9.7;
• downstream plugins other than mysql_native_password or downstream mTLS;
• prepared/binary protocols, compression, local infile, multi-results, stored procedures, binlog/replication commands, or connection-attribute semantics;
• general SQL parsing beyond the Vitess-supported and compatibility-corpus shapes;
• window functions, grouping sets, rollups, ambiguous time axes or buckets, unsafe boolean predicates, non-deterministic outputs, dynamic/non-zero-phase buckets, or inclusive-range DPC;
• arbitrary session-state caching or cross-session mutation invalidation; and
• nested/mixed MySQL User Router topologies or origin credential remapping.

The executable SQL contract is pkg/backends/mysql/testdata/compatibility/v1.json. The maintainer-facing frozen release matrix is in docs/developer/mysql-release-contract.md.