Trickster Metrics

Trickster exposes a Prometheus /metrics endpoint with a customizable listener port number (default is 8481). For more information on customizing the metrics configuration, see configuring.md.


The following metrics are available for polling with any Trickster configuration:

  • trickster_build_info (Gauge) - This gauge is always 1 when Trickster is running

    • labels:
      • goversion - the version of go under which the running Trickster binary was built
      • revision - the commit ID on which the running Trickster binary was built
      • version - semantic version of the running Trickster binary
  • trickster_config_last_reload_successful (Gauge) - The value is 1 when true (the last config reload was successful) or 0 when false

  • trickster_config_last_reload_success_time_seconds (Gauge) - Epoch timestamp of the last successful configuration reload

  • trickster_frontend_requests_total (Counter) - Count of front end requests handled by Trickster

    • labels:
      • backend_name - the name of the configured backend handling the proxy request
      • provider - the type of the configured backend handling the proxy request
      • method - the HTTP Method of the proxied request
      • http_status - The HTTP response code provided by the backend
      • path - the Path portion of the requested URL
  • trickster_frontend_requests_duration_seconds (Histogram) - Histogram of front end request durations handled by Trickster. A CONNECT, whose duration is that of the tunnel or WebTransport session it opens, is not observed

    • labels:
      • backend_name - the name of the configured backend handling the proxy request
      • provider - the type of the configured backend handling the proxy request
      • method - the HTTP Method of the proxied request
      • http_status - The HTTP response code provided by the backend
      • path - the Path portion of the requested URL
  • trickster_frontend_written_byte_total (Counter) - Count of bytes written in front end requests handled by Trickster

    • labels:
      • backend_name - the name of the configured backend handling the proxy request
      • provider - the type of the configured backend handling the proxy request
      • method - the HTTP Method of the proxied request
      • http_status - The HTTP response code provided by the backend
      • path - the Path portion of the requested URL
  • trickster_proxy_requests_total (Counter) - The total number of requests Trickster has handled.

    • labels:
      • backend_name - the name of the configured backend handling the proxy request
      • provider - the type of the configured backend handling the proxy request
      • method - the HTTP Method of the proxied request
      • cache_status - status codes are described here
      • http_status - The HTTP response code provided by the backend
      • path - the Path portion of the requested URL
  • trickster_proxy_upstream_retries_total (Counter) - The number of upstream requests retried under a path’s retry policy.

    • labels:
      • backend_name - the name of the configured backend handling the proxy request
      • path - the configured path whose policy retried the request
  • trickster_proxy_mirror_requests_total (Counter) - The number of requests copied to a path’s mirror backend.

    • labels:
      • backend_name - the name of the configured backend handling the proxy request
      • mirror_backend - the backend receiving the copies
      • result - sent, or dropped when the mirror’s in-flight bound was reached
  • trickster_proxy_sigv4_events_total (Counter) - The number of SigV4 signing events on backends with a sigv4 block.

    • labels:
      • backend_name - the name of the configured backend whose request was signed
      • event - sign_failure when a request could not be signed and was not sent, or credentials_retry when the origin rejected expiring credentials and the request was resent with refreshed ones
  • trickster_proxy_origin_srv_lookups_total (Counter) - The number of SRV lookups made by backends using SRV origin resolution.

    • labels:
      • backend_name - the name of the configured backend that dialed its origin
      • result - hit, miss, stale (a failed refresh served the last good answer), negative (a cached NXDOMAIN, empty answer or failure) or error
  • trickster_proxy_origin_srv_dial_attempts_total (Counter) - The number of dial attempts to SRV targets by backends using SRV origin resolution.

    • labels:
      • backend_name - the name of the configured backend that dialed its origin
      • tier - the target’s priority tier, where 0 is the preferred tier
      • result - success or failure
  • trickster_proxy_truncated_responses_total (Counter) - The number of time series fetches the origin truncated at its series limit, which Trickster proxies rather than caches. Only backends whose flavor declares a series limit report it.

    • labels:
      • backend_name - the name of the configured backend whose fetch was truncated
  • trickster_accesslog_dropped_lines_total (Counter) - The number of access and error log lines dropped because the log could not accept them.

    • labels:
      • backend_name - the name of the configured backend whose logger dropped the line
      • log - access or error
  • trickster_ip_acl_decisions_total (Counter) - Decisions made by an attached IP access list. See ip-acl.md.

    • labels:
      • ip_acl - the name of the access list
      • scope - listener, backend, or path
      • verdict - allow or deny. reject and drop are both deny
  • trickster_ratelimit_decisions_total (Counter) - Decisions made by an attached rate limiter. See rate-limiting.md.

    • labels:
      • limiter - the name of the rate limiter
      • plane - http, stream or moq
      • result - allowed, limited, counted, exempt, or full
  • trickster_ratelimit_keys (Gauge) - Buckets a rate limiter holds at scrape time.

    • labels:
      • limiter - the name of the rate limiter
  • trickster_proxy_points_total (Counter) - The total number of data points Trickster has handled.

    • labels:
      • backend_name - the name of the configured backend handling the proxy request
      • provider - the type of the configured backend handling the proxy request
      • cache_status - status codes are described here
      • path - the Path portion of the requested URL
  • trickster_proxy_timeseries_offgrid_extents_total (Counter) - The number of time series fetch ranges whose bounds fell between buckets and were narrowed to whole buckets before fetching. A non-zero value points to a step or phase mismatch between Trickster and the origin; please report it.

    • labels:
      • backend_name - the name of the configured backend fetching the range
      • provider - the type of the configured backend, or the native protocol (e.g., mysql)
  • trickster_proxy_partial_bucket_fetches_total (Counter) - The number of partial bucket fetches through the object cache, under a partial step alignment mode.

    • labels:
      • backend_name - the name of the configured backend
      • provider - the backend provider
      • edge - start or end: the edge of the requested range the bucket sits on
      • status - the object cache result, such as hit or kmiss, or err when the fetch failed and the bucket was left out of the response
  • trickster_step_alignment_fallbacks_total (Counter) - The number of requests for a step alignment mode the query doesn’t support, served in the query’s default mode instead.

    • labels:
      • backend_name - the name of the configured backend
      • requested - the mode asked for, by the backend’s configuration, an ALB, or a trickster-step-align directive
      • applied - the mode the request was served in
  • trickster_geo_acl_decisions_total (Counter) - The number of clients a geo ACL judged.

    • labels:
      • geo_acl - the name of the geo ACL
      • plane - http, native or stream: the kind of listener the client was judged on
      • verdict - allow, deny, count (denied, but the geo ACL only counts) or exempt (allowed with no lookup), as trickster_ip_acl_decisions_total names its verdicts
  • trickster_geo_locator_lookups_total (Counter) - The number of lookups a geo locator answered. A rising rate of not_found can mean the listener’s trusted_proxies is wrong, so every client resolves to a proxy’s private address.

    • labels:
      • geo_locator - the name of the geo locator
      • result - found, not_found or error
  • trickster_geo_locator_reloads_total (Counter) - The number of replaced data files a geo locator loaded or refused, keeping the last good one.

    • labels:
      • geo_locator - the name of the geo locator
      • result - success or error
  • trickster_geo_locator_build_timestamp_seconds (Gauge) - When the database an mmdb geo locator has loaded was built, in seconds since the epoch.

    • labels:
      • geo_locator - the name of the geo locator
  • trickster_proxy_request_duration_seconds (Histogram) - Time required to proxy a given Prometheus query.

    • labels:
      • backend_name - the name of the configured backend handling the proxy request
      • provider - the type of the configured backend handling the proxy request
      • method - the HTTP Method of the proxied request
      • cache_status - status codes are described here
      • http_status - The HTTP response code provided by the backend
      • path - the Path portion of the requested URL
  • trickster_proxy_max_connections (Gauge) - Trickster max number of allowed concurrent connections

  • trickster_proxy_active_connections (Gauge) - Trickster number of concurrent connections

  • trickster_proxy_requested_connections_total (Counter) - Trickster total number of connections requested by clients.

  • trickster_proxy_accepted_connections_total (Counter) - Trickster total number of accepted client connections.

  • trickster_proxy_closed_connections_total (Counter) - Trickster total number of administratively closed client connections.

  • trickster_proxy_failed_connections_total (Counter) - Trickster total number of failed client connections.

  • trickster_proxy_stream_connections_total (Counter) - The number of connections and UDP sessions accepted by tcp, tls and udp listeners.

    • labels:
      • listener_name - the name of the configured listener
      • protocol - tcp, tls or udp
      • result - proxied, or why the connection was closed instead: not_tls (a tls listener received no ClientHello), no_route (no backend routes the server name), no_upstream (the backend’s pool has no dialable member, or the member chosen refuses its share), dial_failed, refused (a udp listener at its session limit, or a connection arriving as the listener closes), or denied (turned away by the listener’s admission control before anything was relayed)
  • trickster_proxy_stream_active_connections (Gauge) - The number of connections and UDP sessions stream listeners are relaying.

    • labels:
      • listener_name - the name of the configured listener
      • protocol - tcp, tls or udp
  • trickster_proxy_stream_dropped_datagrams_total (Counter) - The number of datagrams udp listeners dropped rather than relayed.

    • labels:
      • listener_name - the name of the configured listener
      • reason - queue_full (the client’s flow, or every flow together, already held its allowance of datagrams waiting to be written), write_timeout (the write to the backend blocked for the whole write bound) or denied (turned away by the listener’s admission control)
  • trickster_proxy_stream_bytes_total (Counter) - The bytes relayed by stream listeners.

    • labels:
      • listener_name - the name of the configured listener
      • protocol - tcp, tls or udp
      • direction - in from the client to the backend, out from the backend to the client
  • trickster_proxy_stream_member_connections_total (Counter) - The number of connections and UDP sessions a stream listener committed to an ALB pool member.

    • labels:
      • listener_name - the name of the configured listener
      • protocol - tcp, tls or udp
      • backend_name - the name of the pool member backend
      • result - proxied, dial_failed (the member could not be connected to) or unreachable (a udp member answered a datagram with a port-unreachable)
  • trickster_proxy_stream_member_active_connections (Gauge) - The number of connections and UDP sessions open to an ALB pool member.

    • labels:
      • listener_name - the name of the configured listener
      • protocol - tcp, tls or udp
      • backend_name - the name of the pool member backend
  • trickster_proxy_stream_member_connect_duration_seconds (Histogram) - The time taken to connect to an ALB pool member.

    • labels:
      • listener_name - the name of the configured listener
      • protocol - tcp, tls or udp
      • backend_name - the name of the pool member backend
  • trickster_proxy_query_range_rejected_total (Counter) - Trickster total number of queries rejected due to exceeding the max_query_range limit.

    • labels:
      • backend - the name of the configured backend rejecting the query
  • trickster_graphite_resolution_lookups_total (Counter) - Count of Graphite step-resolution lookups. Labels never include a metric path or target expression.

    • labels:
      • backend_name - the name of the configured Graphite backend
      • confidence - how the step was established: exact (read from an origin response for this leaf set and age), derived (computed from known leaf ladders), configured (from static_retentions, not yet probe-confirmed), or unknown (no usable step; the request is served unaccelerated)
      • source - where it came from: registry, response, probe, static, function, or none
  • trickster_graphite_probes_total (Counter) - Count of synthetic requests issued to learn a metric’s archive ladder. Expect a spike at startup that collapses toward zero as ladders are learned.

    • labels:
      • backend_name - the name of the configured Graphite backend
      • kind - narrow (a one-second window that also discovers the retention edge), wide (what a real query at that age receives), or find (a /metrics/expand lookup)
      • result - step (a stepped series came back), empty (no series: beyond retention, or no such metric), or error
  • trickster_graphite_ladders (Gauge) - Number of distinct archive ladders known to the resolution registry. Ladders come from storage-schemas.conf patterns, so this should flatten at a small number.

    • labels:
      • backend_name - the name of the configured Graphite backend
  • trickster_graphite_registry_entries (Gauge) - Number of entries in each layer of the resolution registry.

    • labels:
      • backend_name - the name of the configured Graphite backend
      • layer - leaf (metric path to ladder), ladder (the ladders themselves), target (cached wildcard expansions), or negative (paths in resolution backoff)
  • trickster_graphite_step_mispredictions_total (Counter) - Count of origin responses whose step differed from the predicted step. This should always be zero. A non-zero value means a cached ladder was wrong; Trickster discards the prediction, relearns and re-serves the request unaccelerated, so clients still receive correct data.

    • labels:
      • backend_name - the name of the configured Graphite backend
  • trickster_graphite_fallbacks_total (Counter) - Count of render requests served without delta caching. Labels never include a target expression.

    • labels:
      • backend_name - the name of the configured Graphite backend
      • reason - parse_error, non_series_format, function_not_allowlisted, unknown_step, missing_target, mixed_steps, multi_target_step_mismatch, passthrough_max_data_points, misprediction, client_identity, tz_unavailable, or resolution_identity
  • trickster_sql_query_analysis_total (Counter) - Count of SQL query cache-eligibility classifications. Labels never include query text.

    • labels:
      • backend_name - the name of the configured backend analyzing the query
      • dialect - the SQL dialect of the analyzing backend (e.g., clickhouse)
      • cache_mode - the strongest cache mode supported by the query (delta, object, or none)
      • reason - the stable classification reason code (e.g., delta_cacheable, unsafe_predicate, unsupported_bucket)
  • trickster_sql_query_rewrite_failures_total (Counter) - Count of SQL cache-miss extent rewrite failures. Labels never include query text.

    • labels:
      • backend_name - the name of the configured backend rendering the query
      • dialect - the SQL dialect of the rendering backend
      • reason - the fixed internal failure category
  • trickster_druid_query_analysis_total (Counter) - Count of native Druid query cache-eligibility classifications. Labels never include query text or datasource names.

    • labels:
      • backend_name - the configured Druid backend
      • cache_mode - delta, object, or proxy
      • reason - the stable classification reason code
  • trickster_victoriametrics_query_analysis_total (Counter) - Count of MetricsQL API request cache-eligibility classifications. Labels never include query text.

    • labels:
      • backend_name - the configured VictoriaMetrics backend
      • cache_mode - delta, object, or proxy
      • reason - the stable classification reason code
  • trickster_druid_query_rewrite_failures_total (Counter) - Count of Druid cache-miss extent rewrite failures.

    • labels:
      • backend_name - the configured Druid backend
      • reason - the fixed internal failure category
  • trickster_cache_operation_objects_total (Counter) - The total number of objects upon which the Trickster cache has operated.

    • labels:
      • cache_name - the name of the configured cache performing the operation
      • provider - the type of the configured cache performing the operation
      • operation - the name of the operation being performed (read, write, etc.)
      • status - the result of the operation being performed
  • trickster_cache_operation_duration_seconds (Histogram) - The time, in seconds, required to perform an operation on the Trickster cache. Deletions include both requested removals and index reaper evictions.

    • labels:
      • cache_name - the name of the configured cache performing the operation
      • provider - the type of the configured cache performing the operation
      • operation - the name of the operation being performed (get, set, setDirect, del)
      • status - the result of the operation being performed (e.g., hit, kmiss for a full key miss, none)
  • trickster_cache_operation_bytes_total (Counter) - The total number of bytes upon which the Trickster cache has operated. Deletions (del) record bytes only for cache providers that use an index, since other providers don’t track object sizes.

    • labels:
      • cache_name - the name of the configured cache performing the operation
      • provider - the type of the configured cache performing the operation
      • operation - the name of the operation being performed (read, write, etc.)
      • status - the result of the operation being performed
  • trickster_alb_pool_admits_failing (Gauge) - 1 when an ALB pool’s healthy_floor admits members in the unavailable state, 0 otherwise. See alb.md for the recommended floor.

    • labels:
      • backend_name - the name of the configured ALB backend
  • trickster_alb_pool_floor_reset (Gauge) - 1 when an ALB pool’s healthy_floor was reset to 0 at startup because pool members have no health check and could never reach the configured floor, 0 otherwise. See alb.md.

  • trickster_alb_pool_on_backup (Gauge) - 1 while an ALB pool that has backup members is dispatching to them because no other member is available, 0 otherwise. Present only for pools with backup members. See alb.md.

    • labels:
      • backend_name - the name of the configured ALB backend
  • trickster_alb_member_inflight (Gauge) - Current number of requests in flight to an ALB pool member. Exported for the mechanisms that track it (p2c, lc, lt), for requests and for stream connections and sessions alike; read when the metrics endpoint is scraped, at no cost to request routing.

    • labels:
      • alb_name - the name of the configured ALB backend
      • member - the name of the pool member backend
  • trickster_alb_member_draining (Gauge) - 1 for each ALB pool member that is draining: marked drain: true, or discovered while terminating but still serving. It keeps its sticky sessions and takes no new work. Members that are not draining have no series; read when the metrics endpoint is scraped.

    • labels:
      • alb_name - the name of the configured ALB backend
      • member - the name of the pool member backend
  • trickster_alb_member_ejections_total (Counter) - The number of times alb.stream.passive_health took a pool member out of selection after repeated connect failures.

    • labels:
      • alb_name - the name of the configured ALB backend
      • member - the name of the pool member backend
  • trickster_alb_sticky_total (Counter) - The number of requests, stream connections and native sessions through an ALB with sticky sessions, by how their session fared. Each is counted once it reaches its member or is refused, and not at all when it reaches no member; see Sticky Session Metrics.

    • labels:
      • alb_name - the name of the configured ALB backend
      • result - hit (sent to the member its session is pinned to), miss (no token or table entry), expired (a token past its ttl or idle; an expired table entry is a miss), invalid (a token that is altered, signed with another key or issued by another ALB), repick (its member was unavailable and the session moved) or rejected (its member was unavailable and on_unavailable: reject refused it)
  • trickster_alb_sticky_entries (Gauge) - The number of entries in the table of an ALB that keeps sticky sessions in a table, on any listener, expired entries not yet removed included; read when the metrics endpoint is scraped.

    • labels:
      • alb_name - the name of the configured ALB backend

The following metrics are available for ALBs that use TSM global evaluation. Queries answered from the ALB’s cache aren’t evaluated, so they aren’t counted.

  • trickster_alb_global_evaluations_total (Counter) - The number of TSM global evaluations

    • labels:
      • alb_name - the name of the configured ALB backend
      • status - success, partial (evaluated without an unavailable member), or the Prometheus API error type that answered the query: bad_data, execution, timeout, canceled, unavailable or internal
  • trickster_alb_global_evaluation_duration_seconds (Histogram) - The time TSM global evaluations take, from arrival to response, including raw series retrieval and any wait for an evaluation slot

    • labels:
      • alb_name - the name of the configured ALB backend
  • trickster_alb_global_fetched_series_total (Counter) - The number of raw series TSM global evaluations read from pool members or the raw series cache, before deduplication

    • labels:
      • alb_name - the name of the configured ALB backend
  • trickster_alb_global_fetched_samples_total (Counter) - The number of raw samples TSM global evaluations read from pool members or the raw series cache, before deduplication

    • labels:
      • alb_name - the name of the configured ALB backend
  • trickster_alb_global_member_failures_total (Counter) - The number of raw series reads from a pool member that failed during TSM global evaluation

    • labels:
      • alb_name - the name of the configured ALB backend
      • member - the name of the pool member backend
  • trickster_alb_global_raw_reads_total (Counter) - The number of raw series reads TSM global evaluations made of pool members, by how the raw series cache answered them

    • labels:
      • alb_name - the name of the configured ALB backend
      • status - hit when only samples within the volatile window were read from the member, phit when older samples the cache lacked were read too, kmiss when the cache held none, and proxy-only when every sample was within the volatile window, or the read couldn’t be cached
  • trickster_alb_global_raw_subsumed_total (Counter) - The number of raw series reads TSM global evaluations answered in part from the cached raw series of a broader selector on the same metric

    • labels:
      • alb_name - the name of the configured ALB backend

The following metrics are available when ALB Autodiscovery is configured:

  • trickster_alb_discovery_members (Gauge) - Current number of discovered ALB pool members

    • labels:
      • alb_name - the name of the discovery-backed ALB backend
      • discoverer - the name of the discoverer serving the ALB
  • trickster_alb_discovery_member_changes_total (Counter) - Count of discovered pool member additions and removals

    • labels:
      • alb_name - the name of the discovery-backed ALB backend
      • discoverer - the name of the discoverer serving the ALB
      • event - add or remove
  • trickster_alb_discovery_snapshots_total (Counter) - Count of membership snapshots processed, by result

    • labels:
      • alb_name - the name of the discovery-backed ALB backend
      • discoverer - the name of the discoverer serving the ALB
      • result - applied (membership updated), unchanged (no-op), rejected (guardrail-refused, e.g. min_members), or partial (applied with member instantiation failures)
  • trickster_alb_discovery_last_refresh_success_time_seconds (Gauge) - Epoch timestamp of the last successfully processed snapshot, for staleness alerting

    • labels:
      • alb_name - the name of the discovery-backed ALB backend
      • discoverer - the name of the discoverer serving the ALB
  • trickster_discovery_refresh_errors_total (Counter) - Count of provider-side refresh/watch errors (DNS resolution failures, Kubernetes list/sync failures, member-file read/parse failures)

    • labels:
      • discoverer - the name of the discoverer experiencing the error
      • provider - the discoverer’s provider type
  • trickster_tls_certificate_expiration_time_seconds (Gauge) - NotAfter time of a serving TLS certificate, as unix seconds. See tls.md.

    • labels:
      • listener - the name of the listener serving the certificate
      • entry - the certificate’s source identity
  • trickster_tls_certificate_last_load_time_seconds (Gauge) - Epoch timestamp a serving TLS certificate was last loaded from its source

    • labels:
      • listener - the name of the listener serving the certificate
      • entry - the certificate’s source identity
  • trickster_tls_certificate_swaps_total (Counter) - Count of TLS certificates hot-swapped into a live listener by rotation detection

    • labels:
      • listener - the name of the listener serving the certificate
      • entry - the certificate’s source identity
  • trickster_tls_certificate_validation_failures_total (Counter) - Count of detected TLS certificate source changes that failed pair validation (e.g. a mid-rotation partial write) and were not swapped in

    • labels:
      • entry - the certificate’s source identity
  • trickster_tls_watcher_errors_total (Counter) - Count of errors reading watched TLS certificate source files

    • labels:
      • entry - the certificate’s source identity
  • trickster_tls_certificate_store_size (Gauge) - Number of certificates in a listener’s TLS certificate store

    • labels:
      • listener - the name of the listener
  • trickster_acme_orders_total (Counter) - Count of first-time ACME certificate orders. See acme.md.

    • labels:
      • issuer - the name of the ACME issuer
      • result - success or failure
  • trickster_acme_renewals_total (Counter) - Count of ACME certificate renewals

    • labels:
      • issuer - the name of the ACME issuer
      • result - success or failure
  • trickster_acme_challenge_requests_total (Counter) - Count of ACME challenge requests received by listeners

    • labels:
      • type - http-01 or tls-alpn-01
      • result - served, unknown (no matching pending challenge) or error
  • trickster_acme_on_demand_decisions_total (Counter) - Count of on-demand issuance decisions

    • labels:
      • result - allowed, refused, refused_cached, rate_limited or ask_error
  • trickster_acme_startup_wait_seconds (Gauge) - Seconds startup readiness was held waiting for missing ACME certificates

  • trickster_acme_startup_wait_timeouts_total (Counter) - Count of startup waits that timed out before every ACME certificate was issued


The following metrics are available only for Caches Types whose object lifecycle Trickster manages internally (Memory, Filesystem and bbolt):

  • trickster_cache_events_total (Counter) - The total number of events that change the Trickster cache, such as retention policy evictions.

    • labels:
      • cache_name - the name of the configured cache experiencing the event$
      • provider - the type of the configured cache experiencing the event
      • event - the name of the event being performed
      • reason - the reason the event occurred
    • events and reasons:
      eventreasoncounts
      evictionttlremovals of expired objects, a batch at a time
      evictionsize_bytes, size_objectsremovals of least-recently-accessed objects from a cache over its size
      evictionfree_spaceremovals of least-recently-accessed objects for min_free_bytes (Filesystem)
      invalid_objectexpired, corrupt, unknown_formatobjects found to be unservable when read or swept, and removed (Filesystem and bbolt)
      indexcompactionreplacements of the Cache Index journal by a snapshot (Filesystem and bbolt)
      indexsweepcompleted sweeps of the cache (Filesystem and bbolt)
      sweepadoptedobjects a sweep found in the cache that the Cache Index did not list
      sweepdroppedobjects a sweep found the Cache Index to list that the cache no longer held
  • trickster_cache_usage_objects (Gauge) - The current count of objects in the Trickster cache.

    • labels:
      • cache_name - the name of the configured cache$
      • provider - the type of the configured cache$
  • trickster_cache_usage_bytes (Gauge) - The current count of bytes in the Trickster cache.

    • labels:
      • cache_name - the name of the configured cache$
      • provider - the type of the configured cache$
  • trickster_cache_max_usage_objects (Gauge) - The maximum allowed size of the Trickster cache in objects.

    • labels:
      • cache_name - the name of the configured cache$
      • provider - the type of the configured cache
  • trickster_cache_max_usage_bytes (Gauge) - The maximum allowed size of the Trickster cache in bytes.

    • labels:
      • cache_name - the name of the configured cache$
      • provider - the type of the configured cache

The following metrics are available for Static File Server Backends. Requests they serve are also counted, like those of any other Backend, by the trickster_frontend_requests_* metrics with a provider of static. Their Fileserver cache is separate from the caches above, and is not reported by the trickster_cache_* metrics.

  • trickster_fileserver_responses_total (Counter) - The total number of files served, by how the Fileserver cache figured in the response. Responses that send no file (such as a 404 with no not-found file, a redirect or a directory listing) are not counted.

    • labels:
      • backend_name - the name of the configured backend
      • cache_status - hit (served as it was held), phit (the file was held, and was encoded for the response and the rendition then held), kmiss (read from disk for the response, and then held) or disk (sent from disk without being held, as for a large file, a byte range, a HEAD or a 304)
      • encoding - the encoding of the rendition the file server sent: identity, zstd, br, gzip or deflate. A response counted as identity may still be encoded on its way out, as a large compressible file is.
  • trickster_fileserver_cache_events_total (Counter) - The total number of objects removed from the Fileserver cache.

    • labels:
      • backend_name - the name of the configured backend
      • event - eviction (the least recently used, removed to make room) or invalidation (removed because the file changed on disk, or the cache was stopped)

A backend’s series are published only once it is in service, so a configuration that is rejected publishes nothing. They are deleted when a reload removes or renames the backend; across a reload that keeps its name, the counters carry on rather than start over. The four gauges that follow are published only while the backend has a Fileserver cache, and are removed when it is disabled.

  • trickster_fileserver_cache_usage_objects (Gauge) - The current count of objects in the Fileserver cache, including files being read into it. Each held rendition of a file is an object.

    • labels:
      • backend_name - the name of the configured backend
  • trickster_fileserver_cache_usage_bytes (Gauge) - The current accounted size of the Fileserver cache in bytes, which includes each object’s bookkeeping allowance.

    • labels:
      • backend_name - the name of the configured backend
  • trickster_fileserver_cache_max_usage_objects (Gauge) - The configured max_files of the Fileserver cache.

    • labels:
      • backend_name - the name of the configured backend
  • trickster_fileserver_cache_max_usage_bytes (Gauge) - The configured max_size_bytes of the Fileserver cache.

    • labels:
      • backend_name - the name of the configured backend

The following metrics are available when the Kubernetes Gateway/Ingress controller is enabled (the top-level kubernetes section; see kubernetes-gateway.md):

  • trickster_moq_sessions_total (Counter) - Count of MoQ Transport sessions by outcome

    • labels:
      • listener_name - the listener the session arrived on
      • backend_name - the backend it was routed to, empty when it was refused before routing
      • role - downstream
      • transport - quic or webtransport
      • draft - the negotiated draft, empty when none was negotiated
      • result - ok, refused, no_route, busy or failed
  • trickster_moq_active_sessions (Gauge) - Current MoQ Transport sessions

    • labels:
      • listener_name - the listener serving the sessions
      • backend_name - the backend they were routed to
      • role - downstream
  • trickster_moq_requests_total (Counter) - Count of MoQ Transport requests by type and result

    • labels:
      • backend_name - the moq backend
      • type - subscribe, fetch, track_status, publish, publish_namespace, subscribe_namespace, subscribe_tracks or other
      • result - ok or refused
  • trickster_moq_active_subscriptions (Gauge) - Current MoQ Transport subscriptions

    • labels:
      • backend_name - the moq backend
      • direction - upstream (the relay’s subscriptions toward origins and publishers) or downstream
  • trickster_moq_objects_total (Counter) - Count of MoQ Transport objects by direction and delivery

    • labels:
      • backend_name - the moq backend
      • direction - in or out
      • delivery - stream, datagram or fetch
  • trickster_moq_bytes_total (Counter) - Count of MoQ Transport object payload and property bytes

    • labels:
      • backend_name - the moq backend
      • direction - in or out
  • trickster_moq_dropped_objects_total (Counter) - Count of MoQ Transport objects not forwarded

    • labels:
      • backend_name - the moq backend
      • reason - oversize, delivery_timeout, datagram_queue_full or spill_queue_full
  • trickster_moq_fetch_objects_total (Counter) - Count of objects served to FETCH and fills, by where they came from

    • labels:
      • backend_name - the moq backend
      • source - live_edge, cache or upstream
  • trickster_moq_live_edge_bytes (Gauge) - Current payload bytes held in a moq backend’s in-memory live edge

    • labels:
      • backend_name - the moq backend
  • trickster_moq_publishers_active (Gauge) - Current PUBLISH subscriptions publishers hold open to a moq backend

    • labels:
      • backend_name - the moq backend
  • trickster_moq_errors_total (Counter) - Count of MoQ Transport failures

    • labels:
      • backend_name - the moq backend
      • class - malformed_track or cache
  • trickster_kgw_reconciles_total (Counter) - Count of controller reconcile passes, by result

    • labels:
      • result - applied (the data plane was reloaded), unchanged (the generated configuration was already running), or error (the pass failed to compile or apply)
  • trickster_kgw_reconcile_duration_seconds (Histogram) - Duration of a whole reconcile pass, from reading the caches to writing status

  • trickster_kgw_reconcile_errors_total (Counter) - Count of reconcile passes that failed in a stage

    • labels:
      • stage - compile, apply, certificates (a certificate a listener’s store refused) or status (a status write the API server refused)
  • trickster_kgw_watch_events_total (Counter) - Count of Kubernetes watch events received by the controller

    • labels:
      • kind - the object kind (Gateway, HTTPRoute, Ingress, Secret, Service, …)
      • event - add, update or delete
  • trickster_kgw_translate_duration_seconds (Histogram) - Duration of translating the watched objects into the routing model

  • trickster_kgw_apply_duration_seconds (Histogram) - Duration of applying generated configuration to the data plane, observed only on passes that reload it

  • trickster_kgw_generated_objects (Gauge) - Size of the routing model the last pass produced

    • labels:
      • kind - listeners, routes, backends, certificates or policies
  • trickster_kgw_status_write_failures_total (Counter) - Count of status writes the API server refused, after conflict retries

    • labels:
      • kind - the kind of the object whose status could not be written
  • trickster_kgw_leader (Gauge) - 1 when this replica holds the leader election Lease and so writes status and Events, 0 otherwise

  • trickster_kgw_last_successful_sync_time_seconds (Gauge) - Epoch timestamp of the last reconcile pass whose generated configuration is running, for staleness alerting

  • trickster_kgw_route_info (Gauge) - A constant 1 per generated backend, joining it to the Kubernetes object it serves

    • labels:
      • kind - HTTPRoute or Ingress
      • route - the object’s name
      • namespace - the object’s namespace
      • backend_name - the generated backend’s name, as the backend_name label of the request metrics carries it

    For example, request rates per Kubernetes route:

    sum by (kind, namespace, route) (
      rate(trickster_proxy_requests_total[5m])
      * on (backend_name) group_left (kind, namespace, route) trickster_kgw_route_info
    )
    

In addition to these custom metrics, Trickster also exposes the standard Prometheus metrics that are part of the client_golang metrics instrumentation package, including memory and cpu utilization, etc.

Last modified October 27, 2026: bump docs for v2.2 (891dcc4)