Caching

• 1: Cache Overview
• 2: Trickster Caching Retention Policies
• 3: Chunked Caching
• 4: Negative Caching

1 - Cache Overview

Supported Caches

There are several cache types supported by Trickster

• In-Memory (default)
• Filesystem
• bbolt
• BadgerDB
• Redis (basic, cluster, and sentinel)

The sample configuration (examples/conf/example.full.yaml) demonstrates how to select and configure a particular cache type, as well as how to configure generic cache configurations such as Retention Policy.

In-Memory

In-Memory Cache is the default type that Trickster will implement if none of the other cache types are configured. The In-Memory cache utilizes a Golang sync.Map object for caching, which ensures atomic reads/writes against the cache with no possibility of data collisions. This option is good for both development environments and most smaller dashboard deployments.

When running Trickster in a Docker container, ensure your node hosting the container has enough memory available to accommodate the cache size of your footprint, or your container may be shut down by Docker with an Out of Memory error (#137). Similarly, when orchestrating with Kubernetes, set resource allocations accordingly.

Filesystem

The Filesystem Cache is a popular option when you have larger dashboard setup (e.g., many different dashboards with many varying queries, Dashboard as a Service for several teams running their own Prometheus instances, etc.) that requires more storage space than you wish to accommodate in RAM. A Filesystem Cache configuration keeps the Trickster RAM footprint small, and is generally comparable in performance to In-Memory. Trickster performance can be degraded when using the Filesystem Cache if disk i/o becomes a bottleneck (e.g., many concurrent dashboard users).

The default Filesystem Cache path is /tmp/trickster. The sample configuration demonstrates how to specify a custom cache path. Ensure that the user account running Trickster has read/write access to the custom directory or the application will exit on startup upon testing filesystem access. All users generally have access to /tmp so there is no concern about permissions in the default case.

bbolt

The BoltDB Cache is a popular key/value store, created by Ben Johnson. CoreOS’s bbolt fork is the version implemented in Trickster. A bbolt store is a filesystem-based solution that stores the entire database in a single file. Trickster, by default, creates the database at trickster.db and uses a bucket name of ’trickster’ for storing key/value data. See the example config file for details on customizing this aspect of your Trickster deployment. The same guidance about filesystem permissions described in the Filesystem Cache section above apply to a bbolt Cache.

BadgerDB

BadgerDB works similarly to bbolt, in that it is a filesystem-based key/value datastore. BadgerDB provides its own native object lifecycle management (TTL) and other additional features that distinguish it from bbolt. See the configuration for more info on using BadgerDB with Trickster.

Redis

Note: Trickster does not come with a Redis server. You must provide a pre-existing Redis endpoint for Trickster to use.

Redis is a good option for larger dashboard setups that also have heavy user traffic, where you might see degraded performance with a Filesystem Cache. This allows Trickster to scale better than a Filesystem Cache, but you will need to provide your own Redis instance at which to point your Trickster instance. The default Redis endpoint is redis:6379, and should work for most docker and kube deployments with containers or services named redis. The sample configuration demonstrates how to customize the Redis endpoint. In addition to supporting TCP endpoints, Trickster supports Unix sockets for Trickster and Redis running on the same VM or bare-metal host.

Ensure that your Redis instance is located close to your Trickster instance in order to minimize additional roundtrip latency.

In addition to basic Redis, Trickster also supports Redis Cluster and Redis Sentinel. Refer to the sample configuration for customizing the Redis client type.

Trickster supports Redis servers that use TLS encryption by setting use_tls: true in the config. Refer to the sample configuration for more info.

Purging an Item from the Cache

You can purge an item from the cache by making a call to the purge endpoint, as follows:

http://${trickster-address}:${mgmt-port}/trickster/purge/path/${backendName}/${path/to/purge}

For example, if you want to purge /api/v1/labels from backend prom1, a curl might look like:

curl http://localhost:8484/trickster/purge/path/prom1/api/v1/labels

Purging the Full Cache

Full Cache purges should not be necessary, but in the event that you wish to do so, the following steps should be followed based upon your selected Cache Type.

A future release will provide a mechanism to fully purge the cache (regardless of the underlying cache type) without stopping a running Trickster instance.

Purging In-Memory Cache

Since this cache type runs inside the virtual memory allocated to the Trickster process, bouncing the Trickster process or container will effectively purge the cache.

Purging Filesystem Cache

To completely purge a Filesystem-based Cache, you will need to:

• Docker/Kube: delete the Trickster container (or mounted volume) and run a new one
• Metal/VM: Stop the Trickster process and manually run rm -rf /tmp/trickster (or your custom-configured directory).

Purging Redis Cache

Connect to your Redis instance and issue a FLUSH command. Note that if your Redis instance supports more applications than Trickster, a FLUSH will clear the cache for all dependent applications.

Purging bbolt Cache

Stop the Trickster process and delete the configured bbolt file.

Purging BadgerDB Cache

Stop the Trickster process and delete the configured BadgerDB path.

Cache Status

Trickster reports several cache statuses in metrics, logs, tracing, and the X-Trickster-Result response header, which are listed and described in the table below.

2 - Trickster Caching Retention Policies

Basic HTTP Backends

Trickster will respect HTTP 1.0, 1.1 and 2.0 caching directives from both the downstream client and the upstream origin when determining object cacheability and TTL. You can override the TTL by setting a custom Cache-Control header on a per-Path Config basis.

Cache Object Evictions

If you use a Trickster-managed cache (Memory, Filesystem, bbolt), then a maximum cache size is maintained by Trickster. You can configure the maximum size in number of bytes, number of objects, or both. See the example configuration for more information.

Once the cache has reached its configured maximum size of objects or bytes, Trickster will undergo an eviction routine that removes cache objects until the size has fallen below the configured maximums. Trickster-managed caches maintain a last access time for each cache object, and utilizes a Least Recently Used (LRU) methodology when selecting objects for eviction.

Caches whose object lifetimes are not managed internally by Trickster (Redis, BadgerDB) will use their own policies and methodologies for evicting cache records.

Time Series Backends

For non-time series responses from a TSDB, Trickster will adhere to HTTP caching rules as directed by the downstream client and upstream origin.

For time series data responses, Trickster will cache as follows:

TTL Settings

TTL settings for each Backend configured in Trickster can be customized independently of each other, and separate TTL configurations are available for timeseries objects, and fast forward data. See examples/conf/example.full.yaml for more info on configuring default TTLs.

Time Series Data Retention

Separately from the TTL of a time series cache object, Trickster allows you to control the size of each timeseries object, represented as a count of maximum timestamps in the cache object, on a per origin basis. This configuration is known as the timeseries_retention_factor (TRF), and has a default of 1024. Most dashboards for most users request and display approximately 300-to-400 timestamps, so the default TRF allows users to still recall recently-displayed data from the Trickster cache for a period of time after the data has aged off of real-time views.

If you have users with a high-resolution dashboard configuration (e.g., a 24-hour view with a 1-minute step, amounting to 1440 data points per graph), then you may benefit from increasing the timeseries_retention_factor accordingly. If you use a managed cache (see caches) and increase the timeseries_retention_factor, the overall size of your cache will not change; the result will be fewer objects in cache, with the timeseries objects having a larger share of the overall cache size with more aged data.

Time Series Data Evictions

Once the TRF is reached for a time series cache object, Trickster will undergo a timestamp eviction process for the record in question. Unlike the Cache Object Eviction, which removes an object from cache completely, TRF evictions examine the data set contained in a cache object and remove timestamped data in order to reduce the object size down to the TRF.

Time Series Data Evictions apply to all cached time series data sets, regardless of whether or not the cache object lifecycle is managed by Trickster.

Trickster provides two eviction methodologies (timeseries_eviction_method) for time series data eviction: oldest (default) and lru, and is configurable per-origin.

When timeseries_eviction_method is set to oldest, Trickster maintains time series data by calculating the “oldest cacheable timestamp” value upon each request, using time.Now().Add(step * timeseries_retention_factor * -1). Any queries for data older than the oldest cacheable timestamp are intelligently offloaded to the proxy since they will never be cached, and no data that is older than the oldest cacheable timestamp will be stored in the query’s cache record.

When timeseries_eviction_method is set to lru, Trickster will not calculate an oldest cacheable timestamp, but rather maintain a last-accessed time for each timestamp in the cache object, and evict the Least-Recently-Used items in order to maintain the cache size.

The advantage of the oldest methodology better cache performance, at the cost of not caching very old data. Thus, Trickster will be more performant computationally while providing a slightly lower cache hit rate. The lru methodology, since it requires accessing the cache on every request and maintaining access times for every timestamp, is computationally more expensive, but can achieve a higher cache hit rate since it permits caching data of any age, so long as it is accessed frequently enough to avoid eviction.

Most users will find the oldest methodology to meet their needs, so it is recommended to use lru only if you have a specific use case (e.g., dashboards with data from a diverse set of time ranges, where caching only relatively young data does not suffice).

3 - Chunked Caching

Overview

In some caching setups, users may want to increase timeseries_retention_factor or timeseries_ttl forms a given backend to a very large size (e.g., a duration of days or weeks). This can cause issues if the cache provider is filesystem or redis, because the entire time series is loaded to extract even just a few data points. Eventually, this could negate the effects of caching altogether.

To mitigate this, Trickster supports chunking cache data, by splitting large datasets into subdivisions of a configurable maximum size. The chunks are reconstituted upon retrieval. Only the chunks needed to service a client request are accessed, rather than the entire time series cache object.

Chunking can be configured per-cache and applies to both timeseries and byterange data.

Configuration

Chunked caching can be enabled and disabled using use_cache_chunking:

fs1:
    provider: filesystem
    use_cache_chunking: true
    timeseries_chunk_factor: 420
    byterange_chunk_size: 4096

timeseries_chunk_factor determines the maximum extent of timerange chunks, and byterange_chunk_size determines the maximum size of byterange chunks. See Detail for more information.

Detail

Timeseries

Timeseries chunking splits the timeseries to be cached into parts with the same duration, but not necessarily the same literal size.

• Determine a chunk duration by multiplying the timerange step by timerange_chunk_factor (default 420)
• Determine the smallest possible extent that is aligned to the epoch along the chunk duration, while containing the entire timeseries
• To write: Write each chunk size subextent under a subkey
• To read: Read each subkey and merge the timeseries results

Byterange

Byterange chunking splits the byterange into pieces with the same literal size. There are also some extra steps compared to the timeseries implementation to preserve the integrity of both full and partial responses being cached.

• Determine a chunk size from byterange_chunk_size (default 4096)
• Determine a range from the cache read/write request using the provided range or content length
• Failure to determine a range on write results in an error
• Failure to determine a range on read will read until the query fails
• Determine a maximum range aligned along the chunk size that contains the entire byterange
• To write: Write each chunk size range with RangeParts of all provided ranges cropped to that chunk range, under a subkey
• To read: Read each subkey and reconstitute a body from RangeParts, if able

Full Example

This example has one Prometheus backend with a memory cache that has chunking enabled. The memory cache uses 380 as its timeseries chunk factor, and doesn’t define a byterange chunk size, so the default of 4096 will be used.

caches:
  mem1:
    provider: memory
    index:
      max_size_objects: 512
      max_size_backoff_objects: 128
    use_cache_chunking: true
    timeseries_chunk_factor: 380

backends:
  prom1:
    latency_max: 150ms
    latency_min: 50ms
    provider: prometheus
    origin_url: 'http://127.0.0.1:9090'
    cache_name: mem1

logging:
  log_level: warn

4 - Negative Caching

Negative Caching means to cache undesired HTTP responses for a very short period of time, in order to prevent overwhelming a system that would otherwise scale normally when desired, cacheable HTTP responses are being returned. For example, Trickster can be configured to cache 404 Not Found or 500 Internal Server Error responses for a short period of time, to ensure that a thundering herd of HTTP requests for a non-existent object, or unexpected downtime of a critical service, do not create an i/o bottleneck in your application pipeline.

Trickster supports negative caching of any status code >= 300 and < 600, on a per-Backend basis. In your Trickster configuration file, associate the desired Negative Cache Map to the desired Backend config. See the example.full.yaml, or refer to the snippet below for more information.

The Negative Cache Map must be an all-inclusive list of explicit status codes; there is currently no wildcard or status code range support for Negative Caching entries. By default, the Negative Cache Map is empty for all backend configs. The Negative Cache only applies to Cacheable Objects, and does not apply to Proxy-Only configurations.

For any response code handled by the Negative Cache, the response object’s effective cache TTL is explicitly overridden to the value of that code’s Negative Cache TTL, regardless of any response headers provided by the Backend concerning cacheability. All response headers are left in-tact and unmodified by Trickster’s Negative Cache, such that Negative Caching is transparent to the client. The X-Trickster-Result response header will indicate a response was served from the Negative Cache by providing a cache status of nchit.

Multiple negative cache configurations can be defined, and are referenced by name in the backend config. By default, a backend will use the ‘default’ Negative Cache config, which, by default is empty. The default can be easily populated in the config file, and additional configs can easily be added, as demonstrated below.

The format of a negative cache map entry is 'status_code': ttl.

Example Negative Caching Config

negative_caches:
  default:
    '404': 3s # cache 404 responses for 3 seconds
  foo:
    '404': 3s
    '500': 5s # caches 404 response for 3 seconds, and 500/502 for 5 seconds
    '502': 5s

backends:
  default:
    provider: rpc
    # by default will assume negative_cache_name = 'default'
  another:
    provider: rpc
    negative_cache_name: foo