Caching

Cantaloupe offers a sophisticated and customizable caching subsystem that is capable of meeting a variety of needs while remaining easy to use. Several tiers of cache are available:

  1. Client-side caches, which it has no control over but can provide hints to;
  2. A source cache, which caches source images on a local filesystem (if they are not already on one) for faster reading;
  3. A derivative cache, which caches processed images and source image characteristics;
  4. An info cache, which caches source image characteristics in memory.

Client-Side Caching

The Cache-Control response header, which is configurable via the cache.client.* keys in the configuration file, can provide caching hints to clients. To enable this header, set the cache.client.enabled key to true.

The default settings look something like this:

cache.client.max_age = 2592000
cache.client.shared_max_age =
cache.client.public = true
cache.client.private = false
cache.client.no_cache = false
cache.client.no_store = false
cache.client.must_revalidate = false
cache.client.proxy_revalidate = false
cache.client.no_transform = true

These are reasonable defaults that tell clients they can keep cached images for 2,592,000 seconds (30 days).


Server-Side Caching

Source Cache

The source cache caches whole, pre-processed source images onto the filesystem. The only reasons to do this are:

  1. Trading a performance penalty for initial source image access (as the image is downloaded into the source cache) in order to potentially—depending on the performance of the source—improve the efficiency of subsequent accesses. This would be configured by setting the stream retrieval strategy to CacheStrategy and configuring FilesystemCache.
  2. Having to use a processor that doesn't support reading from streams with a source that can only supply streams. By setting the fallback retrieval strategy to CacheStrategy, and then configuring FilesystemCache, the source cache will be utilized to automatically pre-download source images before they are accessed, as a more robust alternative to DownloadStrategy.

Basically, the source cache is a workaround for some kind of suboptimal situation in the image input pipeline—either a slow source or a codec limitation. Ideally, it would not have to be used.

There is only one available source cache implementation—FilesystemCache—and it is used independently of the derivative cache.

The source cache is integrated into the larger caching architecture, so all of the information about modes of operation and maintenance applies to both the source and derivative caches.

Note that when chunking is enabled on sources that support it (see the relevant documentation for HttpSource), in conjunction with a processor that supports seeking for a particular format, the source cache is bypassed, even when enabled.


Derivative Cache

The derivative cache caches post-processed images in order to spare the computational expense of processing the same image over and over again. Derivative caches are pluggable, in order to enable different cache stores.

In typical use, derivative caching will greatly reduce load on the server and improve response times accordingly. There are other ways of caching derivatives, such as by using a caching reverse proxy server, but the built-in derivative cache is custom-tailored for this application and easy enough to set up.

Derivative caching is disabled by default. To enable it, set cache.server.derivative.enabled to true, and set cache.server.derivative to the name of a cache, such as FilesystemCache.

Notes

  • Requests for full-sized, unaltered source images are not cached, and are instead streamed through with no processing.
  • IIIF information response representations are not cached—only image info, which is the only expensive part to acquire. This means that other configuration options that would affect the contents of information responses can be changed without invalidating the cache.
  • The derivative cache is shared across endpoints. Requests for the same image from different endpoints will return the same cached image.

Info Cache

The info cache caches image info objects in the Java heap, independently of the derivative cache. When both are enabled, the info cache acts as a "level 1" cache in front of the "level 2" derivative cache:

  1. Requested image infos are retrieved from the info cache, if available;
  2. If not, they are retrieved from the derivative cache, if available, and also added to the info cache;
  3. If not available in any cache, they are retrieved from a processor, and added to both the info and derivative caches—whichever are enabled.

The info cache can be enabled or disabled via the cache.server.info.enabled configuration key.

The info cache is cluster-safe: when multiple instances are sharing the same derivative cache, there will never (for more than a short period of time) be an info in an instance's info cache that isn't also present in the derivative cache.

The maximum size of the info cache is hard-coded to a reasonable percentage of the maximum heap size, and is not configurable. As infos are very small, the maximum size is unlikely to ever be reached, but if it is, the least-recently-accessed infos will be invalidated as necessary to accommodate fresher ones.

The info cache's content never expires, but it is not persisted.


Modes of Operation

The source and derivative caches can be configured to operate in one of two ways:

Conservative (cache.server.resolve_first = true)
Source images are looked up and verified to exist before cached representations are returned. This precludes returning a cached representation when the underlying resource no longer exists, but also impairs response times by a variable amount, depending on the source.
Aggressive (cache.server.resolve_first = false)
Cached content is returned immediately, if available. This is faster, but inconsistency can develop between the cache and the underlying source image storage if the latter is not static.

Maintenance

Because cached content is not automatically deleted after becoming invalid, there will likely be a certain amount of invalid content taking up space in the cache at any given time. Without periodic maintenance, the amount can only grow. If this is a problem, it can be dealt with manually via the HTTP API, or automatically, using a cache worker, which periodically cleans and purges invalid items. (See the cache.server.worker.* configuration options.)


Limiting

Most caches (with the exception of HeapCache) age-limit their content based on last-accessed or last-modified time. Depending on the amount of source content served, the varieties of derivatives generated, the time-to-live setting, and how often maintenance is performed, the cache may grow very large. Its size is not tracked, as this would be either expensive, or, for some cache implementations, impossible. Managing the cache size is therefore the responsibility of the administrator, and it can be accomplished by any combination of:

  1. Performing maintenance more often;
  2. Reducing the time-to-live (using the cache.server.source.ttl_seconds and/or cache.server.derivative.ttl_seconds configuration keys);
  3. Allocating more storage to the cache.

Implementations

All implementations are thread-, process-, and cluster-safe, where applicable. Multiple application instances can be pointed at the same cache store without conflicting.

FilesystemCache

FilesystemCache caches content in a filesystem tree. The tree structure looks like:

  • FilesystemCache.pathname/
    • source/ (1)
      • Intermediate subdirectories (2)
        • {hashed identifier} (3)
    • image/
      • Intermediate subdirectories (2)
        • {hashed identifier}_{hashed operation list string representation}.{output format extension} (3)
    • info/
      • Intermediate subdirectories (2)
        • {hashed identifier}.json (3)
  1. Empty unless source caching is enabled.
  2. Some filesystems have per-directory file count limits, or thresholds beyond which performance starts to degrade. To work around this, cache files are stored in subdirectory trees consisting of leading fragments of identifier MD5 hashes, configurable by FilesystemCache.dir.depth and FilesystemCache.dir.name_length.
  3. Identifier and operation list strings in filenames are MD5-hashed in order to allow for lengths longer than the filesystem's filename length limit.

Cache files are created with a .tmp extension and moved into place when closed for writing.

On macOS, files' last-modified time is used instead of their last-accessed time, as the latter is not reliable.


HeapCache

HeapCache caches derivative images and metadata in the Java heap, which is the main area of memory available to the JVM. This is the fastest of the caches, with the main drawback being that it cannot be shared across instances.

Unlike most of the other caches, this one does not age-limit content. When the target size (HeapCache.target_size) has been exceeded, the minimum number of least-recently-accessed items are purged that will reduce it back down to this size. (The configured target size may be safely changed while the application is running.)

Because this cache is not time-limited, cache.server.derivative.ttl_seconds does not apply, and, if enabled, the cache worker will remain idle.

When using this cache, ensure that your heap is able to grow large enough to accommodate the desired target size (using the -Xmx VM option), and that you have enough RAM to accommodate this size.

Persistence

This cache can persist its contents to disk using the HeapCache.persist and HeapCache.persist.filesystem.pathname configuration keys. When persistence is enabled, the contents of the cache will be written to a file at shutdown, and loaded back in at startup. If persistence is disabled, the cache contents will be lost when the application exits.

Some thought was given to storing cached data using the same on-disk format used by FilesystemCache, so that persisted data would be compatible between these caches. Unfortunately, this is not possible because of the one-way hashing used in the FilesystemCache format.


JdbcCache

JdbcCache caches derivative images and metadata in relational database tables. To use this cache, a JDBC driver for your database must be installed on the classpath.

JdbcCache is tested with the H2 database. It is known not to work with the official PostgreSQL driver, as of version 9.4.1207. Other databases may work, but are untested.

JdbcCache can be configured with the following options:

JdbcCache.url
JDBC connection URL; for example, jdbc:postgresql://localhost:5432/mydatabase.
JdbcCache.user
User to connect to the database as.
JdbcCache.password
Password to use when connecting to the database. Can be left blank if not needed.
JdbcCache.image_table
Table in which to cache derivative (post-processed) images.
JdbcCache.info_table
Table in which to cache information responses.

JdbcCache will not create its schema automatically—this must be done manually using the following commands, which may have to be altered slightly for your particular database:

CREATE TABLE IF NOT EXISTS {JdbcCache.derivative_image_table} (
   operations VARCHAR(4096) NOT NULL,
   image BLOB,
   last_accessed DATETIME
);

CREATE TABLE IF NOT EXISTS {JdbcCache.info_table} (
  identifier VARCHAR(4096) NOT NULL,
  info VARCHAR(8192) NOT NULL,
  last_accessed DATETIME
);

CREATE INDEX operations_idx ON {JdbcCache.derivative_image_table} (operations);
CREATE INDEX identifier_idx ON {JdbcCache.info_table} (identifier);

S3Cache

S3Cache caches derivative images and metadata in an Amazon Simple Storage Service (S3) bucket. It supports both AWS and non-AWS endpoints.

Although S3 doesn't natively support the concept of a last-accessed time, S3Cache asynchronously makes copies of the objects it accesses, which enables last-modified times (which are set by S3 at object creation and immutable) to serve as last-accessed times.

Configuration

S3Cache is configured (excepting credentials) using the following configuration keys:

S3Cache.endpoint
URI or hostname of an S3 endpont. For AWS endpoints, this will be a value like s3.us-east-2.amazonaws.com. (See the list of AWS S3 regions.)
S3Cache.bucket.name
Name of the bucket to contain cached content.
S3Cache.object_key_prefix
String to prepend to object keys—for example, to achieve a virtual folder hierarchy.
Credentials Sources

See the Credentials Sources information for S3Source. S3Cache works the same way, except that the credentials-related configuration keys, if you choose to use them, are different:

S3Cache.access_key_id
An access key associated with your AWS account. (See AWS Security Credentials.)
S3Cache.secret_key
A secret key associated with your AWS account. (See AWS Security Credentials.)

AzureStorageCache

AzureStorageCache caches derivative images and metadata into a Microsoft Azure Storage container. It can be configured with the following options:

AzureStorageCache.account_name
The name of your Azure account.
AzureStorageCache.account_key
A key to access your Azure Storage account.
AzureStorageCache.container_name
Name of the container from which to serve images.
AzureStorageCache.object_key_prefix
String to prepend to object keys—for example, to achieve a virtual folder hierarchy.

Azure Storage does not provide a last-accessed time in object metadata, so the time-to-live is on the basis of last-modified time instead.


RedisCache

RedisCache, available since version 3.4, caches derivative images and metadata using the Redis data structure store. It supports the following configuration options:

  • RedisCache.host
  • RedisCache.port
  • RedisCache.ssl
  • RedisCache.password
  • RedisCache.database

Unlike with the other caches, cache policy is configured on the Redis side, and cache.server.derivative.ttl_seconds will have no effect with this cache. Likewise, if enabled, the cache worker will remain idle.