Processors

Processors read images from sources, decode them, transform them according to request arguments, and encode and write derivative images back to the client. Processors can be selected in different ways per-request.

Different processors use different underlying codecs and image processing engines, which may have different quality, compatibility, dependency, performance, and licensing characteristics. The ability to choose among different processors is intended to make it straightforward to add support for new image formats; improve support for existing image formats via the substitution of better codecs; and decouple the image server implementation from any one codec.


Format Support

Different processors support different source formats. A table of supported formats is displayed in the Control Panel, as well as in the Supported Source Formats table below. A list of output formats supported for a given source format is contained within the response to an information request (such as /iiif/2/:identifier/info.json).


Selection Strategies

The processor.selection_strategy configuration key controls how a processor is selected on a per-request basis.

AutomaticSelectionStrategy
A processor is chosen automatically based on the source format, installed dependencies, and potentially other criteria, with priority given to those that are expected to be more efficient.
ManualSelectionStrategy
A processor is chosen based on the processor.ManualSelectionStrategy.{format} and processor.ManualSelectionStrategy.fallback keys in the application configuration. This strategy offers more control, but requires knowing which processors support which source formats, and may require testing different processors to find the one that best meets a given use case.

Retrieval Strategies

Processors ultimately read images from sources, of which there are two main types: those that can supply files (FileSources), and those that can supply streams (StreamSources). Correspondingly, there are two types of processors: those that can read from files (FileProcessors), and those that can read from streams (StreamProcessors). These distinctions are important because they influence how data flows through the processing pipeline, which influences performance.

Stream Retrieval Strategy

The stream retrieval strategy (processor.stream_retrieval_strategy) controls how content is fed to stream-based processors from stream-based sources.

StreamStrategy
Source images are streamed from a source when possible, and the fallback retrieval strategy is used otherwise.
DownloadStrategy
Source images are downloaded to temporary files and deleted upon request completion.
CacheStrategy
Source images are downloaded into the source cache using FilesystemCache, which must also be configured. This is significantly more efficient than DownloadStrategy if you can spare the disk space.

Fallback Retrieval Strategy

The fallback retrieval strategy (processor.fallback_retrieval_strategy) controls how an incompatible StreamSource/FileProcessor combination is dealt with.

DownloadStrategy
The source image is downloaded to a temporary file that the processor can read, and deleted after the request is complete.
CacheStrategy
The source image is downloaded into the source cache using FilesystemCache, which must also be configured. This is significantly more efficient than DownloadStrategy if you can spare the disk space.
AbortStrategy
The request fails.

Supported Features

GraphicsMagick ImageMagick Java 2D JAI Kakadu Demo Kakadu Native OpenJPEG FFmpeg PDFBox TurboJPEG
Reading from files
Reading from sequential streams × × ×
Reading from seekable streams × × × × × × ×
Mirroring
Region by percent
Region by pixels
Square region
Right-angle rotation
Arbitrary rotation
Upsizing
Size by whitelisted/confined width/height
Size by forced/distorted width/height
Size by height
Size by width
Size by percent
Size by width/height
Grayscale output
Bitonal output
String overlays × ×
Image overlays ×
Repeating image overlays × ×
Redaction × × ×
Tiled reading × × N/A N/A ×
Multiresolution reading × × N/A N/A N/A
Copying metadata into derivatives of same format × × × N/A N/A ×
ICC profiles ✓* ✓* ✓* ✓* ✓* ✓** ✓** N/A × ✓**
Awareness of EXIF Orientation tag × × × N/A N/A ×
Selectable resample filters × ×
Sharpening

✓* Copied into derivative images.

✓** Derivative image pixel data is modified according to the ICC profile.


Supported Source Formats

GraphicsMagick ImageMagick Java2D JAI Kakadu Demo Kakadu Native OpenJPEG PDFBox TurboJPEG
BMP × × × × ×
GIF
Animation not supported

Animation not supported
× × × × × ×
JPEG CMYK/YCCK not supported × × × ×
JPEG2000 Uses JasPer (requires plugin); no level-reduction or ROI decoding Uses OpenJPEG (requires delegate); no level-reduction or ROI decoding × × × ×
PDF Requires plugin Requires delegate × × × × × ×
PNG × × × × ×
TIFF No multi-resolution or ROI decoding No multi-resolution or ROI decoding × × × × ×
WebP Requires plugin Requires delegate × × × × × × ×

Implementations

Java2dProcessor

Java2dProcessor uses the Java Image I/O and Java 2D libraries to read and process images in a native-Java way. It is a good all-around processor with no dependencies.

This processor has been written to exploit the Image I/O readers as efficiently as possible. Special attention has been paid to its handling of tiled images, such as tile-encoded TIFFs, for which it reads only the necessary tiles for a given request. It is also capable of reading the sub-images contained within pyramidal TIFF images.

Image I/O Plugins

By default, this processor uses the format-specific Image I/O plugins bundled with the JDK. Other plugins can be used instead by setting the processor.imageio.{format}.reader and/or processor.imageio.{format}.writer configuration keys to the fully-qualified class name of a plugin reader and writer, respectively. Alternative plugins could improve this processor's format support without requiring any changes to the application.

Recgonized plugins are logged at startup.

Supported Sources

This processor can read from files and streams. Seekable streams will be more efficient than sequential streams when working with pyramidal TIFF source images.

This processor relies on the default Image I/O JPEG plugin. This plugin implements the JFIF standard strictly and is known to fail to read certain images with error message such as, "Inconsistent metadata read from stream." If this turns out to be a problem, the recommended workaround, for the time being, is to use TurboJpegProcessor instead.


JaiProcessor

Java Advanced Imaging (JAI) is a sophisticated image processing library developed by Sun Microsystems until the mid-2000s. It offers several advantages over Java 2D that make it ideal for an image server: a pull-based rendering pipeline that can reduce memory usage, and efficient region-of-interest decoding with some formats.

Supported Sources

This processor can read from files and streams. Seekable streams will be more efficient than sequential streams when working with pyramidal TIFF source images.

Development on JAI ended a long time ago, and a minor incompatibility has cropped up in Java 9. Given that supporting JAI is likely to become more problematic as time goes on, this processor should be considered deprecated, and it may be removed in a future release.

When using this processor, it is normal to see the following log message:

Error: Could not find mediaLib accelerator wrapper classes. Continuing in pure Java mode.

This is harmless and expected when there is no mediaLib JAR on the classpath. Add the -Dcom.sun.media.jai.disableMediaLib=true VM option to suppress it.


GraphicsMagickProcessor

GraphicsMagickProcessor invokes the GraphicsMagick executable (gm). GraphicsMagick is not included and must be installed separately.

GraphicsMagick supports all of the IIIF transforms and most IIIF output formats (assuming the necessary libraries are installed; see Supported Formats).

GraphicsMagickProcessor is usually faster than ImageMagickProcessor. However, like ImageMagickProcessor, it is neither tile- nor multi-resolution-aware and thus fares poorly with large images.

Supported Sources

This processor can read from files and sequential streams.

Optional Arguments

GraphicsMagickProcessor supports a page URL query argument (which is nonstandard and Cantaloupe-specific) which can be used to return a particular page of a PDF. For example:

http://example.org/iiif/2/document.pdf/full/full/0/default.jpg?page=2

If the page argument is missing, the first page will be returned.


ImageMagickProcessor

ImageMagickProcessor invokes ImageMagick commands—either convert and identify (version 6 and below) or magick (version 7+). (The version will be autodetected at startup.) ImageMagick is not included and must be installed separately.

ImageMagick supports all of the IIIF transforms and all IIIF output formats, assuming the necessary delegates are installed. It can also support a wide array of source formats.

ImageMagick offers excellent output quality at the expense of performance, which suffers increasingly as image size increases.

Supported Sources

This processor can read from files and sequential streams.

Optional Arguments

ImageMagickProcessor supports a page URL query argument (which is nonstandard and Cantaloupe-specific) which can be used to return a particular page of a PDF. For example:

http://example.org/iiif/2/document.pdf/full/full/0/default.jpg?page=2

If the page argument is missing, the first page will be returned.


TurboJpegProcessor

This processor uses the Java binding of the high-level TurboJPEG API on top of the libjpeg-turbo library to read and write JPEG images. It uses the same image processing system as Java2dProcessor.

The design of the TurboJPEG Java binding is somewhat unfortunate in that it requires JPEG image data to be buffered fully in memory before it can be read or written, which costs time and RAM. However, overall performance is still significantly better than Java2dProcessor thanks to the much faster coding performance of libjpeg-turbo compared to the Image I/O JPEG plugin.

libjpeg(-turbo) is also more lenient than Java2dProcessor when reading malformed JPEGs, JPEGs with mismatching color profiles, and so on.

This processor requires libjpeg-turbo 2.0.2 to be installed. Other 2.0.x versions may work, but are untested. libjpeg-turbo must be compiled with Java support (which it often isn't in OS package managers). As of version 2.0.2, this just involves adding the -DWITH_JAVA=1 argument to the cmake command.


KakaduDemoProcessor

This processor was named KakaduProcessor in versions prior to 4.0, when it was the only way to use Kakadu. It was renamed in order to distinguish it from KakaduNativeProcessor.

As of version 4.1, this processor should be considered deprecated, and it may be removed in a future release.

Kakadu is widely considered the fastest available general-purpose JPEG2000 codec. This processor uses Kakadu's kdu_expand demo application to decode JPEG2000 source images. All subsequent operations are performed using Java 2D.

The Kakadu demo binaries must be installed in order for this processor to work. These can be obtained from the Kakadu website. Alternatively, the deps folder of the download archive contains prebuilt binaries for several platforms. kdu_expand will be detected automatically if it is on the path; otherwise, set the KakaduDemoProcessor.path_to_binaries configuration key to the absolute path of the containing directory. The Kakadu shared library files will also need to be installed per the instructions in the KakaduNativeProcessor section.

Supported Sources

This processor can only read from files. For use with sources other than FilesystemSource, one of the fallback retrieval strategies must be used.

Kakadu is not free software. This processor may be used in only one of the following ways:

  1. For demonstration purposes, as detailed on the Kakadu website.
  2. For non-commercial purposes under the terms of the Kakadu Public Service license, as explained in the Kakadu Software License Terms and Conditions document.

KakaduNativeProcessor

Kakadu is a commercial JPEG2000 codec that is widely considered the fastest available general-purpose JP2 codec. In contrast to KakaduDemoProcessor, this processor calls directly into the Kakadu library to decode JPEG2000 source images, and because of that:

  • It doesn't need to invoke a process.
  • It doesn't have to do intermediary format conversions.
  • It can use the fast, high-quality interpolation offered by the Kakadu library for differential scaling.
  • It can read from both FileSources and StreamSources.
  • It respects processor.metadata.preserve.
  • It works equally efficiently in Windows.
  • It's significantly faster in general.

Supported Sources

This processor can read from files and streams. Seekable streams will be more efficient than sequential streams.

Configuration

This processor must be able to locate the Kakadu JNI binding and shared library. The extracted release archive contains a folder named deps, which contains compiled binaries for several platforms. Copy the files from the platform-specific lib folder into one of the locations on the Java library path, which are logged at application startup in a message that looks like:

  [main] INFO  e.i.l.c.ApplicationContextListener - Java library path: .....

For Windows, you may also need to install Microsoft Visual C++ Redistributable, if it isn't already.

This processor was developed using a Kakadu Public Service License and may not be used commercially. See the Kakadu Software License Terms and Conditions for detailed terms.


OpenJpegProcessor

OpenJpegProcessor uses the opj_decompress tool from the open-source OpenJPEG project to decode JPEG2000 source images. All other operations are performed using Java 2D, and basic image characteristics are acquired using custom code.

To use this processor, OpenJPEG must be installed. The OpenJPEG binaries will automatically be detected if they are on the path; otherwise, set the OpenJpegProcessor.path_to_binaries configuration key to the absolute path of the containing directory. The LD_LIBRARY_PATH environment variable will also need to be set to locate the OpenJPEG shared library.

Supported Sources

This processor can only read from files. For use with sources other than FilesystemSource, one of the fallback retrieval strategies must be used.


FfmpegProcessor

FfmpegProcessor uses the FFmpeg tool to extract still frames from video files.

It has been tested with FFmpeg version 2.8. Other versions may or may not work.

FFmpeg is used only for frame extraction. All subsequent steps are handled by Java 2D.

Supported Sources

This processor can only read from files. For use with sources other than FilesystemSource, one of the fallback retrieval strategies must be used, which may be painful due to the large size of video files.

Optional Arguments

This processor supports a time URL query argument (which is nonstandard and Cantaloupe-specific) which can be used to return a frame at a particular second in the source video. The value of this argument should be in HH:MM:SS format, like:

http://example.org/iiif/2/video.mp4/full/full/0/default.jpg?time=00:02:15

If it's missing, the first frame will be returned.


PdfBoxProcessor

PdfBoxProcessor uses the Apache PDFBox library to read and rasterize PDF files. This is a pure-Java library that is bundled in and has no dependencies.

As PDF is a vector format, PdfBoxProcessor will convert to a raster (pixel) image and use a Java 2D pipeline to transform it according to the request arguments. The size of the base raster image, corresponding to a scale of 1, is configurable with the processor.dpi configuration option. When a request asks for a scale of ≤ 50% or ≥ 200%, a fraction or multiple of this will be used, respectively, in order to improve efficiency at small scales, and detail at large scales.

Supported Sources

This processor can read similarly well from all sources.

Optional Arguments

PdfBoxProcessor supports a page URL query argument (which is nonstandard and Cantaloupe-specific) which can be used to return a particular page of a PDF. For example:

http://example.org/iiif/2/document.pdf/full/full/0/default.jpg?page=2

If the page argument is missing, the first page will be returned.