Most requests to an image server will be for cropped tiles and/or downscaled derivatives of a source image. Both of these will contain fewer pixels than the entire source image. A basic image reader, disregarding efficiency, will try to read the entire source image into memory before the cropping and scaling operations are carried out. This works fine for small images, because they can be read quickly and won't consume much memory. But as pixel count increases, read time and memory consumption also increases; an increasing burden is placed on the server; and requests take longer to fulfill.
It would be better if the image reader could read only the requested region, and even employ subsampling to read only the pixels within that region that are needed to satisfy the requested scale factor. This would require three things:
Cantaloupe tries to do the best it can with whatever source formats it is asked to serve, and from whatever data source. However, some image formats are inherently better suited for large sizes and some processor/format combinations will perform better than others.
Most processors can read images that have more than 8 bits per sample. However, as most web clients can't display more than 8 bits per sample, version 3.4 introduces a processor.limit_to_8_bits configuration option that will cause all images with more than 8 bits per sample to be rescaled to 8 bits.
When set to true, the processor.normalize configuration option normalizes the pixel values to utilize the full dynamic range of the output image. This is useful when working with 16-bit source images (for example) that do not use a full 16 bits of dynamic range and would appear overly dark when scaled down to 8 bits.
Most processors that support image sources support this format, some perhaps more efficiently than others, though this is untested.
JPEG is not an ideal format for delivery of high-resolution images because most readers need to read the entire image data into memory before decoding any part of it.
JPEG2000 uses advanced compression techniques to enable fast reduced-scale and region-of-interest decoding. With a performant decoder, it is well-suited for use with very large source images.
KakaduProcessor is the most efficient processor for this format, and it performs very well, even with huge images. Unfortunately, Kakadu is not free.
OpenJpegProcessor uses the OpenJPEG decoder, which is one of the faster open-source JPEG2000 decoders.
ImageMagickProcessor's JPEG2000 delegate, if installed, will also use the OpenJPEG library, but less efficiently as it won't use its region-extraction or level-reduction features.
GraphicsMagickProcessor can read and write JPEG2000 using JasPer, if the necessary plugin is installed. This will probably be too slow to be usable for most purposes.
There is an ImageIO plugin for JPEG2000 based on JJ2000 which would bring support to Java2dProcessor and JaiProcessor, but it is not included because it is too slow to be usable.
Most processors that support image sources support this format, some perhaps more efficiently than others, though this is untested.
In general, this is not an ideal format for delivery of high-resolution images, for the reason explained in the JPEG section.
TIFF is a common format that most processors can read. However, there are some criteria that source images must meet in order to be delivered efficiently.
The Adobe TIFF 6.0 specification permits arrangements of image data in either strips or tiles. Most TIFF encoders produce strip-based TIFFs unless told to do otherwise. These are increasingly slow to read as their size increases. High-resolution TIFFs should be tile-based in order to permit efficient region extraction. An easy way to check is with the tiffdump utility:
$ tiffdump image.tif
For strip-based TIFFs, this will print out some information including StripOffsets, StripByteCounts, and so on. For tile-based TIFFs, it will print TileOffsets, TileByteCounts, and so on, instead.
Multi-resolution TIFF is a special type of TIFF file that can be read more efficiently at reduced scales. In addition to the main image, a multi-resolution TIFF file will contain a sequence of progressively half-sized sub-images: for example, a 10000×10000 pixel main image would include versions of 5000×5000 pixels, 2500×2500 pixels, and so on, in the same file.
Each of the levels in a multi-resolution TIFF file can be striped or tiled, just as in a mono-resolution file. Tiled and pyramidal encodings are complementary: the former improves efficiency with reduced regions at large scales, and the latter improves efficiency at reduced scales. For efficient deep zooming, TIFF images need to be pyramidal, and each level of the pyramid should be tiled.
Ordinary TIFF files are limited to 4 GB in size. BigTIFF uses a different data layout that enables them to scale far beyond this. All processors that understand TIFF also understand BigTIFF.
Most processors can "read the TIFF format," but not all can read it efficiently. Currently, Java2dProcessor and JaiProcessor both support multi-resolution TIFF, which is to say that they read the embedded sub-images and choose the smallest one that can fulfill the request. Additionally, both exploit tiled sub-images. JaiProcessor, however, is able to use the JAI processing pipeline to do this more efficiently, so it is currently the best-performing processor for suitably-encoded high-resolution TIFF images.
Image files may have embedded color profiles, which map the color space in which an image was produced to its internal color data. This enables viewers to reproduce image colors accurately, as they were seen by the producer. By embedding a profile, the producer need not know anything about the displays on which an image will be viewed, and need not destructively modify the color values within the image data itself.
Most processors support embedded color profiles and will automatically copy them into derivative images; see the table of processor-supported features.
There is typically no need to embed a profile into profile-less images, as viewers will tend to automatically assume that these map to an sRGB space, and apply the conversion themselves. There is therefore no provision for embedding profiles into profile-less images.
Many image file formats are capable of storing supplementary technical and/or descriptive metadata alongside the actual image data. Formats may be able to store standard metadata formats like EXIF, IPTC IIM, and XMP, and they may also define their own metadata formats. More than one such format may be present even within the same file.
When an image request is received—unless it calls for the full-sized unmodified source image—the image server will have to dynamically create and return a derivative image. As this is a whole new image, distinct from the source image, populating it with metadata would require an additional step.
When processor.metadata.preserve is set to true in the configuration file, an effort will be made to copy metadata from source images into derivative images.
This currently does not work in all processors; see the Supported Features table for a breakdown. It also does not generally work across formats.