Tutorial


1. Installation

Installing Mistral is quite easy: you just need to include Mistral jar files in the classpath. Instead of a single, large jar file, several smaller files are available to avoid the need of including in the classpath unneeded stuff.

  • EditableImage.jar contains all the base features of Mistral;
  • Renderer.jar contains the classes used for displaying an image in a GUI;
  • Metadata.jar contains the support for metadata extraction;
  • Operations.jar contains the operation definitions, used for image manipulation;
  • JAI-Adapter.jar must be added if you plan to use JAI together with Mistral
  • ImageJ-Adapter.jar must be added if you plan to use ImageJ together with Mistral (not functional yet)
  • Processor.jar contains the classes for parallel and distributed processing;
  • Contributions.jar contains classes whose API has not been finalized yet.
The following third-party product can be optionally used with Mistral:
  • JAI
  • ImageJ
  • JAI-ImageIO
  • metadata-extractor by Drew Noakes

2. EditableImage

The core class of the Mistral project is EditableImage: it represents an image that can be loaded, manipulated, saved and shown on a GUI. EditableImage is an opaque holder, as it hides from the programmer's view the concrete implementation of the image itself. This approach allows to plug-in different imaging engines, as each one could have its own way to represent an image.

2.1. Loading an image

To load an EditableImage the create() method must be used in conjunction with the ReadOp operation:

File file = new File("MyImage.jpg");
EditableImage image = EditableImage.create(new ReadOp(file));
ReadOp contains all the information required for the load operation and can contain the following attributes:

input the source to load the image from Mandatory It can be:
  • a File 
  • a java.io.InputStream
  • a javax.imageio.ImageReader;
type the type of data that we want to read Mandatory It can be:
  • IMAGE, if we want to load both the raster and the metadata;
  • THUMBNAIL, if only the thumbnail should be read;
  • METADATA if only the metadata should be read.
imageIndex the image index Optional
(defaults to 0)
If the source contains more than a single image, this parameter specifies which one should be loaded.
thumbnailIndex the thumbnail index Optional
(defaults to 0)
If the image has more than a single related thumbnail, this parameter specifies which one should be loaded.

The image is always loaded by means of the Image I/O API, which is the standard way to perform I/O on images with Java. This facility is based on plug-ins for dealing with an expandable number of image format by just adding the required code in the classpath. It is possible to query the currently supported file formats by calling:
Collection<String> extensions = EditableImage.getAvailableExtensions();

2.2. Creating an empty image

Alternatively, an image can be created empty. Instead of using a ReadOp, a CreateOp specifies the required attributes. For instance:

EditableImage image = EditableImage.create(
new CreateOp(1024, 768, EditableImage.DataType.BYTE));
creates a single-band image with the given size and data type.

2.3. Creating an image out of a function

It is possible to create a new image whose raster data are set by an user specified function. See CreateFunctionOp.


2.4. Properties

Once an EditableImage has been loaded, some generic properties can be retrieved by means of getter methods. They are:

width, height the dimensions of the image in pixels
bandCount the number of channels of the image (e.g. 1 for monochrome images, 3 for RGB images)
bitsPerBand the number of bits used to represent a single channel (e.g. 8 for the most common image formats; 10, 12 or 16 for some professional formats)
bitsPerPixel the product of bitsPerBand and bandCount
dataType returns a value of the enumeration EditableImage.DataType, which tells about the primitive type used to represent each sample (e.g. BYTE, SHORT, UNSIGNED_SHORT, FLOAT, DOUBLE).
colorModel  

2.5. Metadata

EditableImage supports reading metadata from many different formats. While the primary source of information is Image I/O, Mistral is able to extract information also from other sources, for instance Drew Noakes' metadata-extractor. To enable this feature, it is just needed to put Drew's code in the classpath.

The EXIF directory and the maker note are available by means of two specific methods:

Directory exifDirectory = image.getEXIFDirectory();
Directory makerNote = image.getMakerNote();

3. Quality

As image processing can be CPU intensive, for many operations it is possible to apply a trade-off between the result quality and the speed. This is done by setting a quality property with the following values:

  • Quality.FASTEST: the fastest algorithm is used, regardless of the final image quality;
  • Quality.BEST: the algorithm delivering the highest quality is used, regardless of the speed;
  • Quality.INTERMEDIATE: mid-way between FASTEST and BEST
Each operation has its own way to accept the quality parameter, but usually it is passed in the constructor:
EditableImage image = ...;
image.execute(new ScaleOp(0.5, Quality.BEST));
The quality attribute applies as following:

Quality Java2D  interpolation Java2D  affine transform  
FASTEST Nearest Neighbor Nearest Neighbor  
INTERMEDIATE Bilinear Bilinear  
BEST Bicubic Bicubic  

4. Rendering

A specific component can be used for displaying an EditableImage on a GUI: EditableImageRenderer. This class extends Swing JComponent and can be placed in every Swing-based GUI. Using it can be as easy as:

JFrame frame = new JFrame("Test");
EditableImageRenderer imageRenderer = new EditableImageRenderer();
frame.getContentPane().add(imageRenderer);
frame.setSize(800, 600);
frame.setVisible();
imageRenderer.setImage(image);
By default, the image renderer does not use any scroll bar. It is possible to control whether a pair of functional scroll bars should be visible by setting a property:
imageRenderer.setScrollBarsVisible(true);
Scroll bars can be added and removed at any time, and EditableImageRenderer will adjust as required.
EditableImageRenderer is capable of scaling and panning. The scale can be controlled by means of the following methods:
double scale = imageRenderer.getScale(); // returns the current scale
imageRenderer.setScale(scale * 2); // changes the current scale
imageRenderer.fitToDisplaySize(); // fits the image to show entirely in the renderer
imageRenderer.setFitToDisplaySize(true): // the image will always shows entirely
// also after the component is resized
A special way of controlling the scale makes use of a "pivot point":
Point pivot = new Point(30, 40);
imageRenderer.setScale(imageRenderer.getScale() * 2, pivot);
This means that the scale is changed, but the renderer keeps the specified point "locked" (that is the enlargement/reduction is centered on the pivot).

Panning can be controlled by specifying which pixel of the image is shown in the upper left corner of the renderer (the "image origin"):
imageRenderer.setOrigin(new Point(10, 20));
Point origin = imageRenderer.getOrigin();
It's also possible to center the image on the renderer (keeping the current scale):
imageRenderer.centerImage()
Even though everything can be done by directly operating on the EditableImageRenderer, it's more advisable to use specialized controllers, described in the next paragraphs.

4.1. Margin

When scroll bars are not visible, it's possible to specify a margin around the image. This means that you can pan the image past its borders, as shown in the picture.



For instance, the following code sets a margin around the image which is one quarter of the image size:

EditableImage image = ...;
int hMargin= image.getWidth() / 4;
int vMargin = image.getHeight() / 4;
imageRenderer.setMargin(new Insets(vMargin, hMargin, vMargin, hMargin));

You can set independent values for the top, left, bottom and right margin.

When the image scale factor is so small that the image is shown with the full margin in the renderer, the image is always placed at the center of the renderer and panning is disabled.

4.2. ScaleController

The ScaleController is a better way for changing the current scale on an EditableImageRenderer:

ScaleController scaleController = new ScaleController(imageRenderer);
All the common scale operations are available on the ScaleController:
scaleController.setScale(1.5);
double scale = scaleController.getScale();
scaleController.setScale(2, new Point(30, 40)); // scale with a pivot point
scaleController.setZoomFactor(2); // zoom in/out will double/half the scale
scaleController.zoomIn();
scaleController.zoomOut();
scaleController.showActualPixels(); // equivalent to setScale(1);
scaleController.fitToView();

4.3. AnimatedScaleController

The AnimatedScaleController is a specialization of ScaleController which applies a smooth transition effect when the scale is changed.

ScaleController scaleController = new AnimatedScaleController(imageRenderer);
If a high quality is specified for rendering the image (e.g. Quality.BEST), during the transition effect the AnimatedScaleController might temporarily apply a lower quality for keeping the transition smooth. The high quality is however applied at the end of the transition.

4.4. MouseClickZoomingController

The MouseClickZoomingController allows to toggle zooming between the 1:1 (actual pixels) and the "fit to window" scale by a simple mouse click.

ScaleController scaleController = new AnimatedScaleController(imageRenderer);
MouseClickZoomingController zoomingController = new MouseClickZoomingController(scaleController);
zoomingController.setClickCountToZoom(2); // a double-click is required to trigger
zoomingController.setEnabled(true);
The actual scale behaviour is delegated to a ScaleController; for example, in the above sketch of code the animated effect is used. It is important to disable the controller when it's no more used, as this will remove some listeners to the image renderer.

4.5. MouseWheelZoomingController

The MouseClickZoomingController allows to set the scale of the rendered image using the mouse wheel.

ScaleController scaleController = new AnimatedScaleController(imageRenderer);
MouseWheelZoomingController zoomingController = new MouseWheelZoomingController(scaleController);
zoomingController.setEnabled(true);
The actual scale behaviour is delegated to a ScaleController; for example, in the above sketch of code the animated effect is used. It is important to disable the controller when it's no more used, as this will remove some listeners to the image renderer.

4.6. DragPanningController

The DragPanningController enables the user to control the panning over the image by using the mouse drag gesture.

DragPanningController panningController = new DragPanningController(imageRenderer);
panningController.setEnabled(true);

4.7. RotationController

The RotationController can be used to rotate the image in the renderer. The only operation available is setAngle() which controls the rotation applied to the image. Subclasses might implement animations (e.g. smoothly getting to the desired value).

EditableImagerRenderer imageRenderer = new EditableImageRenderer();
RotationController rotationController = new RotationController(imageRenderer);
...
rotationController.setAngle(44.3); // decimal, ccw

4.8. Events and Listeners

TBD


4.9. Performance

Performance can be an issue when the rendered image is frequently moved, scaled or rotated. By default, EditableImageRenderer performs all the needed scale operations on-the-fly, that is at each paint. This can be a severe performance penalty for medium-sized and large images. There are two ways of trading speed for memory:

  • calling setOptimizedImageEnabled(true), the rendered image is a copy of the original one, but with a data layout which is optimised for the current display;
  • calling setScaledImageCachingEnabled(true), a further scaled-down copy of the image is kept in memory, so it can be directly rendered at each paint
Please note that if you cal setAngle(angle) with an argument different than zero, a cached scaled copy of the image is always kept.
EditableImageRenderer imageRenderer = new EditableImageRenderer();
imageRenderer.setOptimizedImageEnabled(true);
imageRenderer.setScaledImageCachingEnabled(true);
A further way to control performance is by specifying the quality/speed trade off for the rendering operations, with separate settings for scaling and rotating:
EditableImageRenderer imageRenderer = new EditableImageRenderer();
imageRenderer.setScaleQuality(Quality.BEST);
imageRenderer.setRotateQuality(Quality.FASTEST);

5. Implementation providers

Mistral is based on a plug-in mechanism, by which each plug-in provides an implementation of operations. Not all plugins are able to provide the implementation for all the operations, thus Mistral is able to switch provider if needed.

Tip: Since this operation can require a model change, plugin switches can be expensive and should be performed only when needed.

Tip: Switching between the Java2D implementation (the default) and the JAI implementation is not an expensive operation.

By default Mistral comes with a partial implementation of operations on top of Java2D (that is, the standard code in the JRE runtime). Other plugins can be installed in Mistral by loading their ImplementationFactory. For instance, the following code installs the JAI plugin (the JAI-Adapter.jar file and the JAI stuff must be in the classpath):
import it.tidalwave.image.jai.ImplementationFactoryJAI;
...
ImplementationFactoryJAI.getInstance();
An alternate approach can be used that doesn't require dependencies at compile time:
try
{
Class.forName("it.tidalwave.image.jai.ImplementationFactoryJAI");
}

catch (Throwable t)
{
// ....
}
In this case you should always catch eventual errors from the operation, as it can fail if there are missing libraries in the classpath.

5.1. Disabling specific operations from a plugin

Usually the same implementation can be provided by more than a single plugin at the same time (for instance HistogramOp is provided by both the default Java2D implementation and the JAI implementation). If in specific cases you want to be sure that only one is used (for instance, JAI implementation of HistogramOp is much faster than Java2D), you can disable it by operating on the ImplementationFactory. For instance, the following code disables the Java2D implementation of HistogramOp:

ImplementationFactoryJ2D.getInstance().unregisterImplementation(HistogramOp.class);

5.2. Extending Mistral with custom operations

Enhancing Mistral with custom operations is straightforward and can be accomplished in three steps:

  1. write the abstract Operation
  2. write the concrete implementation
  3. bind them in the ImplementationRegistry
For instance, let's suppose we want to create a custom operation that changes the buffer type of the image (for instance from 8 bit to 16 bit, or to one of the many formats defined by the BufferedImage.TYPE_XXX constants. The abstract Operation can be written as:
public class ChangeBufferTypeOp extends Operation
{
private final int bufferType;

public ChangeBufferTypeOp (final int bufferType)
{
if ((bufferType <= 0) || (bufferType > 13))
{
throw new IllegalArgumentException("bufferType: " + bufferType);
}

this.bufferType = bufferType;
}

public int getBufferType()
{
return bufferType;
}
}
The concrete implementation for Java2D is also simple:
public class ChangeBufferTypeJ2DOp extends OperationImplementation<ChangeBufferTypeOp, BufferedImage>
{
protected BufferedImage execute (final ChangeBufferTypeOp operation,
final EditableImage image,
final BufferedImage bufferedImage)
{
final int width = image.getWidth();
final int height = image.getHeight();
final BufferedImage result = new BufferedImage(width, height, operation.getBufferType());
Graphics g = null;

try
{
g = result.createGraphics();
g.drawImage(bufferedImage, 0, 0, null);
}
finally
{
if (g != null)
{
g.dispose();
}
}

return result;
}
}
The new operation and its implementation can be registered with the following code:
ImplementationFactoryJ2D.getInstance().
registerImplementation(ChangeBufferTypeOp.class, ChangeBufferTypeJ2DOp.class);
At this point, the new operation is available to the application just as the standard ones. For instance, the following code works:
File file = new File("photo.jpg");
EditableImage image = EditableImage.create(new ReadOp(file));
image.execute(new ChangeBufferTypeOp(BufferedImage.TYPE_3BYTE_BGR));
image.execute(new WriteOp("JPEG", new File("Result.tif")));

5.3. Writing a Mistral plugin

TBD


6. Core Operations

Core Operations are the most common kind of image manipulation facilities and are available in the "EditableImage" component.

Operation Java 2D      JAI    ImageJ
ConvertToBufferedImageOp y y  
ConvolveOp y y  
CreateFunctionOp   y  
CreateOp y y  
CropOp y y  
DrawOp y y  
ForceRenderingOp   y  
OptimizeOp y y  
PaintOp y y  
ReadOp y y  
RotateOp y    
RotateQuadrantOp y y  
ScaleOp y y  
SizeOp   y  
WriteOp y y  

6.1. CaptureOp

TBD


6.2. ConvertToBufferedImageOp

This operation converts the image into a BufferedImage.

EditableImage image = ...;
BufferedImage bufferedImage = image.execute(new ConvertToBufferedImageOp()).getBufferedImage();

6.3. ConvolveOp

TBD


6.4. CreateFunctionOp

This operation creates a new image whose pixel values are computed from a function (at the moment only monochromatic images are created).

PENDING: add example

Parameter Meaning
 
width the width of the image to create mandatory  
height
the height of the image to create mandatory
function the function to compute pixel values mandatory  
dataType the type for each sample  mandatory  

6.5. CreateOp

Creates a new image given the size and the data type.

EditableImage img;
int w = 1024;
int h = 768;
img = EditableImage.create(new CreateOp(w, h, DataType.BYTE)); // 1-band, 8bpp
img = EditableImage.create(new CreateOp(w, h, DataType.UNSIGNED_SHORT)); // 1-band, 16bpp
img = EditableImage.create(new CreateOp(w, h, DataType.BYTE, 0, 0, 0)); // 3-bands, 24bpp
img = EditableImage.create(new CreateOp(w, h, DataType.BYTE, 255, 255, 255)); // 3-bands, 24bpp
img = EditableImage.create(new CreateOp(w, h, DataType.BYTE, Color.WHITE)); // 3-bands, 24bpp
img = EditableImage.create(new CreateOp(w, h, DataType.FLOAT, 1.0)); // 1-band, 24bpp
img = EditableImage.create(new CreateOp(w, h, DataType.DOUBLE, 1.0, 1.0, 0.5)); // 3-bands,192bpp
Parameter Meaning  
width the width mandatory
height the height mandatory
data type the data type mandatory
filler the initial value to fill the raster with optional
color the initial value to fill the raster with, in form of Color optional

Not all the possible configurations are available with a given provider.

Data type Java2D JAI
BYTE 1 and 3 bands any number of bands
SHORT 1 band any number of bands
UNSIGNED_SHORT 1 band any number of bands
INT not available any number of bands
FLOAT not available any number of bands
DOUBLE not available any number of bands

6.6. CropOp

TBD


6.7. DrawOp

The DrawOp allows to draw arbitrary contents in the image raster. This operation takes as a mandatory argument an instance of DrawOp.Executor. You must implement the draw() method that should contain the actual code that draws into the raster.

EditableImage image = ...;
image.execute(new DrawOp(new DrawOp.Executor()
{
public void draw (Graphics2D g, EditableImage image)
{
g.setColor(Color.BLACK);
g.setFont(...);
g.drawString("Hello world!", 5, 5);
}
}));

6.8. ForceRenderingOp

TBD


6.9. OptimizeOp

This operation converts the image raster into a format which is the most suitable one for the current platform (the so-called "compatible image" in Swing jargon), while mantaining both the raster data and the color space. Usually this operation must be performed just after an image has been loaded or created, as it usually allows for dramatic performance improvements in other operations.
OptimizeOp can optionally change the size of the image according to a scale factor - in this case, it accepts a quality parameter to trade off speed for quality during resizing.

Parameter Meaning    
scale   the scale mandatory
quality   the quality optional (defaults to Quality.FASTEST)

6.10. PaintOp


6.11. PrintOp

TBD


6.12. ReadOp

TBD


6.13. RotateOp

RotateOp rotates the image around its center point. The resulting image is eventually enlarged to prevent clipping. The angle is in degrees and counterclockwise. This operation accepts a quality parameter to trade off speed for quality.
To rotate an image by a multiple of 90 degrees, it's advisable to use RotateQuadrantOp

Parameter   Meaning  
angle   the angle mandatory
quality   the quality optional (defaults to Quality.FASTEST)

6.14. RotateQuadrantOp

RotateQuadrantOp rotates the image by a multiple of 90 degrees. If the angle is 90 or 270 degrees, the resulting image dimensions are swapped. Since this operation just moves pixels, there's no degradation in the image and there's no quality parameter.

Parameter   Meaning  
angle   the angle mandatory

6.15. ScaleOp

ScaleOp changes the number of pixels in the image, reducing or enlarging it according to the scale factor. Two different scale factors, one horizontal and one vertical, can be specified. This operation accepts a quality parameter to trade off speed for quality.

Please note that this operation does not guarantee the highest quality in results.

Parameter Meaning  
xScale the horizontal scale mandatory
yScale the vertical scale optional (defaults to xScale)
quality the quality  optional (defaults to Quality.FASTEST)

6.16. SizeOp

TBD


6.17. WriteOp

To store an EditableImage the WriteOp operation must be used as in the example:

File file = new File("MyImage.jpg");
EditableImage image = ...;
image.execute(new WriteOp("JPEG", file));
WriteOp contains all the information required for the write operation and contains the following attributes:

Parameter Meaning
 
format the file format mandatory  
output the destination to store the image to mandatory It can be:
  • a String
  • a java.io.File 
  • a java.io.OutputStream
imageWriteParam type-specific format parameters optional See the ImageWriteParam javadoc for more information.

The image is always written by means of the Image I/O API, which is the standard way to perform I/O on images with Java. This facility is based on plug-ins for dealing with an expandable number of image format by just adding the required code in the classpath. It is possible to query the currently supported file formats by calling:
Collection<String> extensions = EditableImage.getAvailableExtensions();

7. Operations

The operations described in this chapter are advanced functions available in the "Operations" component. They are an official part of the Mistral API and relatively stable.

Operation Java 2D      JAI    ImageJ
AddOp   y  
ApplyUnsharpMaskOp   y  
AssignColorProfileOp y y  
ConvertColorProfileOp y y  
HistogramOp y y  
MultiplyOp    
TranslateOp   y  

7.1. AddOp

TBD


7.2. ApplyUnsharpMaskOp

TBD


7.3. AssignColorProfileOp

TBD


7.4. ConvertColorProfileOp

TBD


7.5. HistogramOp

HistogramOp omputes an histogram of the given image. The result is stored in an instance of the Histogram class.
This operation can be used as in the following example:

EditableImage image = ...;
Histogram histogram = image.execute(new HistogramOp()).getHistogram();
As this operation can be rather long, you should not call it from a thread that can't be blocked (such as the AWT thread).
The Histogram can be queried about the minimum and maximum values of each band:
int bandCount = histogram.getBandCount(); // 3 for RGB, 1 for monochromatic

for (int i = 0; i < bandCount; i++)
{
int min = histogram.getMin(i);
int max = histogram.getMax(i);
System.err.println("Band #" + i + ": min=" + min + ", max=" + max);
}
It is also possible to retrieve all the frequencies of a given band:
int[] frequencies = histogram.getFrequencies(2);
Tip: Please consider that the JAI implementation is about 20x faster than the default implementation.

7.6. MultiplyOp

TBD


7.7. TranslateOp

TBD


8. Contributed Operations

Contributed operations are third party, project-specific or experimental code that is not yet an official part of the Mistral API. This means that classes in this component can be dropped, added, changed, merged and so on. This component acts as a sort of "incubator": after some time, operations that get consolidated will be moved in the "Operations" component.

Operation Java 2D      JAI    ImageJ
BandMergeOp   y  
CenterShiftOp   y  
ChangeBufferTypeOp y    
ChangeFormatOp   y  
ConjugateOp   y  
DFTOp   y  
DivideByConstOp   y  
DivideComplexOp   y  
IDFTOp   y  
MagnitudeOp   y  
MultiplyComplexOp   y  
PadBlackOp   y  
PadPeriodicOp   y  
PadPeriodicPlanarOp   y  

8.1. BandMergeOp

TBD


8.2. CenterShiftOp

TBD


8.3. ChangeBufferTypeOp

TBD


8.4. ChangeFormatOp

TBD


8.5. ConjugateOp

TBD


8.6. DFTOp

TBD


8.7. DivideByConstOp

TBD


8.8. DivideComplexOp

TBD


8.9. IDFTOp

TBD


8.10. MagnitudeOp

TBD


8.11. MultiplyComplexOp

TBD


8.12. PadBlackOp

TBD


8.13. PadPeriodicOp

TBD


8.14. PadPeriodicPlanarOp

TBD


9. The ImagingProcessor

TBD