Y.Composite.Image is an interface for manipulating multi-dimensional arrays of uncompressed binary data. It is primarily designed for working with images and its internal data is compatible with the RGBA image format used by the canvas context2d object. Above and beyond two-dimensional images, Y.Composite.Image is theoretically capable of supporting unlimited pixel dimensions and unlimited data channels. This opens up a wide range of potential uses including working with compositing layers, animation or video, or voxel data sets like those used for 3D rendering simulations or a world map in games like Minecraft.
An image is basically just a multi-dimensional array and the dimensions determine the size, shape, and number of pixels in the image. If left undefined, dimensions defaults to the two-dimensional square [512, 512]. For standard two-dimensional images, think of this as [width, height]
There must be at least one dimension but there is no upper limit on the number of dimensions an image may have. Each dimension must have at least one pixel but there is no upper limit on the number of pixels a dimension may have.
For example a really long line could be defined with the dimensions [9999999999] or a 3D box could be defined with the dimensions [24, 37, 42] or a 4D hypercube could be defined with the dimensions [21, 21, 21, 21]
This API standardizes on the term `pixel` to mean an element of a multi-dimensional array even when there are more dimensions and a term like voxel might be more technically correct.
Also note that the total size and number of dimensions can have a huge impact on memory usage.
Every pixel within an image may contain one or more values. These are called channels. A default image has four channels referred to as RGBA. The first channel stores the red component of the pixel's color, the second channel stores the green component, then blue is third, and the alpha channel is last. These channels are all unsigned 8-bit integers. In other words the value can be an integer from 0 to 255.
When creating a new image, these default channels may be used or custom channels may be defined. There must be at least one channel but there is no upper limit on the number of channels an image may have. Channels may also be a data type other than unsigned 8-bit integers; they may be any numerical type used by typed arrays.
To specify custom channels, pass in an array. The length of the array will determine the number of channels and the value of each item in the array will denote the data type of that channel. The accepted types are:
If left undefined, channels is set to ['u8', 'u8', 'u8', 'u8']
It is permitted to use channels of different types such as ['u8', 's16', 'f64', 's8', 'f32'] but due to technical and boring reasons including byte alignment and endianness this is much less efficient than using homogeneous channel types.
Also note that the total byte size and number of channels can have a huge impact on memory usage.
Y.Composite.Image internally stores and interacts with its data by using typed arrays. It uses some relatively new features of typed arrays including DataView and Uint8ClampedArray. There is no support for older browsers without these features.
The internal data structure of Y.Composite.Image is an ArrayBuffer, which is like a single-dimensional array of binary numbers. The dimensions and channels are sequentially stacked behind each other. For example, for an image with three channels and dimensions [2, 3] the binary data is arranged like this:
Notice that there are two ways to identify a pixel. A pixel can be identified by its dimensional location or by its unique array index. This API refers to these values as pixelLocation and pixelIndex. In some places the API may accept them interchangeably. Accessing pixels by pixelIndex is generally more efficient.
Check out the API documentation for more information on how to use Y.Composite.Image.