In the middle of a viewbox
I found a dark background
where the straight path was lost
I used to be a Flash developer. It was not only my bread and butter, but also a playground. I was syncing audio with visual effects, keying color channels, growing farms.
Suddenly, the web became incomplete. Where did our Matrix Class go, our geom package, our magical ConvolutionFilter?
Luckily, SVG got enough attention in the community and on browsers to get vectors back. The playground was back. In steroids. SVG is not only about vectors and drawing primitives. You could do that with a canvas. It's also accessibility. Styling. Semantics. Interactivity. Image manipulation.
Enter filters and all their power.
I had two goals when starting this project. First one was creating the histogram of an image. The second was being able to modify the levels of that image as you can do with any image software.
How do they work?
Histograms count pixels and sort them.
They go through every pixel of an image, get the red intensity, and based on it's value, they store them in the corresponding group. So at the end, you get 256 boxes, each one with the number of pixels that matched that intensity. Same for blue, green and alpha channels.
If you represent each red box as a bar where the height is the number of pixels it stores, you can draw 256 bars one next to the other that represent the distribution of red intensity in your image.
Do the same with the 4 channels, and you get a histogram.
How to compute a histogram of an image in your browser?
Using a canvas.
The canvas element has a .getImageData method on it's context that returns an ImageData object with three properties: width, height and data.
That data property, in the latest specifications, is an ArrayBuffer with an Uint8ClampedArray interface. In other words, you get this huge array where, grouped every 4 indexes, you have the information of a single pixel. First position is red, second green, third blue, fourth alpha. And so on for every pixel. Each index value is always a number between 0 and 255.
In order to build your histogram, you just need to count how many pixels belong to every intensity and you're done. You get your 256 boxes with the number of pixels on each. All that's left to do is draw your bars.
The fact that the array is a Uint8ClampedArray instead of an Uint8Array will be handy for the next part. The difference between both, is that the former will approximate any value out of range to a value between 0 and 255. The latter will take the first 8 bit of the value resulting in unwanted values. Also the former rounds decimal numbers and the latter floors them.
So getting the histogram of a canvas is pretty straight forward. But how to draw a histogram of an svg? Unfortunately, there is no infallible solution.
When loading an external svg, you can load it as an <img>'s src, draw it to a canvas and follow the previous process. Same as if you were loading any other image format like PNGs or JPGs.
But when dealing with inline svg elements, there are two possibilities.
One is converting the innerHTML to a Blob, creating an object URL with createObjectURL and load it to an image as in the previous case. But this technique, has some caveats. When you load an svg as an external element, it cannot load any additional assets (for example images or fonts) and you lose any styles applied to them. So this approach might work only in some cases.
The other one is using a library like canvg, that converts svgs to canvas. I haven't tried it myself but they claim to have support for most of the svg specs.
Essentially all these solutions rely on canvas. Whatever you can render on a canvas, you can get it's histogram.
Thanks to svg's filter primitives, you can adjust levels on any svg element or nested element. You can even apply a level filter to an HTML element via css (except on IE and Edge).
And for canvas, provided you do the math yourself (or read until the end of the article), you can apply levels to your imageData and redraw your canvas.
What are levels?
Levels is a way of adjusting or correcting the colors of an image. What it does is basically mapping your input color with an output. You set your input color range, set your output color range, apply a gamma factor and you get your new color channel.
The formula goes something like this:
R' = outputBlack + (outputWhite - outputBlack) x Math.pow((R - inputBlack) / (inputWhite - inputBlack), 1 / gamma)
clamping any input tone lower than inputBlack to 0 and any tone larger than inputWhite to 1
How can you do it on svg?
Using the feComponentTransfer filter. It supports 4 different types of components, one for each color channel (feFuncR, feFuncG, feFuncB, feFuncA). Each of these support four different types of transfers:
The latter is the one I'll be using. The table transfer type needs an attribute called tableValues that takes a sequence of space separated values. Each value in the sequence gets mapped by an input value. Both sequences (input and output) are evenly distributed. So if you have 3 values, for example "0 0.12 0.25" they will be matched by the input "0 0.5 1". The intermediate values will be completed with a linear function.
Using a gamma type
The previous formula could also be achieved using the gamma type instead of the table type (thanks to Amelia for pointing this out). Where:
* exponent = gamma
* amplitude = (outputWhite - outputBlack)/(inputWhite - inputBlack)
* offset = outputWhite/256
But this would work only if you're not offsetting inputWhite and inputBlack. Otherwise, the clamped values under 0 and 1 would overflow instead.
It might work by combining it with another set of feComponentTransfer filters that would clamp those values.
Back to the table type
For creating the levels effect, I need to create a sequence of 256 values that will be matched by the 256 input tones of that color channel (In some special cases, that number could be reduced if output and input were divided by the minimum distance between them and the gamma value stays at 1).
So what you get in the end is an filter, with one or more children elements, each containing 256 tableValues.
Here's a live example of a working levels effect with svg filters.
How to do it on canvas?
For canvas, the process starts similar to svg. You can use the same previous function to build a map of each output value.
Then, iterating the Uint8ClampedArray, you can set the new color value by simply getting the input value in that index and using it as the index key of the map previously created. If you only want to change the red channel, you can apply your level values to the first index and then every 4 indexes. The green, blue and alpha channels work the same, offsetting the first index by 1,2 or 3.
That's it. Levels effect can be achieved directly on the browser instead of relying on an external software. And you could even export the final result by saving your canvas to PNG.
I've published on github 5 modules that I've used during this process.
* I was going to publish a histogram module but I found one that does essentially the same: histogram. It lacks maximum values for each channel in order to normalize and make a graph, but I'd rather do a pull request to his library instead of creating one.
Bonus track: Tritone, Quadtone, Megatone effect
The duotone effect converts a color or grayscale image into a 2 colored image. If you haven't read Amelia Bellamy-Royds's post about color matrices and duotone effects, I really recommend it. That article was part of my research and it's a great resource.
I needed to extend the duotone effect into more than two interpolated colors. My goal was to copy After Effect's Tritone effect, which allows to map luminance to three new colors. One color for hightones, one for midtones and the other one for shadows.
Amelia's technique relies on the constant column of the matrix added to each color input in order to interpolate between the two colors. But to be able to interpolate midtones to a different value, you need a different approach. Nevertheless, for duotone effect, I really recommend using that technique since it's a more compact (and very likely more performant) way to achieve it.
Luckily there is another filter you can use to map an image to multiple output colors. And, if you read the first part of the article, it's no suprise that the same filter that was used for levels is useful for this one, since it's basically doing the same thing: feComponentTransfer
How does it work?
First of all, you decompose your tritone color list (they can be three, four or any amount of colors) into their color components and create three table values (one for the red values, one for greens and one for blues). You'll end up getting three arrays with the values that your input image should be mapped with for each color channel.
You need to apply two sequential filters. The first one is to convert your image to grayscale. That's easily done using a feColorMatrix and setting the values to
0.3333 0.3333 0.3333 0 0
0.3333 0.3333 0.3333 0 0
0.3333 0.3333 0.3333 0 0
0 0 0 1 0
What you're basically doing here is multiplying each input channel with 1/3 of each Red, Green and Blue channels in order to get a gray value with an average of the three luminances.
The second filter is the feComponentTransfer with three children (feFuncR, feFuncG, feFuncB). You assign to each primitive one of the table arrays you calculated first. So when you apply these two filters to your image, you'll end up getting as an output, an image where each gray tone (from 0 to 255) gets mapped to a new color calculated this way:
* the gray value of a pixel gets computed (for example 100).
* It's then normalized (100/256)
* It looks up in the table where that value maps in your color range (between Color1 and Color2)
* An intermediate value between Color1 and Color2 (Color') is calculated and applied as the output value of that pixel.
* Repeat for every pixel
Again, you need to do this math yourself. First, you get the array with the image data. To turn your iamge gray, iterating each block of 4 values (red, green, blue and alpha), you take the first three and get the average of their values by adding them and dividing the result by three.
Ro = (Ri + Gi + Bi)* 0.33;
Go = (Ri + Gi + Bi)* 0.33;
Bo = (Ri + Gi + Bi)* 0.33;
As you can see, their final value is the same, so you can do this calculation only once per pixel. This will turn your pixel gray.
Then, depending on the pixel's luminance, you get the mapped value between Color1 and Color2 (say Color'). Finally, assign the red component of that Color' to the red index of that pixel. Same for green and blue.
Using this technique you don't need to stop at three colors, you can use as many as you want and then map the result in your svg filter, or your canvas pixels.
I'll publish these modules as soon as I clean them and get them ready to share.
We're getting closer to having complex image editing software directly on our browsers.
With these new web technologies, and combining them with other like web workers, you can achieve very interesting effects, that are dynamic, interactive and can always be helpful for animating and bringing attention to your compositions.
Always keep an eye on browser support and providing a backup plan for older browsers.
SVG's filters are a very powerful tool. You can achieve lots of effects in a performant way on your browser (although Firefox at the time of writing this has some really fps issues with filters).
Canvas are more complicated to manipulate and can take too much of your CPU while computing pixel values, so use it cautiously.
I'll keep working into supporting more image processing effects like Color Balance or Black & White.
Whenever I get more effects supported I'll be publishing the modules on github.