You probably know that you can represent any color with three numbers corresponding to Red, Green and Blue intensities, but this might not be the best way to do the job. What about pigments?

A group of researchers from Adobe and George Mason University have been looking into the idea of specifying color using a pigment color space - after all, it is what artists have been doing for centuries.

"When painting, artists choose or create a relatively small set of pigments to be used throughout the painting. We call this set the primary pigment palette. We assume that all observed colors in the painting are created by mixing or layering pigments from the palette. When we view a painting, either directly with our eyes or indirectly after digitizing it into a three-channel RGB image, we observe only the overall reflectance and not the underlying material parameters. In RGB-space, the underlying pigments which combine to form the appearance of a pixel are not accessible for editing. One color in the palette cannot be easily changed or replaced. Translucent objects, common in paintings due to the mixing of wet paint, cannot be easily extracted or inserted."

The idea is to take the RGB representation and analyse it into a set of pigments which are mixed at each pixel to produce its RGB value.

"Our analysis allows us to interpret the deep structure of the painting from the RGB image, which enables editing operations based on that structure."

Deducing a set of pigments is a non-linear problem:

Digital RGB color mixing is a linear operation: all mixtures of two RGB colors lie on the straight line between them. Mixing two physical pigments with the same apparent RGB colors, however, produces a curve of colors when rendering to RGB. Similarly, when an opaque RGB color is gradually thinned to reveal the background, the result is a straight line in RGB-space. When a real physical pigment is thinned, the result is again a curve.

The paper presents an algorithm that estimates the coefficients of a small set of primary pigments to minimize the RGB reconstruction error. To cut a long and interesting story short the job can be done in most cases to an acceptable degree of accuracy.

Ok so now you have the deep structure of the painting - what next?

The answer is that you can now do editing operations in pigment space. The first is that pigment changes are likely to correspond to object changes. The painter is likely to use different pigment mixes for semantic regions in the picture. And it does seem to work in that pigment based masks are closer to object boundaries than RGB masks.

Another interesting idea is that you can recolor a painting, not by some almost arbitrary RGB shift, but by asking what if the painter had used a different pigment or had used less or more of a pigment. Looking at the results below what strikes me first is that I think the painter got it right in the first place!

The color bar at the bottom of the left hand picture is the palette of pigments that it is inferred were used.

The paper goes on to suggest a "painterly" cut, copy and paste

For copy-paste, the user can specify a mask (using any mechanism) and the subset of pigments to copy. The selected region can then be pasted elsewhere as a new layer of paint on top of the image and re-composited. The paste operation can adjust paint properties simultaneously such as the thickness of the pasted layer, to achieve different compositing results, or can be added into the mixture mode as additional paint mixed into the painting layer, with relative scaling and renormalization. These options result in different painterly variations on standard image copy-pasting

See if you know these paintings well enough to spot the additions:

Personally I like the Andy Worhol style Venus.

By analysing the paintings of a particular artist it is possible to discover what pigments they used over a period of time:

We can also use our palette extraction method to analyze collections of images, by amending our method to jointly reconstruct the pixels of multiple images. We use this approach to extract aggregate palettes from paintings of Van Gogh organized by year (Fig. 15b). Two results are clear from this analysis – first, that the range of colors Van Gogh painted with expanded over the 1880s, as we expanded from eight pigments to ten pigments to achieve good reconstruction errors, and second, that the vibrancy increased dramatically as well.

Van Gogh's pigments in each year

More Information

Pigmento: Pigment-Based Image Analysis and Editing

Jianchao Tan, Stephen DiVerdi, Jingwan Lu, Yotam Gingold

Related Articles

Watching Paint Dry - GPU Paint Brush

WebGLStudio.js - 3D Development For The Web

LiquidFun V1.1 Runs In A Browser

Unified Particles Make Better Graphics

Water In WebGL - One You Have To See

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info