Using reinforcement learning to make a computer paint like an oriental Sumi-e artist isn't just a matter of shouting "well done" - and yet when you look at the results that's what you want to do...
The oriental ink painting technique called Sumi-e is instantly recognizable no matter which part of the world you inhabit. The difference is that unlike western stroke based painting which uses layers of strokes to build up an image Sumi-e restricts its strokes in number - it is a minimalist approach to stroke based painting. This makes it very important to make maximum use of each stroke to convey what is to be seen. Thus Sumi-e strokes vary in thickness and style along the stroke. The appearance of the stroke is determined by the shape of an object to paint, the path and posture of the brush, and the distribution of pigments in the brush. As you can probably guess it takes time to learn how to use the brush to create particular types of stroke.
You can attempt to invent filters and physics based approaches to reproducing the skill but three researchers at the Tokyo Institute of Technology have attempted to teach a computer how to do it using standard reinforcement learning.
Reinforcement learning is based on the procedure of providing a reward when the agent gets increasingly close to doing the task correctly. The program models a soft brush agent that makes a sequential decision on which direction to move in and it is rewarded according to the gracefulness of the strokes it used in drawing arbitrary shapes. Once trained to draw a graceful shape the agent can be let loose on specific shapes.
The technical details are the the state space is modeled by a Markov Decision Process (MDP) and the policy gradient method is used to learn a locally optimal policy. Essentially what is happening is gradient accent to maximize the reward.
The state space controls the position, orientation and ink loading of the brush. The reward is a function of how smooth the stroke is. The training set was 80 digitized real single brush strokes commonly used in oriental ink painting. The boundaries of the each stroke path were used and the brush agent had to reproduce these in the smoothest way possible.
Once the brush agent was trained it was used to create ink paintings of photos. The contours that the brush follows were generated manually, so the artistic effect isn't quite as autonomous as it might appear. However, the result are supposed to be very good, even if you know what good Sumi-e technique is.
I can't help but think that the bird looks a lot like something from Angry Birds...
Siggraph doesn't happen till July but it is already showing what amazing things are going to be unveiled. A video condensing some of the work being presented is well worth seeing.
Yes - smell. Diophantine equations are just polynomial equations that use nothing but integers for their coefficients and solutions. They are very hard to solve and often very important.
