Microsoft Research has just released a video of a new Kinect animation system - KinÊtre. All you have to do is scan a 3D object, make a connection between your body and it, and start animating.
As you move so does the object. The key is how the different parts of the body are mapped to the object. So your legs, say, get mapped to the lower half of the object so that as you walk so does it - via a 3D distortion.
"The process to engage is deceptively simple. You use a consumer depth camera to scan an item such as a chair, a desk lamp, a bookcase, or a stepladder. Then you use the same camera to track your body and align your virtual limbs to the geometry of the item. With a word—“Possess!”—your virtual limbs are attached to the item. When you jump, the bookcase jumps. When you dance, the chair dances."
Being able to animate and object in real time opens up lots of story telling possibilities, not to mention games. You can even scan in another person's body and use it as an animated avatar.
Take two people and you can animate a bigger object such as a horse - one person being the front legs and one being the back in traditional pantomime style.
Suddenly you don't need a complex and expensive motion capture facility to create CGI animation. Now anyone can do it.
See the video and it all becomes clear:
Although this looks like a trivial application, notice that it is built on some sophisticated components. In particular you need to be able to scan a 3D object using just one Kinect, i.e. you need KinectFusion, and you need to master the mapping between the human figure and the 3D object.
The system is being presented at this year's SIGGRAPH and as the Microsoft team says, KinÊtre: Animation + Whimsy.
Neural networks achieve great things, but it is slightly worrying that we don't really know how they work. Inceptionism is an attempt to make neural networks give up their secrets by showing us what t [ ... ]