From 3D to 2D

First we need to convert the 3D skeleton co-ordinates into 2D depth image co-ordinates:

float xd, yd;
nui.SkeletonEngine.
 SkeletonToDepthImage(
     location, out xd, out yd);

There is a version of this method that will return a z value as well and another method, DepthImageToSkeleton, that converts in the other direction.

The floats that are returned for the x,y depth co-ordinates vary from 0 to 1 and these have to be converted into pixel co-ordinates:

xd = Math.Max(0,Math.Min(xd * 320,320)); 
yd = Math.Max(0,Math.Min(yd * 240, 240));

Here we are assuming that the size of the depth image is the standard 320x240 pixels. The Max and Mins are needed because it is possible for a location to be outside of the bitmap and so we have to clip the result to fit.

At this point we could use xd,yd to plot the location of the head in a depth image display. However, we want to plot the head position in a video image display, so we have to go through one more co-ordinate conversion. How this works is explained in detail in Part 4.

int x, y;
ImageViewArea iv = new ImageViewArea();
nui.NuiCamera.
 GetColorPixelCoordinatesFromDepthPixel(
  ImageResolution.Resolution640x480, 
  iv, 
  (int)xd, (int)yd, 
  (short)0, 
  out x, out y);

All that really matters, however, is that now we have the x,y pixel co-ordinate of the Head in the video bitmap.  Assuming we have a method called MarkAtxy(x,y) which will draw a cross at the location, we can complete the program with:

 MarkAtxy(x, y);
}

The MarkAtxy method simply sets the pixels in the bitmap to white:

void MarkAtxy(int x,int y)
{
 int i = indexOfPixelinBytes(
           x, y, 
           videoimage.Width, 
           videoimage.BytesPerPixel);
 int stride = videoimage.Width * 
             videoimage.BytesPerPixel;
 for (int j = 0; j < 8; j++)
 {
  videoimage.Bits[i+(j-3)*4]=0xFF;
  videoimage.Bits[i+(j-3)*4+1]=0xFF;
  videoimage.Bits[i+(j-3)*4+2]=0xFF;

  videoimage.Bits[i+(j-3)*stride]=0xFF;
  videoimage.Bits[i+(j-3)*stride+1]=0xFF;
  videoimage.Bits[i+(j-3)*stride+2]=0xFF;
 }
}

To see how this works you need to remember that the bitmap is stored a row at a time and there are four bytes per pixel.  The indexOfPixelinBytes was described in Part 1 and it simply calculates the location of the start of the four bytes corresponding to the pixel at x,y:

int indexOfPixelinBytes(int x, 
              int y,int width, int bpp)
{
 return (x + y * width) * bpp;
}

Now if you run the program you should see a small white cross appear to mark the head of up to two players.

The complete SkeletonFrameReady event handler is:

void SkeletonFrameReady(object sender,
         SkeletonFrameReadyEventArgs e)
{
 if (videoimage.Bits == null) return;
 SkeletonFrame skeletonFrame = 
                       e.SkeletonFrame;
 foreach (SkeletonData data in 
                skeletonFrame.Skeletons)
 {
  if (data.TrackingState==
        SkeletonTrackingState.Tracked )
  {
   Vector location = 
     data.Joints[JointID.Head].Position;

   float xd, yd;
   nui.SkeletonEngine.SkeletonToDepthImage(
             location, out xd, out yd);
   xd = Math.Max(0, Math.Min(xd * 320, 320)); 
   yd = Math.Max(0, Math.Min(yd * 240, 240));

   int x, y;
   ImageViewArea iv = new ImageViewArea();
   nui.NuiCamera.
    GetColorPixelCoordinatesFromDepthPixel(
     ImageResolution.Resolution640x480,
     iv,
    (int)xd, (int)yd, 
    (short)0, 
    out x, out y);
   MarkAtxy(x, y);
  }
  Bitmap bmap = PImageToBitmap(videoimage);
  pictureBox1.Image = bmap;
 }
}

Once you have seen how to track a head the rest of the body is no problem and you will be surprised at how rarely you actually need to draw a skeleton corresponding to where the player is.

You can download the code for the Windows Forms version of this program from the CodeBin (note you have to register first).

Articles in this Series

1. Getting started with Microsoft Kinect SDK
2. Depth
3. Player index
4. Depth and Video space
5. Skeletons (this article)
   
6. The Full Skeleton

To be informed about new articles on I Programmer, subscribe to the RSS feed, follow us on Google+, Twitter, Linkedin or Facebook or sign up for our weekly newsletter.