The complete program for the WPF Version  is:

private void Button_Click(object sender,
                             RoutedEventArgs e)
{
 PXCMSession session =
                  PXCMSession.CreateInstance();
 PXCMSession.ImplVersion version =
                        session.QueryVersion();
 textBox1.Text = version.major.ToString() + "."
                    + version.minor.ToString();


 PXCMSenseManager sm =
                  session.CreateSenseManager();
 sm.EnableStream(
      PXCMCapture.StreamType.STREAM_TYPE_COLOR,
      0, 0);
 sm.EnableStream(
      PXCMCapture.StreamType.STREAM_TYPE_DEPTH,
      0, 0);
 sm.Init();

 pxcmStatus status = sm.AcquireFrame(true);
 PXCMCapture.Sample sample = sm.QuerySample();

 PXCMImage image = sample.color;
 PXCMImage dimage = sample.depth;

 PXCMImage.ImageData data;
 image.AcquireAccess(
    PXCMImage.Access.ACCESS_READ,
    PXCMImage.PixelFormat.PIXEL_FORMAT_RGB32,
    out data);
 WriteableBitmap wbm = data.ToWritableBitmap(0,
                       image.info.width,
                       image.info.height,
                       96.0, 96.0);

 PXCMImage.ImageData data2;
 dimage.AcquireAccess(
   PXCMImage.Access.ACCESS_READ,
   PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH_RAW,
   out data2);
 WriteableBitmap wbm2 = data2.ToWritableBitmap(
                        0,
                        dimage.info.width,
                        dimage.info.height,
                        96.0, 96.0);
 image1.Source = wbm;
 image2.Source = wbm2;
 image.ReleaseAccess(data);
 image.ReleaseAccess(data2);
 sm.ReleaseFrame();
 sm.Close();
 session.Dispose();
}

A Windows Forms Version

If you want to work with Windows Forms then there is nothing really new but there is a small problem when you try to display the 16-bit gray image generated by the depth stream.

The problem is that the Windows Form PictureBox control cannot display a gray level image. 

The solution is to convert the gray level image to a false color image using direct bit manipulation and this is a good place to introduce this technique although there is a lot more to say about it in general. 

To create a Windows Forms version of the program all you need to do is start a new Windows Forms Project, add the two libraries in the same way as before and place a button, textbox and two PictureBoxes on the form. 

The Button's click event handler is exactly the same as the WPF version with minor changes to deal with the different UI controls. 

The first major change is in the conversion of the samples to a bitmap that is suitable for display. For the color image we need:

PXCMImage.ImageData data;
image.AcquireAccess(
    PXCMImage.Access.ACCESS_READ,
    PXCMImage.PixelFormat.PIXEL_FORMAT_RGB32,
    out data);
Bitmap bm = data.ToBitmap(0,
                          image.info.width,
                          image.info.height);

Notice that we are now using ToBitmap because Windows Forms doesn't support WriteableBitmap. 

The real problem is what to do with the depth data. It is supplied as a 16-bit gray level image and, as already stated, the PictureBox control cannot display this. We need to convert it to a false color bitmap. 

One answer, although it is a bit of a cheat, is to simply use the Format16bppRgb555 format which is supported. This treats a 16-bit value so that 5 bits are used for each of R,G and B and the topmost bit is ignored. To perform this conversion we can't simply assign to the PixelFormat property - it is read only. Instead we have to create a new Bitmap object of the correct size and PixelFormat and move the data from the original into it. 

First we need the depth data as a standard 16-bit integer array:

PXCMImage.ImageData data2;
dimage.AcquireAccess(
  PXCMImage.Access.ACCESS_READ,
  PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH_RAW,
  out data2);
Int16 [] buffer = data2.ToShortArray(0,
        dimage.info.width* dimage.info.height);

Next we need a suitable Bitmap of the correct size and format:

Bitmap bm2 = new Bitmap(
              dimage.info.width,
              dimage.info.height,
              PixelFormat.Format16bppRgb555);

Finally we need to lock the data buffer in memory and use the Marshal.copy method to transfer the data into the buffer:

BitmapData bmapdata = bm2.LockBits(
               new Rectangle(0, 0,
                       dimage.info.width,
                       dimage.info.height),
               ImageLockMode.WriteOnly,
               bm2.PixelFormat);
IntPtr ptr = bmapdata.Scan0;
Marshal.Copy(
        buffer,0,
        ptr,
        dimage.info.width *dimage.info.height);
bm2.UnlockBits(bmapdata);

This is a technique that is very useful when you are trying to convert data stored in a standard array i.e. buffer into a viewable image. 

Now we simply display the result:

pictureBox1.Image = bm;
pictureBox2.Image = bm2;

There are better ways to map depth to color but this one has the advantage of being easy to implement. 

The complete click event handler is:

private void button1_Click(object sender,
                              EventArgs e)
{
 PXCMSession session=
              PXCMSession.CreateInstance();
 PXCMSession.ImplVersion version=
                    session.QueryVersion();
 textBox1.Text = version.major.ToString() + "."
                    + version.minor.ToString();
 PXCMSenseManager sm =
                  session.CreateSenseManager();
 sm.EnableStream(
      PXCMCapture.StreamType.STREAM_TYPE_COLOR,
      0,0);
 sm.EnableStream(
      PXCMCapture.StreamType.STREAM_TYPE_DEPTH,
      0,0);
 sm.Init();
 pxcmStatus status = sm.AcquireFrame(true);
 PXCMCapture.Sample sample = sm.QuerySample();
 PXCMImage image = sample.color;
 PXCMImage dimage = sample.depth;

 PXCMImage.ImageData data;
 image.AcquireAccess(
      PXCMImage.Access.ACCESS_READ,
      PXCMImage.PixelFormat.PIXEL_FORMAT_RGB32,
      out data);
 Bitmap bm = data.ToBitmap(0,
                           image.info.width,
                           image.info.height);
 
 PXCMImage.ImageData data2;
 dimage.AcquireAccess(
  PXCMImage.Access.ACCESS_READ,
  PXCMImage.PixelFormat.PIXEL_FORMAT_DEPTH_RAW,
  out data2);
 Int16 [] buffer = data2.ToShortArray(0,
         dimage.info.width* dimage.info.height);

 Bitmap bm2 = new Bitmap(
                  dimage.info.width,
                  dimage.info.height,
                  PixelFormat.Format16bppRgb555);
 BitmapData bmapdata = bm2.LockBits(
           new Rectangle(0, 0,
                      dimage.info.width,
                      dimage.info.height),
                      ImageLockMode.WriteOnly,
                      bm2.PixelFormat);
 IntPtr ptr = bmapdata.Scan0;
 Marshal.Copy(buffer,0,
       ptr,
       dimage.info.width *dimage.info.height);
 bm2.UnlockBits(bmapdata);

 pictureBox1.Image = bm;
 pictureBox2.Image = bm2;

 sm.ReleaseFrame();
 sm.Close();
 session.Dispose();
}

Where Next?

At this point you know how to read a sample from the data streams. However one sample isn't usually enough and so you either have to discover how to use this method to poll the camera in a separate thread or use an callback method of access. We also haven't discovered how to control the basic camera's operation or how to deal with the higher level processing modules.

More soon.

You can download the code for both versions of this program from the CodeBin (note you have to register first).

Real RealSense In C#

1. Getting Started With RealSense In C#
2. Event Streams
3. A Point Cloud - coming soon
4. Hand Tracking - coming soon

Related Articles

BitmapSource: WPF Bitmaps 

WriteableBitmap

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info