A new Kinect SDK brings with it some interesting new high-level features and, at last, HTML support.
The Kinect SDK was updated in March of this year which is a fairly fast rate of turnover. The good news is that the runtime components are still backward compatible.
Most of the new features are extensions to high-level facilities such as Kinect Fusion, which was introduced in the last release of the SDK. Perhaps the most far reaching change is the addition of HTML and JavaScript support.
Kinect Data Streams Webserver New web components give HTML5 applications access to Kinect data. This is intended to allow HTML5 applications running in a browser to connect to the sensor through a server running on the local machine. This opens up new possibilities including WinRT JavaScript apps that make use of the Kinect. Obviously as the data is provided by a webserver you can use whatever language you feel like on the client. The interaction is via a REST based API. The webserver component is a template that can be used as-is or modified as needed. Also included is a sample that demonstrates implementing Kinect-enabled buttons, simple user engagement, and the use of a background removal stream in HTML5.
New background removal. An API removes the background behind the active user so that it can be replaced with an artificial background. This is the sort of application that you would generally need to take a photo against a green or other known color screen and then use this to filter out the background. However for a depth field camera what is in the foreground is very easy to determine - it's all a matter of depth.
Realistic color capture with Kinect Fusion. Kinect Fusion is a 3D model extraction application introduced as part of the previous API. You can use it to build a 3D model of a scene simply by scanning it with the Kinect. A new Kinect Fusion API scans the color of the scene along with the depth information so that it can capture the color of the object along with its three-dimensional (3D) model. The API also produces a texture map for the mesh created from the scan. This feature provides a full fidelity 3D model of a scan not just a bare wireframe.
A full color textured 3D model (from the Extreme Windows Blog)
Improved tracking robustness with Kinect Fusion. This algorithm makes it easier to scan a scene. With this update, Kinect Fusion is better able to maintain its lock on the scene as the camera position moves, yielding a more reliable and consistent scanning.
There are also two very interesting new samples:
Multiple-sensor Kinect Fusion sample. This shows how to use two sensors simultaneously to scan a person or object from both sides—making it possible to construct a 3D model without having to move the sensor or the object.
Adaptive UI sample. This sample demonstrates how to build an application that adapts itself depending on the distance between the user and the screen—from gesturing at a distance to touching a touchscreen. The algorithm in this sample uses the physical dimensions and positions of the screen and sensor to determine the best ergonomic position on the screen for touch controls as well as ways the UI can adapt as the user approaches the screen or moves further away from it. As a result, the touch interface and visual display adapt to the user’s position and height, which enables users to interact with large touch screen displays comfortably. The display can also be adapted for more than one user.
For core improvements to the performance of the Kinect we are all waiting for the Xbox One version of the PC Kinect. The same features provided by its SDK would give the Kinect a chance at application where quality was also an issue.
One of the big problems with the future of the Kinect, and the Kinect SDK in particular, is Microsoft's shift away from core .NET to WinRT. The easiest way to work with the Kinect is to use C# or Visual Basic with WPF. Unfortunately WPF's approach to 3D only goes so far and, without another version of WPF on the horizon, it is difficult to feel secure using it for a complex application. The Kinect SDK is one of the few Microsoft SDKs that is still advocating and largely reliant the use of WPF.
The clamour for safe programming languages seems to be growing and becoming official. We have known for a while that C and C++ are dangerous languages so why has it become such an issue now and is it [ ... ]
IBM is launching a new set of AI software engineering agents designed to autonomously resolve GitHub issues. The agents are being made available in an open-source licensing model.