Starting the device_Augmented Reality with Kinect-QQ阅读男生玄幻网

上QQ阅读APP看书，第一时间看更新

Starting the device

Now it's time to initialize the Kinect device in our own application. There will be a lot of Kinect API functions for us to use without any preparatory lessons. But don't worry; you will find that most of them are self-explanatory and easy to understand. Also, we will introduce each function and their parameters in the Understanding the code section.

We will continue working on the framework we have just created, so existing code lines will not be listed here again.

Initializing and using Kinect in C++

Now we can try to find and start the Kinect device in our own C++ framework.

Add the following include files:

#include <MSHTML.h>
#include <NuiApi.h>
#include <sstream>

Add the necessary global variables for use in all functions:

INuiSensor* context = NULL;
HANDLE colorStreamHandle = NULL;
HANDLE depthStreamHandle = NULL;
std::string hudText;

Add an initializeKinect() function, which will be called before the GLUT main loop. It returns false if the process fails for any reason.

// Check if there are any Kinect sensors connected with
// current PC and obtain the number
int numKinects = 0;
HRESULT hr = NuiGetSensorCount( &numKinects );
if ( FAILED(hr) || numKinects<=0 ) return false;

// Create the sensor object and set it to context
.
// Here we only use the first device (index 0) we find.
hr = NuiCreateSensorByIndex( 0, &context );
if ( FAILED(hr) ) return false;

// Initialize the sensor with color/depth/skeleton enabled
DWORD nuiFlags = NUI_INITIALIZE_FLAG_USES_SKELETON |
                 NUI_INITIALIZE_FLAG_USES_COLOR |
                 NUI_INITIALIZE_FLAG_USES_DEPTH;
hr = context->NuiInitialize( nuiFlags );
if ( FAILED(hr) ) return false;

// Open color and depth video streams for capturing.
// The resolution is set to 640x480 here.
hr = context->NuiImageStreamOpen(
    NUI_IMAGE_TYPE_COLOR, NUI_IMAGE_RESOLUTION_640x480,
    0, 2, NULL, &colorStreamHandle );
if ( FAILED(hr) ) return false;

hr = context->NuiImageStreamOpen(
    NUI_IMAGE_TYPE_DEPTH, NUI_IMAGE_RESOLUTION_640x480,
    0, 2, NULL, &depthStreamHandle );
if ( FAILED(hr) ) return false;

// Enable skeleton tracking
hr = context->NuiSkeletonTrackingEnable( NULL, 0 );
if ( FAILED(hr) ) return false;
return true;

Add a destroyKinect() function after the main loop in which we just release the sensor object we created before.
```
if ( context )
    context->NuiShutdown();
```

In the main entry, we alter the last few lines as follows:

if ( !initializeKinect() ) return 1;
glutMainLoop();
destroyKinect();
return 0;

The program can compile and run now. But it still produces nothing. We don't know whether Kinect works or not as it shows a blank window. So, next we will add a few lines in update() and render() to print some continuous updated Kinect information.

At the beginning of update(), obtain one color frame and output the current frame number and time stamp value into a string:

NUI_IMAGE_FRAME colorFrame;
HRESULT hr = context->NuiImageStreamGetNextFrame(
    colorStreamHandle, 0, &colorFrame );
if ( SUCCEEDED(hr) )
{
    std::stringstream ss;
    ss << "Frame: " << colorFrame.dwFrameNumber << "  "
       << "Time: " << (double)colorFrame.liTimeStamp.QuadPart * 0.001;
    hudText = ss.str();
    context->NuiImageStreamReleaseFrame(
        colorStreamHandle, &colorFrame );
}

In the render() function, render the text on screen as follows:

// Clear last frame buffer
glClearColor( 0.0f, 0.0f, 0.0f, 0.0f );
glClear( GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT );

// Set up the projection matrix for text display
glMatrixMode( GL_PROJECTION );
glLoadIdentity();
glOrtho( 0.0, 1.0, 0.0, 1.0, -1.0, 1.0 );

glMatrixMode( GL_MODELVIEW );
glLoadIdentity();

// Print the text at the bottom of the window
glRasterPos2f( 0.01f, 0.01f );
glColor4f( 1.0f, 1.0f, 1.0f, 1.0f );
glutBitmapString( GLUT_BITMAP_TIMES_ROMAN_24,
                  (const unsigned char*)hudText.c_str() );

Now, execute the compiled program; you may find it a little slower when you start executing. Be patient until the depth sensor starts to lighten. The screen is still dark but you will find a line of animation text at the bottom-left as shown in the following figure:

A snapshot of the application
If the application directly exits without displaying anything, the initialization process may fail. Add some text before returning false to see the value of hr in initializeKinect(); also check if your Kinect sensor is connected and not used by other programs.

Understanding the code

The following table shows all the Kinect functions we have used as well as descriptions of the important parameters.

Note

All functions and methods start with the prefix "Nui". It is just short for Natural User Interface(NUI).

Now the total process of creating and using Kinect in user applications can be summarized as follows:

Find and create the sensor object.
Initialize the sensor with the required features (image streams and skeleton tracking), and enable these features.
Update every frame to get stream and skeleton data for use.
Release the sensor object when exiting.

Quite simple, isn't it? Note that, we didn't introduce the lines in the update() function here. We will explain that in the next chapter, with more interesting live images shown on the screen instead of a boring line of text.

Additional information

Another interesting and challenging task is to implement a multithreaded version of the initialization and updating of Kinect. In fact, some functions here have already supported such uses by accepting event handles as parameters, including NuiImageStreamOpen() and NuiSkeletonTrackingEnable(). Events will change when new video stream/skeleton frames arrive, so we can listen to them with WaitForMultipleObjects() in a separate thread and then obtain related frame data.