Kinect

It is a set of sensors developed by Microsoft:

A depth sensor (512 x 424 pixels)
A color camera sensor (1920 x 1080 pixels)
An array of four microphones

It has the capability to track the skeleton image of one or more people moving within the field of view of the sensors. It also provides acoustic noise suppression, echo cancellation and beam formation.

Es un conjunto de sensores desarrollado por Microsoft:

Un sensor de profundidad (512 x 424 pixeles)
Una cámara de color (1920 x 1080 pixeles)
Un arreglo con cuatro micrófonos

Este tiene la capacidad de seguir el esqueleto en una imagen de una o más personas en movimiento dentro del campo de visión de los sensores. Este también proporciona supresión de ruido acústico, cancelación de eco y formación de haces.

Problem 1

Download and Install Kinect for Windows SDK v2.0_1409. After installing the SDK, verify the location where this was installed (C:\Program Files\Microsoft SDKs\Kinect\v2.0-1409).

Descargue e instale Kinect for Windows SDK v2.0_1409. Después de instalar el SDK, verifique la ubicación dónde este se instaló (C:\Program Files\Microsoft SDKs\Kinect\v2.0-1409).

MSDOS: cmd.exe

C:\Program Files\Microsoft SDKs\Kinect\v2.0_1409>dir
Volume in drive C is OS
Volume Serial Number is 92E1-DCCF

Directory of C:\Program Files\Microsoft SDKs\Kinect\v2.0_1409

<DIR> Assemblies
<DIR> bin
<DIR> inc
<DIR> Lib
<DIR> Redist
<DIR> Samples
109,462 SDKEula.rtf
<DIR> Tools
1 File(s) 109,462 bytes
9 Dir(s) 52,712,239,104 bytes free

C:\Program Files\Microsoft SDKs\Kinect\v2.0_1409>

Problem 2

Create a Wintempla Window application called MotionX to test the Kinect for Windows. After creating the project, open Solution Explorer, select the MotionX project and right click to open the context menu. Then, select Properties. Once the Property Pages dialog is open, select All configurations and modify the Include Directories and the Library Directories as shown below. If you compile in 32 bits, use the X86 folder for the Library Directories. Otherwise use the x64 folder. If you get compilations errors when using 64 bits, compile it on 32 bits.

Cree una aplicación de Ventana llamada MotionX para probar el Kinect para Windows. Después de crear el projecto, abra Solution Explorer, seleccione el proyecto MotionX y haga clic con el botón derecho del ratón para abrir el menú de contexto. Entonces seleccione Properties. Una vez que diálogo de Property Pages este abierto, seleccione All configurations y modifique Include Directories y Library Directories cómo se muestra debajo. Si usted compila en 32 bits, use la carpeta de x86 para el Library Directories. De otra forma use la carpeta x64. Si usted obtiene errores de compilación cuando se usan 64 bits, compile el programa en 32 bits.

Step A

Edit the stdafx.h file to include the file Kinect.h.

Edite el archivo stdafx.h para incluir el archivo Kinect.h

stdafx.h

. . .
#include "Wintempla.h"
#include "WintemplaWin.h"
//#include "DX11.h"
using namespace std;
//_____________________________________________________________ 1. Kinect
#include <Kinect.h>
#pragma comment(lib, "kinect20.lib")
_COM_SMARTPTR_TYPEDEF(IKinectSensor, __uuidof(IKinectSensor));
_COM_SMARTPTR_TYPEDEF(IFrameDescription, __uuidof(IFrameDescription));
_COM_SMARTPTR_TYPEDEF(IColorCameraSettings, __uuidof(IColorCameraSettings));
_COM_SMARTPTR_TYPEDEF(ICoordinateMapper, __uuidof(ICoordinateMapper));
//_____________________________________________________________ 2. Kinect frame source
_COM_SMARTPTR_TYPEDEF(IColorFrameSource, __uuidof(IColorFrameSource));
_COM_SMARTPTR_TYPEDEF(IDepthFrameSource, __uuidof(IDepthFrameSource));
//_____________________________________________________________ 3. Kinect frame reader
_COM_SMARTPTR_TYPEDEF(IColorFrameReader, __uuidof(IColorFrameReader));
_COM_SMARTPTR_TYPEDEF(IDepthFrameReader, __uuidof(IDepthFrameReader));
_COM_SMARTPTR_TYPEDEF(IMultiSourceFrameReader, __uuidof(IMultiSourceFrameReader));
//_____________________________________________________________ 4. Kinect frame
_COM_SMARTPTR_TYPEDEF(IColorFrame, __uuidof(IColorFrame));
_COM_SMARTPTR_TYPEDEF(IDepthFrame, __uuidof(IDepthFrame));
_COM_SMARTPTR_TYPEDEF(IMultiSourceFrame, __uuidof(IMultiSourceFrame));
_COM_SMARTPTR_TYPEDEF(IDepthFrameReference, __uuidof(IDepthFrameReference));
_COM_SMARTPTR_TYPEDEF(IColorFrameReference, __uuidof(IColorFrameReference));
. . .

Step B

Open Wintempla, double click on the GUI editor to open the Window properties. On the event tab, check the events: Idle and Size.

Abra Wintempla, haga clic doble en el editor de GUI para abrir las propiedades de la ventana. En la pestaña de Events, marque Idle y Size.

Step B

Edit the MotionX.h file as shown.

Edite el archivo MotionX.h como se muestra.

Motion.h

#pragma once //______________________________________ MotionX.h
#include "Resource.h"

class MotionX: public Win::Window
{
public:
     MotionX()
     {
          ::CoInitialize(NULL);
     }
     ~MotionX()
     {
          if (kinect != NULL)
          {
               kinect->Close();
               kinect = NULL;
          }
          ::CoUninitialize();
     }
     CG::DIBitmap bitmap;
     CG::DDBitmap backBuffer;
     Sys::Stopwatch stopwatch;
     IKinectSensorPtr kinect;
     IColorFrameReaderPtr colorFrameReader;
     const wchar_t * GetClassName(){return L"MotionX";}
     void RenderScene(CG::Gdi& gdi);
protected:
     . . .
};

Step C

Edit the MotionX.cpp file as shown. Note that we have made several changes in the wWinMain function (Use a null brush for the window background and use the MessageLoopIdle instead of MessageLoop.

Edite el archivo MotionX.cpp como se muestra. Observe que hemos hecho varios cambios a la función wWinMain (Usar una brocha nula para pintar el fondo de la ventana y usar el MessageLoopIdle en lugar de MessageLoop.

MotionX.cpp

#include "stdafx.h" //________________________________________ MotionX.cpp
#include "MotionX.h"

int APIENTRY wWinMain(HINSTANCE hInstance, HINSTANCE , LPTSTR cmdLine, int cmdShow){
     MotionX app;
     app.CreateMainWindow(L"MotionX", cmdShow, IDI_MotionX, NULL, (HBRUSH)::GetStockObject(NULL_BRUSH), hInstance);
     return app.MessageLoopIdle(IDC_MotionX);
}

void MotionX::Window_Open(Win::Event& e)
{
     HRESULT hr;
     Com::Exception ex;
     IColorFrameSourcePtr colorFrameSource;
     IFrameDescriptionPtr frameDescription;
     BOOLEAN isOpen = false;
     int frameWidth = 0;
     int frameHeight = 0;
     unsigned int bytesPerPixel = 0;
     try
     {
          //____________________________________________________ 1. GetDefaultKinectSensor
          hr = ::GetDefaultKinectSensor(&kinect);
          ex.ok(L"GetDefaultKinectSensor", hr);
          if (kinect == NULL) ex.ThrowNullPointer(L"GetDefaultKinectSensor");
          //____________________________________________________ 2. kinect->Open()
          hr = kinect->Open();
          ex.ok(L"kinect->Open", hr);
          //____________________________________________________ 3. kinect->get_IsOpen()
          hr = kinect->get_IsOpen(&isOpen);
          ex.ok(L"kinect->get_IsOpen", hr);
          if (isOpen == false) ex.Throw(L"Kinect is not open", hr);
          //____________________________________________________ 4. kinect->get_ColorFrameSource()
          hr = kinect->get_ColorFrameSource(&colorFrameSource);
          ex.ok(L"kinect->get_ColorFrameSource", hr);
          //____________________________________________________ 5. colorFrameSource->OpenReader()
          hr = colorFrameSource->OpenReader(&colorFrameReader);
          ex.ok(L"colorFrameSource->OpenReader", hr);
          //____________________________________________________ 6. colorFrameSource->CreateFrameDescription()
          hr = colorFrameSource->CreateFrameDescription(ColorImageFormat::ColorImageFormat_Bgra, &frameDescription);
          ex.ok(L"colorFrameSource->CreateFrameDescription", hr);
          //____________________________________________________ 7. frameDescription->get_Width()
          hr = frameDescription->get_Width(&frameWidth);
          ex.ok(L"frameDescription->get_Width", hr);
          //____________________________________________________ 8. frameDescription->get_Height()
          hr = frameDescription->get_Height(&frameHeight);
          ex.ok(L"frameDescription->get_Height", hr);
          //____________________________________________________ 9. frameDescription->get_BytesPerPixel()
          hr = frameDescription->get_BytesPerPixel(&bytesPerPixel);
          ex.ok(L"frameDescription->get_BytesPerPixel", hr);
          //____________________________________________________ 10. Memory allocation
          bitmap.Create(8 * bytesPerPixel, 0, frameWidth, frameHeight, true);
     }
     catch (Com::Exception& exc)
     {
          exc.Display(hWnd, L"MotionX");
     }
     catch (_com_error exc)
     {
          Com::Exception::Display(hWnd, exc, L"MotionX");
     }
}

void MotionX::RenderScene(CG::Gdi& gdi)
{
     RECT rect = { 0, 0, width, height };
     //_____________________________________________________________ 1. Paint white background
     gdi.Rectangle(rect);
     //_____________________________________________________________ 2. Draw Kinect image
     gdi.DrawBitmap(bitmap, rect, true);
}

void MotionX::Window_Idle(Win::Event& e)
{
     //_____________________________________________________________ 1. Get image from Kinect
     IColorFrame* colorFrame = NULL;
     if (FAILED(colorFrameReader->AcquireLatestFrame(&colorFrame))) return;
     if (FAILED(colorFrame->CopyConvertedFrameDataToArray(bitmap.GetBitsByteCount(), bitmap.GetBits(), ColorImageFormat_Bgra))) return;
     if (colorFrame != NULL) colorFrame->Release();
     //_____________________________________________________________ 2. Camera settings
     //IColorCameraSettingsPtr cameraSettings;
     //colorFrame->get_ColorCameraSettings(&cameraSettings);
     //float gain;
     //TIMESPAN exposure;
     //cameraSettings->get_Gain(&gain);
     //cameraSettings->get_ExposureTime(&exposure);
     //_____________________________________________________________ 3. Render on backBuffer
     RECT rcPaint = { 0, 0, width, height };
     CG::Gdi gdi(backBuffer, rcPaint, false);
     RenderScene(gdi);
     //_____________________________________________________________ 4. Present backBuffer to screen
     HDC screen = ::GetDC(hWnd);
     ::BitBlt(screen, 0, 0, width, height, backBuffer.GetBitmapDC(),
          0, 0, SRCCOPY);
     ::ReleaseDC(hWnd, screen);
     //_____________________________________________________________ 5. Slow down the frames per second
     static float timeSec = 0;
     while (stopwatch.GetSeconds() - timeSec < 0.016f) Sleep(1); //while (FPS>62.5)
     timeSec = (float)stopwatch.GetSeconds();
}

void MotionX::GetRegisterClass(WNDCLASSEX& wcex)
{
     wcex.style = CS_CLASSDC;
     wcex.hbrBackground = NULL;
}

void MotionX::Window_Size(Win::Event& e)
{
     Win::Window::Window_Size(e);
     backBuffer.CreateCompatible(hWnd, width, height);
}

Problem 3

Create a Wintempla Window application called Deep to test the depth camera of the Kinect for Windows. You need to perform the same configuration performed in the previous problem. You also need to check the events: Idle and Size. Finally, do not forget to edit the stdafx.h file and modify the wWinMain function.

Cree una aplicación de Ventana de DirectX llamada Deep para probar la cámara de profundidad del Kinect para Windows. Usted necesita realizar la misma configuración hecha en el problema anterior. También necesita checar los eventos: Idle y Size. Finalmente, no se olvide de editar el archivo stdafx.h y modificar la función wWinMain.

Deep.h

#pragma once //______________________________________ Deep.h
#include "Resource.h"
class Deep: public Win::Window
{
public:
     Deep()
     {
          ::CoInitialize(NULL);
     }
     ~Deep()
     {
          if (kinect != NULL)
          {
               kinect->Close();
               kinect = NULL;
          }
          ::CoUninitialize();
     }
     CG::DIBitmap bitmap;
     CG::DDBitmap backBuffer;
     Sys::Stopwatch stopwatch;
     IKinectSensorPtr kinect;
     IDepthFrameReaderPtr depthFrameReader;
     const wchar_t * GetClassName(){ return L"Deep"; }
     void RenderScene(CG::Gdi& gdi);
protected:
     void GetRegisterClass(WNDCLASSEX& wcex);
     . . .
};

Deep.cpp

#include "stdafx.h" //________________________________________ Deep.cpp
#include "Deep.h"

int APIENTRY wWinMain(HINSTANCE hInstance, HINSTANCE , LPTSTR cmdLine, int cmdShow){
     Deep app;
     app.CreateMainWindow(L"Deep", cmdShow, IDI_Deep, NULL, (HBRUSH)::GetStockObject(NULL_BRUSH), hInstance);
     return app.MessageLoopIdle(IDC_Deep);
}

void Deep::Window_Open(Win::Event& e)
{
     HRESULT hr;
     Com::Exception ex;
     IDepthFrameSourcePtr depthFrameSource;
     IFrameDescriptionPtr frameDescription;
     BOOLEAN isOpen = false;
     int frameWidth = 0;
     int frameHeight = 0;
     unsigned int bytesPerPixel = 0;
     try
     {
          //____________________________________________________ 1. GetDefaultKinectSensor
          hr = ::GetDefaultKinectSensor(&kinect);
          ex.ok(L"GetDefaultKinectSensor", hr);
          if (kinect == NULL) ex.ThrowNullPointer(L"GetDefaultKinectSensor");
          //____________________________________________________ 2. kinect->Open()
          hr = kinect->Open();
          ex.ok(L"kinect->Open", hr);
          //____________________________________________________ 3. kinect->get_IsOpen()
          hr = kinect->get_IsOpen(&isOpen);
          ex.ok(L"kinect->get_IsOpen", hr);
          if (isOpen == false) ex.Throw(L"Kinect is not open", hr);
          //____________________________________________________ 4. kinect->get_DepthFrameSource()
          hr = kinect->get_DepthFrameSource(&depthFrameSource);
          ex.ok(L"kinect->get_DepthFrameSource", hr);
          //____________________________________________________ 5. depthFrameSource->OpenReader()
          hr = depthFrameSource->OpenReader(&depthFrameReader);
          ex.ok(L"depthFrameSource->OpenReader", hr);
          //____________________________________________________ 6. depthFrameSource->get_FrameDescription
          hr = depthFrameSource->get_FrameDescription(&frameDescription);
          ex.ok(L"depthFrameSource->get_FrameDescription", hr);
          //____________________________________________________ 7. frameDescription->get_Width()
          hr = frameDescription->get_Width(&frameWidth);
          ex.ok(L"frameDescription->get_Width", hr);
          //____________________________________________________ 8. frameDescription->get_Height()
          hr = frameDescription->get_Height(&frameHeight);
          ex.ok(L"frameDescription->get_Height", hr);
          //____________________________________________________ 9. frameDescription->get_BytesPerPixel()
          hr = frameDescription->get_BytesPerPixel(&bytesPerPixel);
          ex.ok(L"frameDescription->get_BytesPerPixel", hr);
          //____________________________________________________ 10. Memory allocation
          bitmap.Create(8, 256, frameWidth, frameHeight, true);
          bitmap.SetGrayColorTable(true);
     }
     catch (Com::Exception& exc)
     {
          exc.Display(hWnd, L"MotionX");
     }
     catch (_com_error exc)
     {
          Com::Exception::Display(hWnd, exc, L"MotionX");
     }
}

void Deep::RenderScene(CG::Gdi& gdi)
{
     RECT rect ={0, 0, width, height};
     //_____________________________________________________________ 1. Paint white background
     gdi.Rectangle(rect);
     //_____________________________________________________________ 2. Draw Kinect depth image
     gdi.DrawBitmap(bitmap, rect, true);
}

void Deep::Window_Idle(Win::Event& e)
{
     //_____________________________________________________________ 1. Get depth image from Kinect
     IDepthFrame* depthFrame = NULL;
     if (FAILED(depthFrameReader->AcquireLatestFrame(&depthFrame))) return;
     unsigned int length = 0;
     unsigned __int16* buffer = NULL;
     if (FAILED(depthFrame->AccessUnderlyingBuffer(&length, &buffer))) return;
     const int bitmapWidth = bitmap.GetWidth();
     const int bitmapHeight = bitmap.GetHeight();
     int row, col;
     unsigned char* bits = bitmap.GetBits();
     unsigned char* p;
     size_t rowOffset = bitmap.GetBytesRowSize();
     //_____________________________________________________________ 2. Get maximum and minimum depth
     int maximum = buffer[0];
     int minimum = buffer[0];
     for (unsigned int i = 0; i < length; i++)
     {
          if (buffer[i] > maximum) maximum = buffer[i];
          if (buffer[i] < minimum) minimum = buffer[i];
     }
     const int delta = maximum - minimum;
     //_____________________________________________________________ 3. Convert from depth to color
     for (row = 0; row < bitmapHeight; row++)
     {
          p = bits + row*rowOffset;
          for (col = 0; col < bitmapWidth; col++, p++)
          {
               *p = (unsigned char)((256*((*buffer) - minimum))/ delta);
               buffer++;
          }
     }
     depthFrame->Release();
     //_____________________________________________________________ 4. Render on backBuffer
     RECT rcPaint ={0, 0, width, height};
     CG::Gdi gdi(backBuffer, rcPaint, false);
     RenderScene(gdi);
     //_____________________________________________________________ 5. Present backBuffer to screen
     HDC screen = ::GetDC(hWnd);
     ::BitBlt(screen, 0, 0, width, height, backBuffer.GetBitmapDC(),
          0, 0, SRCCOPY);
     ::ReleaseDC(hWnd, screen);
     //_____________________________________________________________ 6. Slow down the frames per second
     static float timeSec = 0;
     while (stopwatch.GetSeconds() - timeSec < 0.016f) Sleep(1); //while (FPS>62.5)
     timeSec = (float)stopwatch.GetSeconds();
}

void Deep::GetRegisterClass(WNDCLASSEX& wcex)
{
     wcex.style = CS_CLASSDC;
     wcex.hbrBackground = NULL;
}

void Deep::Window_Size(Win::Event& e)
{
     Win::Window::Window_Size(e);
     backBuffer.CreateCompatible(hWnd, width, height);
}

Kinect coordinate mapper

This mapper allows combining the depth frame with the color frame to create a point cloud (or other 3D primitives). The figure below illustrates how the coordinate mapper is used to extract two arrays (a CameraSpacePoint array and a ColorSpacePoint array).

Este mapeador permite combinar la imagen de profundidad con la imagen de color para crear una nube de puntos (u otra primitiva 3D). La figura de abajo ilustra cómo el mapeador de coordenadas es usado para extraer dos arreglos (un arreglo de CameraSpacePoint y un arreglo de ColorSpacePoint).

Problem 4

Create a DirectX application called Human to display a Point Cloud produced by a Microsoft Kinect. After creating the project, open Solution Explorer, select the Human project and right click to open the context menu. Then, select Properties. Once the Property Pages dialog is open, select All configurations and modify the Include Directories and the Library Directories as shown below. If you compile in 32 bits, use the X86 folder for the Library Directories. Otherwise use the x64 folder. If you get compilations errors when using 64 bits, compile it on 32 bits.

Cree una aplicación de DirectX llamada Human para mostrar una Nube de Puntos producida por un Kinect de Microsoft. Después de crear el projecto, abra Solution Explorer, seleccione el proyecto Human y haga clic con el botón derecho del ratón para abrir el menú de contexto. Entonces seleccione Properties. Una vez que diálogo de Property Pages este abierto, seleccione All configurations y modifique Include Directories y Library Directories cómo se muestra debajo. Si usted compila en 32 bits, use la carpeta de x86 para el Library Directories. De otra forma use la carpeta x64. Si usted obtiene errores de compilación cuando se usan 64 bits, compile el programa en 32 bits.

Step A

Edit the stdafx.h file to include the file Kinect.h.

Edite el archivo stdafx.h para incluir el archivo Kinect.h

stdafx.h

. . .
#include "Wintempla.h"
#include "WintemplaWin.h"
#include "DX11.h"
using namespace std;
//_____________________________________________________________ 1. Kinect
#include <Kinect.h>
#pragma comment(lib, "kinect20.lib")
_COM_SMARTPTR_TYPEDEF(IKinectSensor, __uuidof(IKinectSensor));
_COM_SMARTPTR_TYPEDEF(IFrameDescription, __uuidof(IFrameDescription));
_COM_SMARTPTR_TYPEDEF(IColorCameraSettings, __uuidof(IColorCameraSettings));
_COM_SMARTPTR_TYPEDEF(ICoordinateMapper, __uuidof(ICoordinateMapper));
//_____________________________________________________________ 2. Kinect frame source
_COM_SMARTPTR_TYPEDEF(IColorFrameSource, __uuidof(IColorFrameSource));
_COM_SMARTPTR_TYPEDEF(IDepthFrameSource, __uuidof(IDepthFrameSource));
//_____________________________________________________________ 3. Kinect frame reader
_COM_SMARTPTR_TYPEDEF(IColorFrameReader, __uuidof(IColorFrameReader));
_COM_SMARTPTR_TYPEDEF(IDepthFrameReader, __uuidof(IDepthFrameReader));
_COM_SMARTPTR_TYPEDEF(IMultiSourceFrameReader, __uuidof(IMultiSourceFrameReader));
//_____________________________________________________________ 4. Kinect frame
_COM_SMARTPTR_TYPEDEF(IColorFrame, __uuidof(IColorFrame));
_COM_SMARTPTR_TYPEDEF(IDepthFrame, __uuidof(IDepthFrame));
_COM_SMARTPTR_TYPEDEF(IMultiSourceFrame, __uuidof(IMultiSourceFrame));
_COM_SMARTPTR_TYPEDEF(IDepthFrameReference, __uuidof(IDepthFrameReference));
_COM_SMARTPTR_TYPEDEF(IColorFrameReference, __uuidof(IColorFrameReference));
. . .

Step A

Add a new class called PointCloud to store and process the Kinect point cloud from.

Agregue una clase nueva llamada PointCloud para almacenar y procesar la nube de puntos del Kinect.

PointCloud.h

#pragma once
class PointCloud : public DX11::ColorVertexBuffer
{
public:
     PointCloud();
     ~PointCloud();
     DirectX::XMMATRIX world;
     //_______________________________________________________________ 1. Kinect
     IKinectSensorPtr kinect;
     ICoordinateMapperPtr coordinateMapper;
     IMultiSourceFrameReaderPtr multiSourceFrameReader;
     void KinectSetup(HWND hWnd);
     bool KinectGetFrame(HWND hWnd, ID3D11Device* device, ID3D11DeviceContext* deviceContext);
     CameraSpacePoint* cameraSpacePoint = NULL;
     ColorSpacePoint* colorSpacePoint = NULL;
     unsigned char* colorImagePixels = NULL;
};

PointCloud.cpp

#include "stdafx.h"
#include "PointCloud.h"

PointCloud::PointCloud()
{
     world = DirectX::XMMatrixIdentity();
}

PointCloud::~PointCloud()
{
     if (kinect != NULL)
     {
          kinect->Close();
          kinect = NULL;
     }
     if (cameraSpacePoint != NULL) delete[] cameraSpacePoint;
     if (colorSpacePoint != NULL) delete[] colorSpacePoint;
     if (colorImagePixels != NULL) delete[] colorImagePixels;
}

void PointCloud::KinectSetup(HWND hWnd)
{
     HRESULT hr;
     Com::Exception ex;
     IColorFrameSourcePtr colorFrameSource;
     IFrameDescriptionPtr frameDescription;
     BOOLEAN isOpen = false;
     int frameWidth = 0;
     int frameHeight = 0;
     unsigned int bytesPerPixel = 0;
     try
     {
          //____________________________________________________ 1. GetDefaultKinectSensor
          hr = ::GetDefaultKinectSensor(&kinect);
          ex.ok(L"GetDefaultKinectSensor", hr);
          if (kinect == NULL) ex.ThrowNullPointer(L"GetDefaultKinectSensor");
          //____________________________________________________ 2. get_CoordinateMapper
          hr = kinect->get_CoordinateMapper(&coordinateMapper);
          ex.ok(L"kinect->get_CoordinateMapper", hr);
          //____________________________________________________ 3. kinect->Open()
          hr = kinect->Open();
          ex.ok(L"kinect->Open", hr);
          //____________________________________________________ 4. kinect->get_IsOpen()
          hr = kinect->get_IsOpen(&isOpen);
          ex.ok(L"kinect->get_IsOpen", hr);
          if (isOpen == false) ex.Throw(L"Kinect is not open", hr);
          //____________________________________________________ 5. kinect->OpenMultiSourceFrameReader()
          hr = kinect->OpenMultiSourceFrameReader(FrameSourceTypes::FrameSourceTypes_Depth | FrameSourceTypes::FrameSourceTypes_Color,
               &multiSourceFrameReader);
          ex.ok(L"kinect->OpenMultiSourceFrameReader", hr);
     }
     catch (Com::Exception& exc)
     {
          exc.Display(hWnd, L"Human");
     }
     catch (_com_error exc)
     {
          Com::Exception::Display(hWnd, exc, L"Human");
     }
}

bool PointCloud::KinectGetFrame(HWND hWnd, ID3D11Device* device, ID3D11DeviceContext* deviceContext)
{
     static int colorWidth = 0;
     static int colorHeight = 0;
     static int depthWidth = 0;
     static int depthHeight = 0;
     //____________________________________________________ 1. AcquireLatestFrame
     IMultiSourceFramePtr multiSourceFrame;
     if (FAILED(multiSourceFrameReader->AcquireLatestFrame(&multiSourceFrame))) return false;
     //____________________________________________________ 2. get_DepthFrameReference
     IDepthFrameReferencePtr depthFrameReference;
     if (FAILED(multiSourceFrame->get_DepthFrameReference(&depthFrameReference))) return false;
     //____________________________________________________ 3. Acquire depthFrame
     IDepthFramePtr depthFrame = NULL;
     if (FAILED(depthFrameReference->AcquireFrame(&depthFrame))) return false;
     //depthFrameReference = NULL;
     //____________________________________________________ 4. Get depthWidth and depthHeight, then setup and create the DX11::ColorVertexBuffer
     if (depthWidth == 0 && depthHeight == 0)
     {
          IFrameDescriptionPtr frameDescription;
          if (FAILED(depthFrame->get_FrameDescription(&frameDescription))) return false;
          if (FAILED(frameDescription->get_Width(&depthWidth))) return false;
          if (FAILED(frameDescription->get_Height(&depthHeight))) return false;
          const int pointCount = depthWidth*depthHeight;
          if (this->Setup(pointCount, pointCount, D3D11_USAGE_DYNAMIC, D3D_PRIMITIVE_TOPOLOGY_POINTLIST) == false) return false;
          for (size_t i = 0; i < this->GetIndexCount(); i++) index[i] = i;
          if (this->Create(hWnd, device) == false) return false;
     }
     //____________________________________________________ 5. depthFrame->AccessUnderlyingBuffer
     unsigned int buffer_len = 0;
     unsigned __int16* buffer = NULL;
     if (FAILED(depthFrame->AccessUnderlyingBuffer(&buffer_len, &buffer))) return false;
     //____________________________________________________ 6. mapper->MapDepthFrameToCameraSpace (X, Y, Z)
     if (cameraSpacePoint == NULL) cameraSpacePoint = new CameraSpacePoint[buffer_len];
     if (FAILED(coordinateMapper->MapDepthFrameToCameraSpace(buffer_len, buffer, buffer_len, cameraSpacePoint))) return false;
     //____________________________________________________ 7. mapper->MapDepthFrameToColorSpace (X, Y)
     if (colorSpacePoint == NULL) colorSpacePoint = new ColorSpacePoint[buffer_len];
     if (FAILED(coordinateMapper->MapDepthFrameToColorSpace(buffer_len, buffer, buffer_len, colorSpacePoint))) return false;
     //depthFrame = NULL;
     //____________________________________________________ 8. multiSourceFrame->get_ColorFrameReference
     IColorFrameReferencePtr colorFrameReference;
     if (FAILED(multiSourceFrame->get_ColorFrameReference(&colorFrameReference))) return false;
     //____________________________________________________ 9. colorFrameReference->AcquireFrame
     IColorFramePtr colorFrame = NULL;
     if (FAILED(colorFrameReference->AcquireFrame(&colorFrame))) return false;
     //colorFrameReference = NULL;
     //____________________________________________________ 10. Get colorWidth and colorHeight
     if (colorWidth == 0 && colorHeight == 0)
     {
          IFrameDescriptionPtr frameDescription;
          if (FAILED(colorFrame->get_FrameDescription(&frameDescription))) return false;
          if (FAILED(frameDescription->get_Width(&colorWidth))) return false;
          if (FAILED(frameDescription->get_Height(&colorHeight))) return false;
     }
     //____________________________________________________ 11. colorFrame->CopyConvertedFrameDataToArray
     const int colorPixelByteCount = 4 * colorWidth*colorHeight;
     if (colorImagePixels == NULL) colorImagePixels = new unsigned char[colorPixelByteCount];
     if (FAILED(colorFrame->CopyConvertedFrameDataToArray(colorPixelByteCount, colorImagePixels, ColorImageFormat_Rgba))) return false;
     //colorFrame = NULL;
     //____________________________________________________ 12. create Color vertex
     if (this->GetVertexCount() > 0)
     {
          unsigned int i = 0, j = 0;
          int k;
          int ci;
          usedIndexCount = 0;
          //_________________________________________ 12.1. Compute usedIndex and vertexes
          for (i = 0; i < 424; i++)
          {
               for (j = 0; j < 512; j++)
               {
                    k = j + 512*i;
                    if (-10000.0f < cameraSpacePoint[k].X && cameraSpacePoint[k].X < 10000.0f
                         && -10000.0f < cameraSpacePoint[k].Y && cameraSpacePoint[k].Y < 10000.0f
                         && -10000.0f < cameraSpacePoint[k].Z && cameraSpacePoint[k].Z < 10000.0f
                         && 0.0f <= colorSpacePoint[k].X && colorSpacePoint[k].X < 1920.0f
                         && 0.0f <= colorSpacePoint[k].Y && colorSpacePoint[k].Y < 1080.0f)
                    {
                         this->vertex[usedIndexCount].position.x = cameraSpacePoint[k].X;
                         this->vertex[usedIndexCount].position.y = cameraSpacePoint[k].Y;
                         this->vertex[usedIndexCount].position.z = cameraSpacePoint[k].Z;
                         //
                         ci = (int)colorSpacePoint[k].X + 1920*(int)colorSpacePoint[k].Y;
                         this->vertex[usedIndexCount].color.w = 0.0f;
                         this->vertex[usedIndexCount].color.x = colorImagePixels[4*ci + 0]/255.0f;
                         this->vertex[usedIndexCount].color.y = colorImagePixels[4*ci + 1]/255.0f;
                         this->vertex[usedIndexCount].color.z = colorImagePixels[4*ci + 2]/255.0f;
                         //
                         usedIndexCount++;
                    }
               }
          }
          if (this->DynamicUpdate(deviceContext, true, false) == false) return false;
     }
     return true;
}

Step B

Edit the files Human.h and Human.cpp as shown below.

Edite los archivos Human.h y Human.cpp cómo se muestra debajo.

Human.h

#pragma once //______________________________________ Human.h
#include "Resource.h"
#include "PointCloud.h"

class Human: public DX11::Window
{
public:
     Human()
     {
          ::CoInitialize(NULL);
     }
     ~Human()
     {
          ::CoUninitialize();
     }
     //____________________________________________________ 1. Application
     PointCloud pointCloud;
     . . .
};

Human.cpp

. . .
void Human::Window_Open(Win::Event& e)
{
     . . .
     //________________________________________________________________ 4. Setup the camera
     camera.position.x = 0.0f;
     camera.position.y = 0.0f;
     camera.position.z = -1.0f;
     //
     camera.lookAt.x = 0.0f;
     camera.lookAt.y = 0.0f;
     camera.lookAt.z = 1.0f;
     //
     camera.SetViewSize(modeDesc.Width, modeDesc.Height);
     //________________________________________________________________ 5. Setup the application objects
     pointCloud.KinectSetup(hWnd);
}

void Human::RenderScene()
{
     //________________________________________________ 1. Clear screen and camera update
     Clear(0.0f, 0.0f, 0.0f, 1.0f);
     camera.Update();
     //________________________________________________ 2. Application objects
     pointCloud.KinectGetFrame(hWnd, device, deviceContext);
     pointCloud.Render(deviceContext);
     colorShader.Render(deviceContext, pointCloud.usedIndexCount, pointCloud.world, camera);
}

Kinect

Kinect

Kinect coordinate mapper

© Copyright 2000-2021 Wintempla selo. All Rights Reserved. Jul 22 2021. Home