We have designed a low cost motion capture system with Microsoft's cheap Kinect sensors for Xbox 360.

 

The solution is scalable, we can use one Kinect sensor, two Kinect sensors, or three Kinect sensors to work, the app will recognize all Kinect sensors automatically.

 

Why we want to use multiple Kinect sensors? Because we want to capture 360-degree range of performance, we need the Kinect sensors to look at us from different angles.

 

The motion capture system works in realtime, no need to calibrate the Kinect sensors, when we finished performance, we get the motion files immediately.

 

The motion capture system is completely FREE for anyone, any purpose, even for commercial purposes.

 

Feel free to use the motion capture system, we will continue to improve the project.

Kinect Mocap Toolkit For Windows v1.0(Last updated on October 28, 2018):

Source code: mocap-x86-vs2017.zip(36K bytes)

Binary pack: mocap-x86-binary.zip(106M bytes)

Release note: This version fixed the position shift issue when using multiple Kinect sensors.

Windows 7/8/10 (32bit or 64bit).

Kinect sensors for Xbox 360.

Download Kinect for Windows SDK 1.8, install the SDK.

Download OpenNI 2 SDK Binaries, install the SDK.

Download NiTE-Windows-x86-2.2.zip, install the SDK.

Download opencv 2.4.11 Win Pack, install the SDK to C:\(Root of drive C:).

Open the project with Visual Studio Community 2017, press 'F7' to build.

Make sure any of the Kinect sensors are not plugged into the USB port of the computer, download Kinect for Windows SDK 1.8, install the SDK.

 

Plug in the Kinect sensors, the drivers will load automatically.

Plug in only one Kinect sensor, run 'bin\tiltkinect.exe', enter a tilt angle, such as '-12', the Kinect sensor will tilt to that angle, see if the Kinect sensor looks at our body, if the tilt angle is not good, run the app again, try another tilt angle, repeat the step, until the Kinect sensor looks at our body properly.

 

Repeat the above step for every Kinect sensor, until all the Kinect sensors look at our body properly.

 

Before performance, we need to wear a pair of dark-colored trousers, and wrap a piece of light-colored cloth round the left knee, or simply wear a pair of asymmetric-colored trousers, like this:

This step is very important, we will use the difference of two knees' brightness to correct skeleton flipping.

 

Plug in all the Kinect sensors, run 'bin\mocap.exe' to work, when all the Kinect sensors have recognized our body, the computer will start to beep once per second, the beep sound will repeat for ten times, while beeping, move our body slightly, let the Kinect sensors get familiar with our body, we need to stand straightly at the moment of last beep, the app will use this moment of body orientation as working orientation.

 

When the beeping stopped, we can perform actions, only one player will be captured, if we want to finish the performance, we can simplely leave the viewport, the motion capture system will stop automatically, but when we come into the viewport again, a new motion capture instance will start automatically, in this way, we can capture multiple performance clips.

 

When we want to stop working, press 'ESC' key to quit the app.

 

The motion files will be saved in 'data' subdirectory, the files will be named by timestamp, the file format is BVH, all major 3d softwares can import the motion files.

Q: What is the difference between the GitHub version and this version?

A: The GitHub version uses smart tricks to guess unknown status, so it is a fake 360-degree range of motion capture system; While this version is an enhanced version, which supports multiple Kinect sensors, it is a true 360-degree range of motion capture system.

 

Q: Can I use Kinect v2 sensors?

A: No, the project only supports Kinect v1 sensors, and we have no plan to support Kinect v2 sensors. The reason is that the project uses NiTE SDK to obtain joint positions, but NiTE SDK does not support Kinect v2 sensors.

 

Q: How about a Linux or a Mac version?

A: We have found that Microsoft's official Kinect driver for Windows is quite stable, so we decide to work on Windows. We have no plan to port the project to Linux and Mac, but anyone can have a try, it might not be too difficult.

 

Q: How many Kinect sensors can I connect to my computer?

A: Kinect sensor for Xbox 360 has a USB 2.0 connector, our computer may has two, three, or four independent USB controllers, open 'Device Manager' from 'Control Panel' of Windows, under 'Universal Serial Bus controllers', we can see all 'USB Root Hub' items.

If there are four 'USB Root Hub' items, it means that there are four independent USB controllers in the computer, we may connect up to four Kinect sensors to the computer, but if some of the USB controllers are not Family USB Enhanced Host Controllers, we cannot use them, so the actual number of Kinect sensors we can connect to our computer is the number of Family USB Enhanced Host Controllers.

If the computer does not have enough Family USB Enhanced Host Controllers, we have to add additional independent USB controllers by installing PCI-E to USB 2.0/3.0 Expansion Cards, or USB controllers for ExpressCard and eSATA slots to the computer. For Mac users, thunderbolt 2/3 to USB 3.0 adapter also works.

 

Q: How to calibrate multiple Kinect sensors?

A: The Kinect sensors will calibrate automatically, no need to calibrate them manually. We just need to fix them steadily at a proper height, then tilt their angles manually or via the 'tiltkinect' utility, let them look at our body.

 

Q: Why the skeleton is flipping frequently?

A: We may forget to wrap a piece of light-colored cloth round the left knee or the piece of cloth is too small to cover the whole knee part, remember to turn on the light in the night.

 

Q: How many Kinect sensors should I use?

A: It is up to our purpose, if we just play for fun, one Kinect sensor is enough; If we need a fairly good result, we can use two Kinect sensors; If we want to capture a dance performance, three Kinect sensors will be better.

 

Q: Can I use more than three Kinect sensors?

A: Sure, the pre-built version supports up to four Kinect sensors, if we want to support more than four sensors, we have to modify the source code, change the macro definition of 'MAX_KINECT_NUM' to a bigger number, then recompile the source code, we don't recommend using more than three Kinect sensors, unless the computer has a powerful CPU and more independent USB controllers.

 

Q: How to use the BVH file?

A: In Blender, we can retarget the animation to different type of armatures, please search the Internet to learn how to do a retargeting in Blender. My favorite Animation Retargeting Tool.

 

Q: Why is the binary 32bit?

A: Because the 32bit of binary can run on both 32bit and 64bit of Windows. If we prefer a 64bit of binary, we can build the project from source code, but all the 32bit of libraries must be replaced with the 64bit of libraries, including all runtime dynamic-link libraries in 'bin' directory.

 

Q: Why does the app hang after started?

A: Plug the Kinect sensor into another USB port, or restart the computer.

The operating system may distribute two Kinect sensors to one USB controller, restart the computer may solve the problem.