Instructions

Video

The following video demonstrates how a simple clip is converted to motion data, and then imported into Autodesk MotionBuilder, where it can be used to animate a 3D model. If you prefer a written tutorial, you can scroll on.

Tutorial

1. Prepare Data

KinectAnimationStudio supports two input modes: live capture and pre-recorded data. In case you want to use live capture, simply make sure the Kinect Sensor is connected to your machine. For pre-recorded data, you need to open a Kinect recording file (.xef) through Kinect Studio, and make sure it is connected to Kinect service, as follows:


KinectAnimationStudio runs together with Kinect Studio.
2. Run KinectAnimationStudio

First, select the output FBX file, where the motion data will be written. Second, click on "Start Recording". At this moment, any incoming frame from the sensor will be saved (for live capture). If Kinect Studio is being used, there is more control over your data, since only played frames will be recorded. For this reason, ensure that correct starting and ending points are selected. Third, click on "Stop Recording", motion data should be on the file.

Output FBX Data

Here is a list of important considerations about the data that is generated.

Skeleton follows Autodesk MotionBuilder joint naming convention

We decided to follow MoBu's naming convention to make it easier to animate rigged skeletons. The exported skeleton can be characterized in MotionBuilder on a single-click.

Pre-Rotation for T-Pose

Kinect orientation data is not exactly aligned to the floor plane. This means that when exported, the skeleton will be slightly inclined throughout the whole clip. For this reason, we assume that, on the first frame, the skeleton hips are orthogonal to the floor. Based on this assumption, the orientation of the skeleton's hip joint is corrected by adding some pre-rotation around the x-axis (value should vary for each clip). If pre-rotation is not desired, it can be easily disabled on the exported data. For example, you can use MotionBuilder as follows:

Smooth your data!

Kinect Skeleton data is extremely noisy. You will probably notice a lot of twitching both on Kinect Studio, as well as on the exported FBX data. I decided not to add any quaternion smoothing technique to KinectAnimationStudio, since smoothing is already widely available for FBX data. To smooth your skeleton on MotionBuilder, simply select all skeleton joints and apply apply the smooth filter.

Missing Roll rotation

There is no roll rotation for feet and head joints. This is due to a limitation from the Kinect sensor. Quaternion information for each joint refers to its parent bone, except for end-joints. For these joints, there is no valid quaternion data. In KinectAnimationStudio, we estimate their orientation based on positional data. However, we cannot estimate roll rotation based only on positional data.