PLATFORMS: Kinect 2

 

ROLE: Programmer

 

DURATION: 3 weeks

* This is a course project of Building Virtual Worlds (BVW). Students were randomly assigned to a team of 5 to 6, and build virtual worlds within 2 to 3 weeks using new platforms and technologies.

 

GOALS:

Tell a compelling story where the guest has some ability to interact in order to bring it to a fulfilling climax.

 

DESCRIPTION:

Ma Yaa! was a course project for Building Virtual Worlds. It told a story of an unfinished model trying to perfect itself before the deadline. The player needed to use their hands to wipe themselves to finish the model.

 

The most compelling part of this project was the way the guest interact with the world. Wiping yourself to accomplish a goal was not seen very often. There were some difficulties, however. To establish an illusion of perfecting the model, we exposed different textures when the guests wipe themselves. At first, the mapping between the coordinates of the texture map and the coordinates in 3D space for the model was easy when the model remained stationary. It became tough, however, after the model started taking in the data from Kinect.

 

The first solution we came up with was trilateration. Yet the input from Kinect was not stable enough to offer a precise positioning. Our final solution was to create a stationary T model alongside the actual model receiving the input. We then attached colliders to the actual model. We mapped the coordinate of the collision to the relative position on the T model, and then converted it to the coordinates on the texture map. The map were then combined with the actual model to generate the final effect.

 

My responsibility in this round was mainly to interact with the interface of Kinect 2. I wrote a wrapper that matched the joint positions from Kinect to the model's gestures, and linked them together. I also implemented a recorder to save the gestures for later use.