Kinect Interaction

Goal Description Hints Passoff

Due Date - Mar 9


Learn simple gesture recognition based on an RGBD (depth) image.
 CS 656

Using the Kinect API in C# build a program that will sense gestures based on the user's body position. You can only use the RGB and Depth information for this project. You may not use Microsoft's skeleton recognition software. The project would be way too easy :-)

When the user runs your program it should find the person standing in front of the Kinect sensor and then recognize the following gestures:

  • Hand extended with the palm up as if carrying something in your hand
  • Hand extended but tipped vertically as if dropping something out of your hand
  • Hand not extended at all

Your program should randomly place a ball and a bucket on the screen as shown in the picture. When the user extends their hand, the cupped hand should appear at the location where their hand is. With their cupped hand they should pick up the ball (you are showing this on the screen) and carry the ball to the bucket. When the user tips their hand the ball should drop (and hopefully land in the bucket). When the user withdraws their hand, the hand should disappear from the screen and the ball should stay where it is.

Have the user repeat this 5 times and display the total game time :-)

Ball and Bucket



Kinect and C# resourses

It is recommeded that your application have a training mode where camera can be recorded and attached to the desired gestures so that you can train your gestures.

Kinect Documentation

Kinect Programming Guide

Kinect Video Tutorials



__ 1) Ball and bucket correctly displays

__ 2) Hand appears when extended and disappears when withdrawn

__ 3) Hand correctly picks up and carries the ball. (It should not carry if the hand is not directly under the ball)

__ 2) Ball correctly drops into the bucket.