This detailed, hands-on guide provides the technical and conceptual information you need to build cool applications with Microsoft’s Kinect, the amazing motion-sensing device that enables computers to see. Through half a dozen meaty projects, you’ll learn how to create gestural interfaces for software, use motion capture for easy 3D character animation, 3D scanning for custom fabrication, and many other applications.

Perfect for hobbyists, makers, artists, and gamers, Making Things See shows you how to build every project with inexpensive off-the-shelf components, including the open source Processing programming language and the Arduino microcontroller. You’ll learn basic skills that will enable you to pursue your own creative applications with Kinect.

• Create Kinect applications on Mac OS X, Windows, or Linux
• Track people with pose detection and skeletonization, and use blob tracking to detect objects
• Analyze and manipulate point clouds
• Make models for design and fabrication, using 3D scanning technology
• Use MakerBot, RepRap, or Shapeways to print 3D objects
• Delve into motion tracking for animation and games
• Build a simple robot arm that can imitate your arm movements
• Discover how skilled artists have used Kinect to build fascinating projects

About the Author
After a decade as a musician, web programmer, and startup founder, Greg Borenstein recently moved to New York to become an artist and teacher. His work explores the use of special effects as an artistic medium. He is fascinated by how special effects techniques cross the boundary between images and the physical objects that make them: miniatures, motion capture, 3D animation, animatronics, and digital fabrication. He is currently a grad student at NYU’s Interactive Telecommunications Program.

Chapter 1 What Is the Kinect?

• How Does It Work? Where Did It Come From?
• Kinect Artists

Chapter 2 Working with the Depth Image

• Images and Pixels
• Project 1: Installing the SimpleOpenNI Processing Library
• Project 2: Your First Kinect Program
• Project 3: Looking at a Pixel
• Converting to Real-World Distances
• Project 4: A Wireless Tape Measure
• Project 5: Tracking the Nearest Object
• Projects
• Project 6: Invisible Pencil
• Project 7: Minority Report Photos
• Exercises

Chapter 3 Working with Point Clouds

• What You’ll Learn in This Chapter
• Welcome to the Third Dimension
• Drawing Our First Point Cloud
• Making the Point Cloud Move
• Viewing the Point Cloud in Color
• Making the Point Cloud Interactive
• Projects
• Project 8: Air Drum Kit
• Project 9: Virtual Kinect
• Conclusion

Chapter 4 Working with the Skeleton Data

• A Note About Calibration
• Stages in the Calibration Process
• User Detection
• Accessing Joint Positions
• Skeleton Anatomy Lesson
• Measuring the Distance Between Two Joints
• Transferring Orientation in 3D
• Background Removal, User Pixels, and the Scene Map
• Tracking Without Calibration: Hand Tracking and Center of Mass
• Projects
• Project 10: Exercise Measurement
• Project 11: “Stayin’ Alive” Dance Move Triggers MP3
• Conclusion

Chapter 5 Scanning for Fabrication

• Scan and Print: Rapid Prototyping Objects
• Intro to Modelbuilder
• Intro to MeshLab
• Making a Mesh from the Kinect Data
• Looking at Our First Scan
• Cleaning Up the Mesh
• Looking at Our Corrected Model
• Prepping for Printing
• Reduce Polygons in MeshLab
• Printing Our Model on a MakerBot
• Sending Our Model to Shapeways
• Conclusion: Comparing Prints

Chapter 6 Using the Kinect for Robotics

• Forward Kinematics
• Inverse Kinematics
• Conclusion

Chapter 7 Conclusion: What’s Next?

• Beyond Processing: Other Frameworks and Languages
• Topics in 3D Programming to Explore
• Ideas for Projects

Appendix Appendix

• SimpleOpenNI Cheat Sheet
• Chapter 2
• Chapter 4
• Chapter 6

Colophon