ON THE ESTIMATION OF 3D HUMAN BODY MODELS AND POSE FROM MULTIPLE CAMERAS

We present a completely automatic algorithm for initializing and tracking the articulated motion of humans using image sequences obtained from multiple cameras. We discuss the challenges in solving this problem and compare our work to some of the state of the art techniques today. We use a detailed articulated human body model composed of sixteen rigid segments that allows both translation and rotation at joints. Voxel data of the subject obtained from the images is segmented into the different articulated chains using Laplacian Eigenmaps. The segmented chains are registered in a subset of the frames using a single-frame registration technique and subsequently used to initialize the pose in the sequence. A temporal registration method is then used to identify the partially segmented or unregistered articulated chains in the remaining frames in the sequence. The tracker uses motion cues such as pixel displacement as well as 2D and 3D shape cues such as silhouettes, motion residues and skeleton curves. The use of complementary cues in the tracking algorithm alleviates the twin problems of drift and convergence to incorrect solutions. The use of multiple cameras also allows us to deal with the problems due to self-occlusion and kinematic singularity. We present tracking results on sequences with different kinds of motion to illustrate the effectiveness of our approach.