3D Hand Pose Estimation
COM2 Level 4
Executive Classroom, COM2-04-02
Abstract:
3D hand pose is challenging due to the complicated variations caused by high Degree of Freedom articulations, multiple viewpoints, self-similar parts, severe self-occlusions, different shapes, and sizes. I will first introduce my work tackling the problems of hand pose estimation: 1) the multiple viewpoints and complex articulations of hand pose estimation by decomposing and transforming the input and output space by spatial transformations following the hand structure; 2) the multi-modality of the locations for occluded hand joints by a hierarchical mixture density deep network which leverages the state-of-the-art hand pose estimators based on Convolutional Neural Networks to facilitate feature learning while models the multiple modes in a two-level hierarchy to reconcile single-valued (for visible joints) and multi-valued (for occluded joints) mapping in its output; 3) the lack of complete labeled real hand datasets by a tracking system with six 6D magnetic sensors and inverse kinematics to automatically obtain 21-joints hand pose annotations of depth maps. Continuing my research on the hand pose, I will unveil the hand tracking technology behind the exciting AR device, HoloLens 2.
Biodata:
Dr Qi Ye is a scientist of Microsoft HoloLens science team in Cambridge, United Kingdom. She is interested in and working on human-computer interaction, particularly vision understanding involving hands, and a more general setting where humans interact with the environment. She is also passionate about 3D vision problems, their potential applications and bridging the gap of the 3D vision research and the industrial applications, particularly in Mixture Reality and Augmented Reality. Prior joining Microsoft, Dr Ye was a Ph.D. student at Computer Vision and Learning Lab, Imperial College London, under the supervision of Dr Tae-Kyun Kim. Her Ph.D. research focuses on 3D hand pose estimation, and these works have been published in ECCV, CVPR, and PAMI. She also co-organised the ICCV2017 Hand Workshop and the 2017 Hands in the Million Challenge. Before studying at Imperial, she received her M. Eng. degree in Electronic Engineering from Tsinghua University in 2014 and her B. Eng. degree in Electronic Engineering from Beijing Normal University in 2011 in China.