Award Abstract #0313184
ITR/NGS: Stochastic Multicue Tracking of Objects with Many Degrees of Freedom

NSF Org:	CNS Division of Computer and Network Systems
Initial Amendment Date:	August 26, 2003
Latest Amendment Date:	September 19, 2006
Award Number:	0313184
Award Instrument:	Continuing grant
Program Manager:	David Hung-Chang Du CNS Division of Computer and Network Systems CSE Directorate for Computer & Information Science & Engineering
Start Date:	September 1, 2003
Expires:	August 31, 2007 (Estimated)
Awarded Amount to Date:	$399999
Investigator(s):	Dimitris Metaxas dnm@cs.rutgers.edu (Principal Investigator) Dimitrios Samaras (Co-Principal Investigator)
Sponsor:	Rutgers University New Brunswick 3 RUTGERS PLAZA NEW BRUNSWICK, NJ 08901 732/932-0150
NSF Program(s):	ITR SMALL GRANTS
Field Application(s):	0000099 Other Applications NEC
Program Reference Code(s):	HPCC, 9215, 2884, 1652
Program Element Code(s):	1686

ABSTRACT

The dynamic data driven estimation of 3D human shape and motion from optical sensors (cameras)

is a fundamental problem that has many applications such as non-invasive security and monitoring

systems and human-computer interaction. The difficulty of the problem stems from a) the shape

complexity and the many degrees of freedom due to the high articulation and deformations of the

human body, b) the noise introduced by the sensors, c) the dynamically changing appearance of the

human body in an image sequence in terms of shape and intensity, and d) the unknown distributions

of the visual cues (e.g., edges and optical ow) and the lack of a principled methodology of how to

combine them. Lagrange dynamics-based 3D deformable models have the potential of being successful in analyzing the shape and motion of non-rigid or articulated data such as the face and hands since they

can adapt to the shape and motion variations across individuals. This proposal aims to develop a deformable model-based framework for human shape and motion estimation which can cope with the dynamic changes of the input visual data and the resulting need for the dynamic integration of visual cues extracted from the input data. Our proposed approach should be able to evaluate automatically and dynamically the \trustworthiness" of each visual cue and integrate subsequently the cues in a manner that reects their importance. The methods that are proposed here are general and are not only applicable to

face/hand tracking but to whole body tracking.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

B. Rosenhahn, R. Klette and D. Metaxas (editors). "Human Motion Understanding, Modelling, Capture and Animation," Springer, Computational Imaging and Vision, Vol 36, 2008 (Book), 2008.

Cristian Sminchisescu, Dimitris N. Metaxas, Sven J. Dickinson. "Incremental Model-Based Estimation Using Geometric Constraints.," IEEE Transactions PAMI, v.27(5), 2005, p. 727-738.

D. Metaxas, G. Tsechpenakis, Z. Li, Y. Huang, and A. Kanaujia. "Dynamically Adaptive Tracking of Gestures and Facial Expressions," International Conference on Computational Science, 2006.

G. Tsechpenakis, and D. Metaxas. "CRF-driven Implicit Deformable Model," IEEE Conference on Computer Vision and Pattern Recognition, 2007.

G. Tsechpenakis, D. Metaxas and C. Neidle. "Combining Discrete and Continuous 3D Trackers," Human Motion Understanding, Modelling, Capture and Animation, B. Rosenhahn, R. Klette and D. Metaxas and Editors, Springer Computational Imaging and Vision, v.36, 2008, p. 133.

G. Tsechpenakis, D. Metaxas, and C. Neidle. "Learning-based Coupling of Discrete and Continuous Trackers," Computer Vision and Image Understanding, v.104(2-3, 2006, p. 140.

Please report errors in award information by writing to: awardsearch@nsf.gov.