Communicating Across Wired and Wireless Networks
Researchers at Rutgers University have been developing
advanced multi-modal devicesusing sight, sound, and touchfor
applications including telemedicine, documenting traffic accidents, and
assisting in crisis management. The technologies involve communication across
both wired and wireless networks.
Spearheading the effort was James L. Flanagan, director of
the Center for Advanced Information Processing (CAIP) and the university's vice
president for research. The projectwhich involved six other faculty
members as well as a number of graduate and undergraduate studentswas
supported by a grant under the National Science Foundation's KDI program.
Flanagan observes, "We are trying
to make computer interfaces that transcend the limitations of mouse and keyboard.
We want to use sight, sound, and
touch to communicate with complex information systems in ways that humans find
convenient, ways that humans exchange information with one another, face to
face, by sight, sound, and touchusing all of those modalities."
Graduate student Ning Huang (seated) demonstrates multimodal deictic reference to map coordinates by simultaneous eye gesture and speech recognition. Looking on are Professor James Flanagan, Research Professor Sorin Dusan, and Ph.D. candidate Ashutosh Morde.
"We developed interface techniques where we used technologies such as eye tracking,
so that the machine knows where you are looking on the screen," he explains, adding
that the test device "has speech recognition and speech synthesis, so you can carry
on a conversation with the machine in a reasonably meaningful way. It had gesture
and tactile interaction by virtue of an instrumented tactile glove that could give
Flanagan says that his
experimental team extended its work to wireless hand-helds, such as PDAs
(personal digital assistants). According to Flanagan, practical applications include
reporting and documenting traffic accidents, with the police officer
interacting with his PDA by voice, pointing, and other means.
Ph.D. Candidate Yuang Cheng (seated) shows Professor James Flanagan the
Rutgers multimodal enhancement of the MIT GALAXY collaborative architecture. The
demonstration vehicle is a virtual chess game conducted interactively over the network
by simultaneous voice and manual gesture.
In the area of telemedicine, the Rutgers investigators have
been working with the university's medical school to develop a system to help
stroke victims with their rehabilitation. This involves patients practicing
finger and hand movements while interacting with a computer system designed to
give force feedback and to measure finger pressure.
Flanagan's research group also ran a study in conjunction
with the New Jersey National Guard on how to collaboratively deploy assets in a
disaster relief situation. Mapping out deployment by computer networking proved
highly successful. According to Flanagan, "Their opinion was that it was much
better than the traditional technique of an acetate overlay on a topographical
map that you'd mark with grease pencils."
Graduate students working on the KDI grant project included
Liang Cheng, who was involved with developing the software system for
collaborative networking. Flanagan notes that "Cheng did various aspects of the
software system that permits multi-site conferencing, with multimodal
He contributed strongly to the collaborative networking part
Cheng earned his B.S. degree in 1994 from Huazhong University of
Science and Technology, his M.S. in 1997 from Tsinghua University, and his
Ph.D. in 2002 from Rutgers. He subsequently was appointed an assistant
professor of computer science and engineering at Lehigh University.
Cheng, also the Director of Laboratory Of Networking
Group (LONGLAB), comments that
he benefited substantially from his experience on the Rutgers project. "The hands-on
experience of how to set up a test bed and how to make up-to-date, advanced laboratory
has really helped me in my current career."
According to Flanagan, CAIP now has 25 member companies
from private industry that participate in researchproviding equipment,
advising on experiments, hiring graduates, and interacting with faculty. But according to the
Rutgers research vice president, the principal onus for carrying out path-breaking
research falls on U.S. universities, supported by the federal government.
Industry is hard pressed these days, particularly in the economic downturn, and not many
industries will invest in very speculative, risky research," Flanagan says. "The
stockholders just won't
support it. So we are left with the research universities in this country and
our federal establishment to make sure that we are maintaining leadership in
knowledge creation. That's the way we protect our future."
Back to Top of