KDI Home    Contact Us   

Laboratory Of Networking Group (LONGLAB) Dr. Cheng's Homepage

Communicating Across Wired and Wireless Networks

Researchers at Rutgers University have been developing advanced multi-modal devices—using sight, sound, and touch—for applications including telemedicine, documenting traffic accidents, and assisting in crisis management. The technologies involve communication across both wired and wireless networks.

Spearheading the effort was James L. Flanagan, director of the Center for Advanced Information Processing (CAIP) and the university's vice president for research. The project—which involved six other faculty members as well as a number of graduate and undergraduate students—was supported by a grant under the National Science Foundation's KDI program.

Graduate student Ning Huang (seated) demonstrates multimodal deictic reference to map coordinates by simultaneous eye gesture and speech recognition. Looking on are Professor James Flanagan, Research Professor Sorin Dusan, and Ph.D. candidate Ashutosh Morde.

Flanagan observes, "We are trying to make computer interfaces that transcend the limitations of mouse and keyboard. We want to use sight, sound, and touch to communicate with complex information systems in ways that humans find convenient, ways that humans exchange information with one another, face to face, by sight, sound, and touch—using all of those modalities."

"We developed interface techniques where we used technologies such as eye tracking, so that the machine knows where you are looking on the screen," he explains, adding that the test device "has speech recognition and speech synthesis, so you can carry on a conversation with the machine in a reasonably meaningful way. It had gesture and tactile interaction by virtue of an instrumented tactile glove that could give force feedback."

Ph.D. Candidate Yuang Cheng (seated) shows Professor James Flanagan the Rutgers multimodal enhancement of the MIT GALAXY collaborative architecture. The demonstration vehicle is a virtual chess game conducted interactively over the network by simultaneous voice and manual gesture.

Flanagan says that his experimental team extended its work to wireless hand-helds, such as PDAs (personal digital assistants). According to Flanagan, practical applications include reporting and documenting traffic accidents, with the police officer interacting with his PDA by voice, pointing, and other means.

In the area of telemedicine, the Rutgers investigators have been working with the university's medical school to develop a system to help stroke victims with their rehabilitation. This involves patients practicing finger and hand movements while interacting with a computer system designed to give force feedback and to measure finger pressure.

Flanagan's research group also ran a study in conjunction with the New Jersey National Guard on how to collaboratively deploy assets in a disaster relief situation. Mapping out deployment by computer networking proved highly successful. According to Flanagan, "Their opinion was that it was much better than the traditional technique of an acetate overlay on a topographical map that you'd mark with grease pencils."

Graduate students working on the KDI grant project included Liang Cheng, who was involved with developing the software system for collaborative networking. Flanagan notes that "Cheng did various aspects of the software system that permits multi-site conferencing, with multimodal interfaces. … He contributed strongly to the collaborative networking part of that."

Cheng earned his B.S. degree in 1994 from Huazhong University of Science and Technology, his M.S. in 1997 from Tsinghua University, and his Ph.D. in 2002 from Rutgers. He subsequently was appointed an assistant professor of computer science and engineering at Lehigh University.

Cheng, also the Director of Laboratory Of Networking Group (LONGLAB), comments that he benefited substantially from his experience on the Rutgers project. "The hands-on experience of how to set up a test bed and how to make up-to-date, advanced laboratory systems … has really helped me in my current career."

According to Flanagan, CAIP now has 25 member companies from private industry that participate in research—providing equipment, advising on experiments, hiring graduates, and interacting with faculty. But according to the Rutgers research vice president, the principal onus for carrying out path-breaking research falls on U.S. universities, supported by the federal government.

Industry is hard pressed these days, particularly in the economic downturn, and not many industries will invest in very speculative, risky research," Flanagan says. "The stockholders just won't support it. So we are left with the research universities in this country and our federal establishment to make sure that we are maintaining leadership in knowledge creation. That's the way we protect our future."

Back to Top of Page