Mathematics in Image Processing, Computer Graphics, and Computer Vision
T. Chan
The research areas of Image Processing (IP), Computer Graphics (CG) and Computer Vision (CV) are emerging inter-related computer science subdisciplines that offer tremendous intellectual opportunities for the mathematical sciences. In fact, one can argue that together they offer a unique opportunity. There are several reasons for this.
First, IP/CG/CV has many applications, ranging from medical imaging (PET, MRI, fMRI), to astronomical imaging, to virtual reality and special effects in digital entertainment, to robotics. It is a fundamental component of the ongoing information technology revolution. Second, as can be seen from the few examples mentioned above, the applications cut across many different areas of science and technology. Third, just as importantly, this application domain also cuts across many different areas of mathematics, including analysis, geometry, computational mathematics, probability and statistics, and discrete mathematics.
Research in this area has traditionally been conducted by computer scientists and electrical engineers, many of whom are also adept at mathematical skills. In spite of this, mathematicians can bring new ideas, techniques and perspectives to this important area. One of the powers of mathematics is to act as a medium to translate ideas from one scientific area to another. A striking example of this is the recent emergence of the use of PDE and CFD techniques in IP/CG/CV, which have had a tremendous impact in the field. Mathematical scientists can also step back from the immediacy of the applications and study the more fundamental, structural foundation of the key concepts and techniques, which will pave the way for a deeper understanding and future breakthroughs. Conversely, the IP/CG/CV area offers many challenges and new problems and concepts for mathematics. Thus, strengthening the interaction between mathematics and IP/CG/CV is mutually beneficial.
It should be noted that the Board on Mathematical Sciences of the National Academy of Sciences recently (April 2000) conducted a two-day workshop on "The Interface of Three Areas of Computer Science with the Mathematical Sciences" in which IP/CG/CV was one of three topics selected for panel discussions (the author was the moderator). (See http://www.cs.umd.edu/~oleary/nasworkshop.html for a summary.)
The three areas of IP, CG and CV are fundamentally related. CG attempts to generate two-dimensional images from a three-dimensional world and CV consists of the inverse problem of reconstructing models of the three-dimensional world from single or multiple images of different views. IP is used to either enhance the images generated in the forward CG problem, or as pre-processing to help solve the inverse CV problem.