For my second point, I will use an idea from Neil Postman, who says you cannot do philosophy with smoke signals.³ In the good old days, people used smoke signals to alert neighboring castles that the enemy was coming. Postman's claim is that you cannot engage in philosophy this way, and my claim is that current computer systems, to a large extent, do not support the goals of enhancing creativity, imagination, contextualized learning, or learning-on-demand.

The notion of ease-of-use is also very misleading. In the Knowledge Navigator tape we saw a little while ago, and in many other futuristic tapes, the future of learning is pictured as a person sitting in a chair and pushing a button here and a button there. The message conveyed is that if you want to learn something, this is how it will happen. But learning always requires engagement. It will require affection and personal relevance. Learning to play the piano well isn't easy, and yet a lot of people do it. When we think about technology, we should not only reflect on ease-of-use, but also on these other dimensions.

What would it take to build a successful interdisciplinary investigation? We have been involved in a number of collaborations, but I think the problem that Snow, in his two cultures--or, I would say today, many cultures--identified is one of the critical problems; people have illustrated this in the previous talks.

And what about the current state of affairs at universities? Having spoken about other organizations, I want to point out that I'm a faculty member of a university and that I believe there are quite a few things wrong with current universities. They are lecture-dominated and curriculum-dominated. We have taught some classes where we wanted to incorporate more learning-on-demand elements. But where is there room for authentic, self-directed learning activities if, at the beginning of a three-month course, you provide students with micro-managed curriculum telling them exactly what type of things will be dealt with and when? I think that in our university environment we overemphasize the notion that students solve given problems. In the real world, problems really do not exist. We always intermix problem framing with problem solving. Another artificial aspect of most university or school problems is that there are right or wrong answers. In the real world, this is rarely the case. We have to rethink, from a lifelong learning perspective, what a university education will mean in the future.

What is the long-term societal impact of this research? Here is a quote from Einstein: "Wisdom is not the product of schooling, but the lifelong attempt to acquire it." We have to think about what I call the redefining of the roles of high-tech scribes. In the Middle Ages, the average person had to go to a scribe to express herself or himself. In today's world, we find a similar situation with respect to computational media: Only high-tech scribes can master them. And the high-tech scribes have an interest in sustaining this role. This will be one of the big challenges: To avoid creating an elite group of high-tech scribes.

Another thing I consider fundamental in what we are facing is a change of mindsets. I mentioned earlier the notion of teachers as lifelong learners, which means teachers should not look upon themselves as truthtellers or oracles, but as coaches, facilitators, mentors and learners (see table on page 6). It would be an incredible educational experience for a student in high school to see his or her teacher struggling with a problem. But, today, that is frightening to many teachers. We want to create a society that is not limited to consumers of modern technology. My complaint about the World Wide Web would be that currently most people dealing with it are consumers. I briefly mentioned the "500 TV channel" future. Illich, in his outspoken criticism, calls schools and universities reproductive organs of a consumer society.⁴ We can devise many technical challenges from this.

What will be the basic skills in the future? There's a lot of debate in all the sciences about "basic skills." If most relevant knowledge for the working world must be learned on demand, which is a claim I would defend, what is the role of basic education? One could argue it is to empower people to learn on demand.

By focusing on a lifelong learning perspective, we have had the opportunity to think a lot about school-to-work transition. This has been a concern for many organizations. I strongly believe that the world of working and living relies on--you can agree with me or not--collaboration, creativity, definition of and framing of problems, dealing with uncertainty, change, distributed cognition (distributed intelligence), and symmetry of ignorance. If this is the case then the world of schools and universities needs to prepare students for a more meaningful life in the world. Planning to employ learning systems and intelligent systems in our educational institutions, we must judge them from these perspectives.

Comment: I have a couple of broad comments. One of the hardest things for us to do is to enrich our notions of what we mean by interdisciplinary. As long as we have the semantic tag or label that we throw on this, we're not going to deepen our understanding of the technology that we need and the kinds of processes required to really make it work. There are two points not brought out as strongly as they could be. One goes back to theoretical foundations, which is also a very long-term activity, but one that often goes in parallel with interdisciplinary work. So, part of what we need is much better theory, much better formalisms; for example, formal foundations that allow people to make the assertions and the boundaries of their discipline more explicit, more processable, more analytic, so that they can, in fact, be combined with other disciplines.

Speaking as a neuroscientist and as someone who has worked on space systems--two highly interdisci-plinary areas--part of what happens is that in the building of a real system or even in scientific endeavors in brain science, we often develop a sort of data-dictionary level of interaction. People learn to talk, and they gradually intuit the problems and the foundations of another field. But what we really need to do is create some new foundations for scientists, including, for example, the ability to integrate models at a formal, explicit level.

To put it a slightly different way, this is something that Nora Sabelli, John Cherniavsky and several others of us have been dealing with between our NSF and DARPA educational projects. Right now, people think that the big problem with technology demonstrations is turning them into technology insertions. That certainly is one problem, and we're dealing with it. So you package, let's say, an educational tool with real curriculum or real pedagogy, and you make that transfer.

But there's something else that needs to go on that people are not addressing. It is not the technology transfer; rather, it is what we call climbing up to the systemic level. How do I take superb technology demonstrations and insertions and actually climb up to a systemic level? What is the information? What is the instrumentation that I have to do at the technology insertion level or transfer level that allows an organization like a school or a hospital or a manufacturer to make system-level tradeoffs about how to use the technology?

It's not just what we're referring to when we talk about scaling up, which is a commonly known as model replication; this is much more difficult, because it involves real tradeoffs between how you are doing things now and how you envision doing them in the future. There are some qualities that haven't even been represented and made explicit enough to reason about.

Comment: I have a comment in reaction, in part, to hearing some of the wonderfully wise statements that Norbert Wiener made a couple decades ago. Disciplines haven't always been around. They're evolving entities in themselves and they come into existence for various reasons. In my own career, I've seen interdisciplinary work lead to new subdisciplines and disciplines. There's an interesting distinction between, on the one hand, nurturing the formation of an active new discipline--which often comes as interdisciplinary work with its own peer review process, its own potentially revised or modified jargon and terms, and its own conferences--and, on the other hand, nurturing a flash-in-the-pan new scholarship that goes on when somebody who finds himself steadfast in statistics, for example, starts thinking about graphical models and computational constraints. There's an interesting interplay between what we mean by interdisciplinary work in terms of the creation and nurturing of new disciplines and the evolution of old disciplines, versus nurturing individuals and what they're doing.

Dr. Joseph Bordogna, National Science Foundation: I will stand on dangerous ground here and try to summarize what everybody has been saying. It has to do with changing the academy from reverence to reductionism, to innovation through integration. It is a real shift, and I am not sure we can insert things with the present leadership. We have to spend time educating the new students. I hear a lot of things; for example, in engineering, one might call it project-based learning: You do things, or hands-on, teamwork, invent the future, and so on. What we're attempting to do in engineering education, with all three engineering schools, is to really implement these things so that graduates have independence through not only knowing something in depth, but also possessing a functional literacy across a variety of disciplines. As a result, they can see disparate pieces and integrate them to get something out the door.

So, I feel comfortable with what you are saying; it validates the sort of ad hoc way in which we have made a big investment at NSF in the last 15 years, in changing the undergraduate engineering paradigm. In essence, we still graduate people who have been taught that to know your special thing deeply is the reason you're revered--not necessarily the reason you are hired and compensated, but the reason you are revered. That is the holy grail. We have to change this so that, at the baccalaureate level to begin with, students graduate with an understanding that it's much more complicated than that. The word interdisciplinary does not have the right cachet. You have all talked about that, especially the first speaker. And because the cachet is in an area where it mitigates against making these changes, it can be very emotional. We have to attack it at its root.

Comment: I think the last comment, in particular, about the universities having trouble restructuring is particularly important. I wonder if members of the panel have any comments or advice on what NSF might be able to do, with regard to its structures, to position itself to respond to the creativity of the community, and not just to have you fill holes in boxes that we seem to create. Somebody must have thought of that--what they would do if they were czar.

Comment: My comment has to do with the question of having at least part of the funding devoted to very high-risk proposals, as a way of getting away from the conservatism of both established programs and established review groups. These established groups do a fine job in a lot of areas, so you wouldn't want to get rid of them. But certainly a fraction of their funding could go into a program where you simply say to somebody: Okay, here's a certain amount of money, go out and find people you think would be interesting to support. And use your own judgment, like the military agencies tend to do.

I am going to talk about my core interest, which is the design and construction of systems, both social and technological, that support teaching and learning in places where people learn. My primary interest is in classrooms, but I think this problem extends well beyond that. I'm interested in how interdisciplinary research can transform what we know about helping people teach and learn in these highly social situations.

I'm sure many of you have heard the following story, but I think it bears repeating. It is a story about what would happen if people with different kinds of expertise came forward from the past to the present time--for example, a doctor and a teacher. A doctor coming to today's operating theater would look around and decide that he or she didn't know how to work in this new context. If we're talking about, say, 100 years, the doctor may not even recognize the place of work as at all related to where he or she worked in the past. But if a teacher were to come forward to this time and look around, he or she would probably take a step back, ponder for a moment, and say, "Well, I know how to work here."

The goal of our work is to figure out how to be really transformative in the context of teaching and learning. With respect to that goal, here is how I have approached today: If someone were to deliver to me a plate with the answers to help me progress towards my goal, what would I like to see on that plate? And further, what sort of interdisciplinary research would give me the answers? I've been fortunate enough to have a fair amount of interdisciplinary work experience in my career. When I was in graduate school at the University of California at Berkeley, I was one of the first cohort of people doing a thing called cognitive psychology, soon to be called cognitive science, which represented an interdisciplinary focus in education, anthropology, computer science and other fields. Later, I was fortunate enough to work at both Bell Laboratories and Bell Corp., where the coin of the realm was interdisciplinary groups trying to solve a problem. Today at Northwestern, we have a group of people with different kinds of expertise working on the problem of teaching and learning.

I'll start with the general architecture for my remarks. I'd like to begin by sharing some successes in the world of interdisciplinary research and how interdisciplinary research has had a transformative impact on teaching and learning. I'd then like to point out what I perceive as the core problem to be solved. I'll close with some comments on what makes interdisciplinary teams work and what the roadblocks are. I think it's important for you to get a clear sense of what the problem is; to help me define and clarify it, I've brought several videotaped segments.

In all of these cases, a group of people--some doing cognitive psychology, some computer science, and, in some cases, anthropology and linguistics--have combined to bring forth these insights and this kind of transformation. This includes people who have conducted research in understanding prior conceptions and have tried to build the mental models that learners have. These models have led to rethinking the teaching technique wherein learners learn through articulation of what they believe to be true of a state before they are given new information about that state. This involves articulation of naive reasoning or making conceptions apparent before teaching.

Another example has to do with people who have done work in what are called "communities of practice." They are trying to understand how organizations of people combine through practices and language to shape what new members of a community learn. They are trying to chart that very carefully, looking at how expertise is distributed across people and things and how that expertise is often organized to help new people join a community. This work has led to new kinds of thinking about instruction in the form of "cognitive apprentices," wherein you try to orchestrate, in school situations and other learning situations, a kind of apprenticeship learning that divorces teachers from standard roles of simple, straightforward deliverers of information.

I want to take that point as the springboard to talking about the issue on which I'd like to see us make progress. I coined this issue "understanding and supporting learning in a complex context." The classroom is an incredibly complicated place. When you try to develop a system to support teaching and learning, the things that go on in that place are, in fact, the data that you need to analyze. As a discipline, we are woefully underprepared to treat that activity as data and to analyze it. This problem has several components. One is taking what goes on in contexts such as classrooms and treating it in a rigorous and in-depth way. This means analyzing the activity so that it can then inform design progress.

A second factor is that learning environments, especially schools, are places where a lot goes on, not just the delivery of the facts. Yet, we don't really know how to construct situations that form a tighter coupling among the cognitive, social and psychosocial roles of teachers as they go through the teaching and learning process. Many people--myself included--have vaguely formed intuitions, and it would help tremendously if we understood how learning goes on in a complex context, and if we knew more about how culture influences the use and utility of technologies and other things that support learning.

The key goal here is to have more complete theoretical and methodological techniques for more nuanced analyses of activity content. Today, when we analyze what goes in classrooms for the purpose of designing better systems, it is typical to do one of two things. Either we collect examples of behavior and tell stories about very complex things, or we take highly stereotypical kinds of the behavior, count them and reduce the complexity. I don't think we have well worked-out techniques that allow us to work in the vast middle where all the content really is. I believe this both at the level of how to think, theoretically, about what's going on in those situations and how to have, methodologically, the techniques and technology to do it.

This is a standard classroom interaction with teacher and student--an example of teachers doing stereo consultation with students in a classroom setting. The key problem here is: When you think about situations like this and decide that you want to change them, or insert new technologies, you really want something that supports an effective space like this. In this example, this woman is one of the best science teachers in her high school.

So, how can we use interdisciplinary expertise to help us think about, first, recording the interaction as data; second, taking a technological or organizational intervention; and third, analyzing its impact. This requires several efforts: Cultural analysis, cognitive and social modeling, and linguistic analysis. You need analytic notations that help you parse the interaction and put it together in new and different ways. You also need better technology for video display and manipulation to make that happen more smoothly. If we had those things, we'd have a deeper understanding of highly situated behavior, which I believe will lead to better learning environments.

Those of us who work in teaching and learning often behave as though our sole job is to support the teaching of the facts or to support the teaching of practice. But when you talk to most teachers, especially pre-college teachers, about what they think they're doing, they believe that they're trying to support the development of a whole person; this includes intellectual development and social development. Teachers tell us quite frankly that often, especially in highly challenged teaching situations, the subject--chemistry, for example--is just an excuse to have a meaningful interaction with the student and to further his or her development. What we fail to do, perhaps because we don't understand the situation well enough, is to develop technologies and techniques that make it easier for teachers and others to merge support for social development with support for intellectual development. An example of this might be a teacher, working under the guise of teaching earth science, trying to help a student understand what it is really like to do research in a college situation. The ostensible goal for them being together was earth and physical sciences.

There are any number of systems and approaches being produced today that try to take advantage of microcultural differences in the United States. Some people believe it possible to draw on aspects of cultural experience to facilitate some form of teaching and learning. When we try to do this, however, it's a shot in the dark. We don't really understand what that nexus is about. It would be incredibly important to have a kind of cultural analysis, a linguistic analysis, even collaboration from the worlds of entertainment and design, to help us think about how to design systems that make meaningful contact to cultural experience so that people learn better.

The objective of the designer of this system was to take an aspect of these little girls' cultural experience, hand-clapping rhymes, put it in a reading instruction program, and try to engage these kids more deeply. Did it work? We really don't know. Did she pick the right thing? I don't know that either. It was a shot in the dark, because there was no base of information that could arguably come from something called learning and intelligent systems that would inform the design of something like that. The key impact would be a better understanding of when and if design prescriptions ... [inaudible] ...

Over the years of trying to do some form of interdisciplinary research, lots of things turn out to be important, like shared expertise and the mutual opportunity of having some people learn what others already know. When I try to reduce this to what really makes it happen, it comes down to three things. First, it takes trust. In interdisciplinary partnerships that don't work, there always seems to be a sense of senior partnership. When you have a sense of senior partners, a sense of first among equals, other people don't feel as positive about participating. In some cases, computer scientists feel like they should be in the driver's seat and behavioral scientists should support them.

Second, it takes persistence. Jim Anderson talked about this. We often have a clock for getting something done and it is usually driven by our experience with one group of people of a very narrow form of expertise working together. We don't give interdisciplinary efforts enough time to happen. Finally, it takes faith. You take a problem like the last one I mentioned. Is there a real nexus of culture and learning? Well, I think so. Teachers tell me it's possible. Designers feel they have some evidence that they have been able to engage users because of it. But it's really a kind of faith, and you need to have that kind of faith to embark on a high-risk, potentially high-gain research effort that would elucidate it.

I'm here today to talk about my experience wearing many of the different hats involved in this initiative. All of the different speakers today have resonated with me--Dr. Anderson in particular, when he spoke about the importance of interdisciplinary training programs.

I was told early in my career that I would have to be very proactive about looking for a job because I would never see one advertised for someone like me, and I would never encounter a department that automatically understood why they needed someone like me. As a result, I ended up having the opportunity at Rutgers to help create from scratch a new department, called the Center for Molecular and Behavioral Neuroscience. Our primary vision was to develop across the domain of neuroscience an atmosphere of trust in which each important aspect of neuroscience, from molecular to behavioral, would be given equal weighting and equal prominence. There would be no sense that the basic "touching-the-brain" neuroscience was any more or less of a basic science than the cognitive/behavioral aspect.

What we've done at Rutgers is unlike what I see being done in most other current neuroscience programs around the country, in that we give equal weight to the molecular and cognitive aspects of neuroscience and, at the same time, give opportunities for in-depth expertise in other areas as well. Our faculty includes those looking at potassium channels, molecular genetics, animal research in vision, perception and attention, as well as human research relating to normal and abnormal neurological and mental processes. My own research focuses on the neurobiological basis of speech perception and its implications for language-based learning disabilities. This category of disability affects 10-15 percent of our population and is of growing concern.

I would like to give an actual example of a multidisciplinary research project. But before I get to that, I want to talk about some of the barriers to doing this kind of work. Clearly, there are barriers to integrating effectively across many disciplines--we've heard about this already from many of the previous speakers. There are a couple of barriers that haven't been covered; one of them is particularly interesting and, in fact, was in this morning's USA Today. Right on the front page, it says, "Women missing good jobs in key growth industry." The article states that inside the corporate headquarters of most computer, engineering, or any modern developing industries, one could "fire a cannon through the cubicles and hit only men."

Someone looking around the main table in the front of this room could see the same thing. I raise this issue because it means we're missing a lot of talent in these industries. As we sit around thinking about what the goals are for this initiative, if we don't build in mechanisms for training, beginning at a very early age, that encompass our whole population, we're missing a great opportunity. We currently are in a crisis at a couple of different levels. One, clearly pointed out in today's paper, is that women are not being educated at the same level to reap the benefits and participate in this growing initiative of interdisciplinary research, technology and practical application. This can still be clearly seen from elementary school to university, through industry and in our society as a whole.

Another socioeconomic barrier is the lack of access to the hardware and software needed to grow up feeling comfortable interacting with computers. This barrier will have far-reaching impact as jobs and access to information increasingly demand technology. Lack of equal access to computers and the Internet in schools has major effects on the extent to which we can use whatever we develop in these kinds of initiatives, particularly in education. I'll be speaking in a moment from a more practical point of view, based on the initiative in which Mike Merzenich, Bill Jenkins, Steve Miller and I are currently engaged. We are translating our basic research into practical application, through the use of CD-ROMs and the Internet, to help improve the learning potential of language-learning impaired children. This initiative is going to meet with a major barrier to market--the lack of equal access to computers and the Internet in a lot of schools and clinics around the United States.

You must choose a question you can investigate. And although all of us would like very much to understand language at the highest linguistic levels, we must first understand how the smaller components of sound that make up speech are processed in the brain. Children with specific developmental language disabilities may offer us a unique window into what really is going on as the brain struggles to process speech. These are children who are developing normally in every respect, except that they are failing to develop language at the expected age, and they usually go on to develop reading problems as well. What I've found over many, many experiments during almost 20 years of work is that language-impaired children are just like normally developing children in all of their information processing, hearing abilities and visual abilities, except for one very profound difference: The ability to integrate two or more brief bits of information entering the nervous system simultaneously or in rapid succession.

We asked children to listen to two events and tell us whether they're the same or different--for example, a low tone and a high tone. We found that normally developing children need only tens of milliseconds between two 75-millisecond tones. Language-impaired children can understand the task and learn to perform perfectly well, but they need an order of magnitude more time--200 to 300 milliseconds--between events. This profound delay in brain processing tasks has been replicated over and over now in a variety of different sensory information studies across modalities with language-impaired children. In a nutshell, the brains of these children process more slowly, whether you look at speech perception or speech production, visual or motor performance. How does that translate into something that might interfere specifically with language? Why is it that these children are so normal in every other respect, and yet so very abnormal in language?

To answer these questions, one needs to understand what speech is like and what it is the brain might have to do to make important speech discriminations. Here is just one example of the kind of speech contrast that every baby lying in a crib has to listen to and somehow sort out. This is one speech sound: "Ba." This is a different speech sound: "Da." If you look at the acoustic pattern produced when these speech sounds are said, you see an acoustic frequency over time spectrogram. What we see here is that although the frequencies are fairly complex, they differ in only the initial 40 millisecond onset, during which a very rapid frequency change over time occurs that signals the difference between "ba" and "da."

What the brain needs to do is extract this rapidly changing frequency information from other information that immediately follows it. The interesting fact that I discovered early on, and have continued to work on since then, is that language-impaired children can't make these kinds of rapid discriminations at all. Basically, their brains cannot process anything in as little as 40milliseconds when this is followed rapidly in succession by further information. They need hundreds of milliseconds. Now this is where technology comes in. We used computers to artificially stretch out the acoustic portions of the signal that our research had indicated were most impaired in these children. We developed a computer algorithm to alter the acoustics of ongoing speech, to stretch and amplitude enhance the salient temporal components within the speech signal.

About four years ago, I began working with colleagues at the University of California, San Francisco. They were doing single-cell electrophysiology in monkeys and mapping the somatosensory cortex of the monkey's hand. What Mike Merzenich at UCSF and his colleagues were fascinated by was the ability to show that if you map the actual surface of a monkey's hand in terms of individual electrode placements--physiological basis, single-cell recording--you get an actual representation, a physical map inside the somatosensory cortex. This neatly maps out the thumb and each finger. It's quite remarkable and shows that we all have little homunculi representing our brains in the neurophysiology of the brain.

What Merzenich and his colleagues have found is that these representations are not static. Through specific training or stimulation occurring on the fingertips of two fingers, for example, over a short period of time the representation in the brain at the single-cell level actually remaps itself and is influenced by learning. This is quite remarkable and very important for this conference. When we're sitting in a classroom and the teachers are teaching about some subject, what we're doing is literally remapping and reorganizing the brains. All learning has an effect in physiology because brains are indeed plastic, not static. Another interesting and important point here is that Merzenich's studies were done in adult monkeys. The effect does not depend on getting in at some critical early periods. This means that sensory representation--the very basic representation of brain reorganization--is modifiable by training.

So how does that relate in any way to what's going on with language-learning impaired children? Merzenich and I decided to ask the following question: If one can influence and remap through training and learning, can we remap the brain out of a defect? If language-impaired children are not processing information at a rate that allows them to extract sensory information out of the environment fast enough for speech and language processing, can we speed their neural processing rate up by training? We've developed a series of computer games based on remapping, basic learning algorithms and experience. We mapped that on top of our findings that extending and altering the acoustics within speech might help these children. We put it all together in a fairly complex series of programs we call Fast ForWord that allow a child to "play," for hours at a time, at remapping.

The games are designed to both enhance the ability to process the speech that the child does hear, because it's been acoustically altered to fit their brain processing rate, and, at the same time, to alter the brain so it can process more quickly, thus dealing better with the real-world speech environment. The games are also adaptive in that the computer is constantly changing the input based on the child's last two or three responses. The results of our first couple of studies were published in the January 1996 issue of Science, with two back-to-back articles.

We reported changes in processing rate, speech and language functioning in a group of language-impaired children who spent a month playing these computer games in which the acoustics were altered and the focus was adaptive learning and training. One group of children used programs with modified acoustic speech and temporally adaptive training exercises, another group used the exact same program with natural speech and visual, non-adaptive training. Everything else was the same for both groups; it was a very well-controlled study. We saw dramatic improvements not only in the temporal thresholds (very dramatic news in itself, that something as basic as your physiological threshold can be changed through adaptive learning) but also in speech discrimination, language processing and even grammatical comprehension. These are the hallmarks of these children's problems. In a month of training with these new computer-generated, acoustically modified training exercises, we saw the equivalent of approximately two years of actual language growth based on standard clinical tests.

I will close with comments on the final hat that I wear. This year I'm on sabbatical from Rutgers University and I am working in a new start-up company that my colleagues Merzenich, Miller and Jenkins and I have developed, called Scientific Learning Principles, in which we're now transferring this technology into programs that will benefit language-learning impaired children in classrooms and clinics. We encourage all of you to track our progress on the Web at http://www.scilearn.com.

One last comment: When I helped found our Neuroscience Center, at Rutgers we had 16 faculty positions. We've now recruited 15 of them, and eight of them are women, and eight cognitively oriented in the field of neuroscience, which is traditionally a male field focused on molecular and systems approaches. I also spent 20 years doing basic science laboratory research which turned out to have practical applications. Through the university-based technology transfer system, my scientific colleagues, joined by business partners, founded a new company, Scientific Learning Corp., aimed at linking basic neuroscience research, industry, business, schools and clinics. What I have learned from these experiences is the benefit of stretching traditional boundaries to forge new links and greater interaction between previously segmented groupings within our community. This is often strongly resisted, but the benefits for society are surely worth it.

This morning we are each speaking slightly differently, about different angles, as an interdisciplinary presentation. I'm going to provide a view from the standpoint of human language--both human language with which we communicate and human language technology. I'm focusing on this for two reasons. First, many people readily accept human language as a true sign of intelligence; we can communicate by using language. Second, it is something I know about from many perspectives. I was trained as an electrical engineer and I am now a computer scientist. I spent the first 10 years of my professional life trying to understand human-to-human communication, and the last 10 years trying to build machines that can achieve something like it.

I want to provide two different perspectives within the context of this topic. On the one hand, the capabilities of machines, bandwidth, software and information are growing exponentially; on the other, human capabilities evolve on an evolutionary scale. It's very important to deal with this widening gap between where we are and with" what we are confronted. We need to help people acquire, manage and process information.

Human language technology (HLT) can play two roles within the context of learning and intelligent systems. First, HLT facilitates learning by providing an interface. We all know speech is very natural: We learn how to speak before we know how to write and read. Speech is efficient: I can talk 10 times faster than I can write, four times faster than I can type. Speech is very flexible: I don't have to touch anything or see anything. Just as important, however, is the fact that information is often a remnant of human communication. The kind of information we seek is linguistic in nature and we have to provide easy access to it. I will give you an example of this in a minute.

Let me concentrate on the first of my two perspectives and think within the context of providing information for the user. Imagine a person sitting here, for example, interested in asking the question: Can I have the telephone number of the nearest hospital? We have some sort of source information in some sort of linguistic archive that has been painstakingly processed by human beings and turned into some sort of special database. It could be the Yellow Pages or White Pages or some other collected set of information. Today, you typically access this information using a keyboard--usually with a pair of hands that belongs to someone else. In the future, what we want to be able to do is just pick up the phone, ask the question and have an intelligent and graceful interface that allows us to access the information by speaking fluently and conversationally. What is just as important is the ability to process the information--to annotate videotape or audiotape, for example, and index them so that we can ask questions and get directly to the information. This would be important progress as compared to asking for the information in a prescribed way.

There are many different applications for this in learning. In the areas of collaborative learning and distance learning, for example, these kinds of capabilities are going to be very important. Now, you may be thinking that because I have been working in speech for a long time, it is natural for me to see speech as a solution to all things. You know the saying, "If you have a hammer, everything looks like nails." Nevertheless, it is very important to realize that what we really want is based on human language, although the modality may vary from speaking to handwriting to typing. In certain environments, such as in this particular meeting, for example, handwriting will probably work better for taking notes than talking to your little computers.

What is important is for us to be able to capture the linguistic competence of human beings and to apply that linguistic competence through different modalities and media. We all know that when we converse, there is the notion of turn-taking. There is also the notion of clarification. And the notion of referencing: Whether I want to use the pronoun "it" as I'm typing or speaking, or I simply want to point to something to reference it. Those are all speech acts, and the goal or desire is to capture something like that pictorially. The use of multiple modalities is an essential concept here. We must integrate all of these as a graceful interface inspired by our linguistic competence. This is something that separates us from other animals.

Most people will probably agree that speech recognition is the most fully developed of the new technologies. In this workshop, in fact, people have talked about that. So, how well can we execute this speech recognition? I'd like to show you three or four examples of where we are. The first piece of the video is actually a visionary video; it's not real. This is something that was done almost 10 years ago by Apple Computers, called the Knowledge Navigator. I will show you about 30 to 40 seconds of it to emphasize what you have to go through when you're trying to learn something. Watch for clarification, dialog, using hands, using speech--all these kinds of things.

From this brief video, you get a sense of the kind of interface we would like to achieve. The videotape also gives us a reality check: It's mid-1996, and where are we?

The next piece is something that's done at BBN, a company in Cambridge. It's dictation, and the vocabulary includes tens of thousands of words. This is an example of what you can do sitting in front of your workstation and wanting to create a document such as a letter or report.

A third example is a system called InfoMedia News on Demand, under development at Carnegie Mellon University. The idea here is to index a large number of videotapes and access sections by content. So you have to tape the radio announcer's speech, transcribe and index it, and provide the interface to access the information. You ask a question and you can see the speech being indexed.

The last example is the very beginning, the rudimentary capability described in the Knowledge Navigator scenario. A person is actually interacting with a machine through conversation. So far, the first example requires nothing but recognizing the words. The second one has some capability of understanding, and the third example I'm going to show you is one in which the user actually carries on a conversation with a computer to solve real problems.

Present technology is far from what we would really need to provide a graceful human interface to access information. I decided to prepare a quick chart to give you some idea, within the framework of the auditory channel, of human capabilities as compared to machines. In speech recognition, for example, humans are far better than machines by at least an order of magnitude--maybe two orders of magnitude--if you compare error rates. Language understanding is something a machine can do reasonably well, provided there is a very restricted domain. If you want to talk about travel, for example, that's okay, but don't ask about other topics in commerce.

In other areas, machines are actually doing better than humans--voice verification, for example. A machine nowadays can tell identical twins from one another. In the case of sound detection, you can sit on one side of a stadium and hear a conversation on the other side. In a sense, the machine is cheating. The microphone can take the sound spectrum and do translation so that you can hear subsonic and ultrasonic types of sounds, with no concern about auditory fatigue. Machines are also much better than humans in terms of sound localization--for example, hearing a siren as you're driving. So in some cases a machine can do better, but that's typically in focused areas where constrained pattern recognition techniques can be applied. With regard to the kind of speech capability we would like to have, however, we are far from it.

By far, the most successful approach to achieving high-performance speech recognition and language understanding is a data-driven approach using statistical modeling techniques. Although we have done very well in improving its performance, it's still far from human performance. Achieving human capabilities would require a tremendous amount of training data--hundreds, if not thousands, of hours of data, testing, training the machine to do a focused thing. And even that might not be sufficient. The current way we approach the problem may not be sufficient as task complexity increases, and we certainly pay little attention to human cognition and perception.

I would like to echo what Jim Anderson just said, that interdisciplinary collaboration turns out to be a big but rewarding undertaking. People working on speech are collaborating with people who were led to the notion of speech understanding by natural language processing. It turns out to be a wonderful thing: Not just transcribing speech, but understanding and executing it. It's a total win. So far, we haven't gone nearly far enough, and I certainly agree with one of the comments made here today. We don't have an educational program that could train people appropriately, even in such a narrow field as human language.

Now, although I am talking about one particular area, the same probably holds true for other areas as well. If we're interested in this kind of human-computing interface, I think we have to emphasize the set of language-based principles that integrate and coordinate multiple modalities and multiple media. We need to promote an interdisciplinary program that brings together people from different backgrounds and different expertise. We actually have done quite a bit of that at MIT, but not nearly enough. For example, I think people working in speech must understand vision; there are a lot of complimentary things we can learn from each other.

The last point I want to make is that if you want to do this, rather than having people work in individual pieces, you must be working on integrated solutions to real-world problems. If you understand the how of human intelligence, how we learn, the best way to demonstrate that understanding is through implementation. I strongly urge that we include a component in this program to emphasize this--to put your money where your mouth is, so to speak.

Question: I noticed that most of your thinking here is sort of still "inside the box." A user sits down at this large or small box and that is the environment. As we're starting to envision what we need to invest in to really deal with some of the exciting and interesting changes in our culture that will occur five or 10 years out, there are two obvious concerns.

One concern is mobility: I'm in my car and on my car phone and I want to ask that question about the nearest hospital. What kind of issues are you seeing about that?

The other concern relates to my own programs and one of the things I've been emphasizing in them. What is it like to have a variety of utilities there as support for human-human, human-agent interactions, but as a collaborator experiencing some environment together? In a lot of my work, we have virtual reality environments where the programs are personified as characters; they walk around, talk with themselves and others; interesting issues arise. There is a different kind of intelligence here, a whole different set of strategies in action. What does it mean to have the commentary taking place in the common arena? Can you comment on these points?

Dr. Zue: Yes. Let me very quickly comment, and then maybe there are other people who can join me. I completely agree with you about your second concern; it is extremely important. Both of your points are important. The metaphor of a person sitting in front of a screen clearly doesn't go far enough. Let me give you an example: Soon, there will be a toll-free number you can call at MIT from anywhere in the country and ask about the weather condition in 750 cities. You will be able to get a weather forecast by talking to a machine.

One of the interesting things about this is that if you take away a display, the communication is drastically different. We make a distinction between "displayful" communication, human-human and human-machine, versus "displayless" communication. The output from the other party needs to be clearly filtered. You can't just dump all the words and have people suffer from information overflow and not be able to capture the meaning. Clarification dialogue then becomes much more important.

Dr. Zue: Absolutely. You're right.

We've had an exciting symposium so far, and I want to thank the five speakers who gave very stimulating presentations this morning. I know that we all have additional questions and comments that we want to share. In a few minutes, we will hear from Herb Simon on these issues, and I look forward to that as well.

The purpose of this afternoon session is to hear what each of your organizations is doing and to describe a bit what the National Science Foundation is planning on doing in this area. We welcome your feedback on what we're thinking about. We also want to hear about any activities or plans that you have in the different related areas, and we especially want to discuss the potential for various productive partnerships with you.

For a long time, NSF has had research that fits within this overall rubric. I won't go through all of it, but it has largely been within particular areas: We've looked at technology, learning and cognition, animal behavior, and behavioral and developmental neuroscience. Programs in computer science and engineering have long supported work in this area. Over the last decade, we have also funded research on technology and cognition, and science and mathematics education. Hopefully, at some point in our discussions today, specific examples of these will come to the table. More recently we have recognized that we need to think about this as a broader effort cutting across NSF.

Two years ago, we had what we called the Director's Opportunity Fund and we invited proposals from around the Foundation. The proposals had to cross over directorates, with at least two of the Foundation's seven directorates involved. We had $50 million in the fund, overall, and we received many proposals. Three proposals were submitted, however, that were very similar to each other. The first was called Learning in Natural and Artificial Systems; it came from five of our directorates. Another was titled Educational Technology and was the product of two directorates. The third, Intelligent Systems, was submitted by three directorates. From their descriptions as well as from the titles, these proposals clearly had significant areas of overlap. We asked those submitting the proposals to work together to see if something productive could be designed.

I was going to cover what it is that really excites me about this area, but I think we've already observed inspiring work this morning that overlaps with my own interests.

I do want to emphasize one point: We need to be sure that we consider human beings. We must avoid wasteful examples of the kind we've all heard: Somebody gives computers to a school and then assumes that learning will happen. But that school might not use the gift and might leave the computers in boxes. There is no wiring, no software, and, especially, no people who know what to do with this computer resource to use it for learning. We truly have a lot of work ahead of us.

There are many important questions that could not be easily addressed with the way NSF typically does programs. I'm sure we all have our own set of burning questions that we think need to be addressed but that are not being effectively answered. But, as we begin to think about what we want to do in terms of our program that won't merely replicate what is already going on, it would be wise for us to find out what industry is doing, what private foundations are doing, and, especially, what other federal agencies are doing. Also, we hope to discuss whether there are possibilities for cooperating.

Let me turn to Dr. Herbert Simon. I think that for probably everyone in this room, he needs absolutely no introduction. He's been the mainstay of this entire field, stimulating lots of ideas for quite a long time. He's a Nobel Laureate in economics. And he's provided a lot of stimulation of both conceptual scientific work and more practical applications.

I have heard some very important issues addressed today. There is the sense of a common mission coming through all of the talks we have heard. There's a reason for this, and it begins with the history that we started with this morning. Jim Anderson related the history of cognitive science--I'll use the word cognitive science to cover all the things we've been talking about over the past few hours--which is a very long history, stretching back at least to the Cybernetics period, and probably even before that. The events he was discussing are now about 50 years old; one interesting question that I raise is: If this is such an exciting field, what's been happening in the last 50 years?

I think there are several things to be said about that. First, a lot has been happening on the interdisciplinary front in the last 50 years. The whole profession of computer scientist was created during that period. Computer science, in itself, doesn't cover exactly what we're talking about, for it extends beyond it in one direction and perhaps starts with another goal, although artificial intelligence sometimes likes to expand itself into these areas. That's a whole new profession that was created.

We shouldn't be faint-hearted about interdisciplinary efforts, although that's an example of another lesson: As soon as we recognize an interdisciplinary area and train people in it, they become a new, narrow discipline that is in danger of the same narrowness we experienced in the generation before. So this doesn't work for an indefinite period. It worked for a while, and then we had to start a new revolution, which is perfectly all right.

So we have another discipline: Cognitive science. And we now have people also doing neuroscience. That term is not a new term, again, but as a field in which you can get a Ph.D. and do things, it has arrived. And, yes, universities are regressive in many respects. They're still giving lectures. We're all fond of pointing this out, and some of us are fond of doing it; others have tried to reform a little bit. But we can point to an enormous amount of change and an enormous number of new kinds of mixtures of disciplines, or interdisciplines, over the past 50 years.

Second, we've been making a tremendous amount of progress in the substance itself. Before I get down to that, I want to say why there has been the start of this re-disciplinary shuffle and why there has been this progress. Not all scientific problems are born equal. There are good scientific problems, better scientific problems and really great scientific problems. As far as I'm concerned, it seems easy to pick out at least four problems as being right up there at the top. If you have a favorite fifth, I won't quarrel with you. If you start mentioning six and seven, I probably will quarrel with you.

There are four problems that human beings have been concerned with from time immemorial. First is the nature of matter. High-energy physicists get really excited about that, and they should. Some of the rest of us do too, when we listen to them. Second is the origin of the universe: The Big Bang, as some of us think of it now--at least until the next theory comes along--and all of the things that go with it. Third is the nature of life. There again, we've had a big revolution--the molecular biology revolution, which is also interdisciplinary. And many of you know the history of the Rockefeller Foundation's role--in particular, Warren Weaver's role--in bringing that about and making the new interdiscipline possible. The fourth problem, of course, is the nature of mind: How the human mind operates, what it can do, what it does do, and, today, the imitation of mind by machine.

Now those are great scientific problems, exciting scientific problems. It is hard to think of problems that could be more exciting. They each have characteristics of the kinds of things that we should be providing resources for and supporting and that NSF obviously should and does support. But there's another criterion to consider: Science shouldn't be supported on a large scale unless there's something you know to do about it; unless there are some techniques for approaching it; unless we have the machinery and the means for going at it, or some shadow of an idea. Of course, if you have all the ideas, then you're done and you should go look for a new problem. But you have to know how to take the first step to make research on a topic viable.

In the past 50 years, give or take, there has been substantial and successful demonstration of the possibilities for exploring and understanding the human mind on a variety of levels. This goes all the way from the molecular level up to the level of social systems, including social systems today that contain important electronic components for computing and communicating. We saw and heard examples at all of these levels in the discussions this morning. We saw perhaps a half a dozen examples, and we could have seen hundreds--tens of hundreds--of similar examples. I have a few favorite examples of my own.

This particular proposed initiative specifically emphasizes the learning aspects of intelligence. This is the idea that intelligence isn't exhibited simply in the performance of an intelligent system, whether human or computerized, but, in the human case, by the fact that the human being can acquire that intelligence through a series of appropriate experiences. The whole trick, of course, is to a) provide those experiences; and b) motivate the human beings to learn from experiences, which is a very large part of the game.

I don't want to continue talking about the things we now know about human beings, because if we knew them all, we wouldn't have to be supporting any research. Victor Zue, for example, showed us some of the things we didn't know or didn't know quite well enough about human speech recognition and human speech understanding. Here we have a field that is well underway, a field in which substantial progress has been made to address one of the really important and exciting scientific problems. This problem is not only exciting intellectually, but practically as well, because it bears on everything we human beings do and the success of it all: On our learning, on our performance, on our decision making, on our problem solving--you name it, speech recognition and understanding is deeply concerned with it.

We're at a particularly good time now--we have been for perhaps maybe 10 years--for pursuing cognitive science with vigor because, at the moment, equipment and the cost of equipment are not really a bottleneck. The NSF people may groan a little when I say that, because they deal with budget limits and all sorts of issues like that. But it's not really a bottleneck. We did our first AI program--The Logic Theorist--on one of the largest machines in the world, the JOHNNIAC. And we barely got it on. As I remember, we had 6,000 words in the core and 10,000 words on a drum, but it was scratched, so we really didn't have 10,000. Today, as you know, you don't even need a PC to run programs like that.

Contrary to predictions made a few years ago, most sophisticated computer programs for either doing intelligent things or modeling intelligent processes are not being run on supercomputers. They're being run on a PC, if not a laptop. So we have no more excuses that we can't tackle a problem because the machines aren't big enough. The machines are plenty big enough. The real limitations today are our own imaginations, our own ability to push through the research to create the concept. It is a really terrifying thought. Assuming, of course, that our salaries are going to be taken care of--but that's peanuts when you look at the investment our society has made, quite correctly, in the other three problems that I have mentioned. As soon as we realize what progress we can make, we're going to make similar investments in this problem. That may be what we are just beginning to do.

As I said, we had excellent examples this morning of areas where important progress has been made. Let me mention a couple of my other favorites in order to illustrate some specific points. I have watched the development of this field primarily from the viewpoint of a psychologist trying to understand human thinking by simulating it with computer programs. This is just one of the many approaches that was mentioned this morning--as it should be.

The basic strategy in this particular field over the years has been the following: First, pick a task human beings do that everyone would agree requires intelligence. Then, see if you can figure out enough about it by running laboratory experiments, by taking thinking-aloud protocols from human beings, by doing all the things you can do, such as studying brain-injured people and so forth. See if you can figure out enough about it and understand the processes well enough so that you can build a computer program that will not only get the same result, but will do it in a humanoid way. That, again, is a scientific problem: Finding out what actual processes the humans were using along the way and making sure that those are the processes you are using in your simulation. When you can do that for a particular task to a satisfactory degree of accuracy, so that there is a good match, for example, between the detailed trace of the computer program and the thinking-aloud protocol of a subject, then you have a theory that makes almost second-by-second predictions of the subject's behavior, and you can begin to test the generality of the theory for other subjects. An early program of this kind was the general Problem Solver, which demonstrated the central role of means-ends analysis in human problem solving in a dozen different tasks.