|previous section | Table of Contents | next section|
Part I • Outcome Goals and Results for FY 1999
This section describes how the taxpayer benefits from investments made by NSF’s programs. NSF’s long-term outcome goals address how the investments made by programs have led to results important to the broad mission of the agency. These outcome goals do not lend themselves to quantitative reporting, therefore NSF has developed an "alternative format"—a qualitative scale that allows NSF to report whether the agency has been "successful" or "minimally effective" in achieving its outcome goals. Also, because many research results appear long after an investment is made, this assessment report of NSF’s program performance is mostly retrospective. That is, the outcome results reported in FY 1999 are from investments made prior to FY 1999. The results of the investments made in FY 1999 will not begin to be reported until beyond FY 2000.
How Research Results Are Assessed: Each NSF program is reviewed every three years by independent external panels of experts—Committees of Visitors (COVs) and Advisory Committees (ACs). In FY 1999, the Foundation sponsored more than 190 programs. Of these, 82 programs were reviewed by COVs, producing a total of 43 COV reports. Each COV is a subcommittee of an Advisory Committee, which reviews the products of the COVs. Both COVs and ACs are asked to judge whether NSF is successful in meeting its goals. Their judgment is important for the management of programs and in advising NSF. The COV and AC meetings are subject to Federal Advisory Committee Act rules.
Examples of Research Results: The following discussion of NSF performance includes only a sample of the many noteworthy achievements reported by NSF programs in FY 1999. The examples presented here are illustrations of the impact and success of NSF programs in achieving its outcome goals and are selected to demonstrate the range of activities supported by NSF. In many instances, they reflect connections with other agencies or other countries, or areas of special emphasis identified in the FY 1999 NSF Performance Plan. Many have been identified as significant examples in COV reports. While these examples may appear to be anecdotal, in each case a grant number issued by NSF can be used to identify the example for purposes of verification.
Summary of Results: Overall, based on FY 1999 evaluations, NSF is largely successful in meeting its performance goals. NSF’s programs were consistently judged by external evaluators to have high quality scientific outputs and outcomes. Two areas identified as needing improvement include increasing participation of underrepresented groups, and implementing use of the new merit review criteria that were introduced in 1998. In cases where COVs or ACs did not rate the goals, the committee reports indicated that the goals did not apply to the programs under review, or indicated that they did not have adequate information to provide a rating.
|OUTCOME GOAL 1: Discoveries at and across the frontier of science and engineering.|
|1999 Performance Goal: NSF is successful in meeting this goal when, in the aggregate, NSF grantees (1) make important discoveries; uncover new knowledge and techniques, both expected and unexpected, within and across traditional boundaries; and (2) forge new high-potential links across those boundaries.|
|1999 Results: Successful. In FY 1999, 35 COV and 8 AC reports rated NSF for this goal. All reports rated NSF successful in achieving this goal.|
NSF supports cutting edge research that yields new discoveries over time. These discoveries are essential for maintaining the nation’s capacity to excel in science and engineering and lead to new and innovative technologies that benefit society.
NSF’s key strategy for success is to support the most promising ideas in research and education, as identified through merit review of competitive proposals. Innovation and creativity, cooperative research through partnerships, and education and training are emphasized and encouraged.
In addition, the FY 1999 government-wide performance plan contains a performance goal that is particularly relevant to NSF research and education programs:
1999 Performance Goal: An independent assessment will judge NSF research programs to have the highest scientific quality and an appropriate balance of projects characterized as high-risk, multidisciplinary, or innovative.
|1999 Results: This goal was achieved. In FY 1999, all COV reports indicated that NSF programs were of high scientific quality. Of the 30 COV reports that gave an opinion on balance of projects in the programs under review, 24 reports indicated that the balance was appropriate. Six reports indicated that a few programs under review did not have appropriate balance, with lack of investments in high-risk projects being the most frequent concern. To address these concerns, NSF staff will be asked to make investments that address this goal, and will strive to identify high-risk projects for reviewers and evaluators. NSF is looking at how to improve the indicators for this goal, and how to provide better data to evaluators of this goal for FY 2000.|
The following examples illustrate the impact and success of NSF’s programs in achieving important discoveries. The examples were chosen to address the FY1999 areas of emphasis across NSF’s programs and to demonstrate linkages across traditional disciplinary boundaries through collaborations.
Advances in Biology
The Nobel Prize in Chemistry, awarded annually by the Royal Swedish Academy, recognizes major discoveries over the lifetime of the recipient’s career and is an indication of the influence that the individual has had on chemistry. The 1999 Prize was awarded to a researcher whose NSF support over a 20-year period has resulted in an understanding of the details of the birth and death of molecules during chemical reactions. Just as the motion of dancers can be frozen in time using the pulsed light from a strobe, the motion of molecules during chemical reactions can be followed in time using pulsed lasers, much like a rapid strobe. The work for which the Nobel Prize was awarded used rapid laser pulses only a few femtoseconds in duration. A femtosecond is to a second what a second is to 32 million years. This area of chemical research has been named femtochemistry. This research has begun to uncover the details of reactions that form the basis for important processes ranging from catalysis to vision.
NSF Research in the Antarctic and Arctic
Discoveries in How the Young Learn
|OUTCOME GOAL 2: Connections between discoveries and their use in service to society.|
|1999 Performance Goal: NSF is successful in meeting this goal when in the aggregate, the results of NSF awards: (1) are rapidly and readily available; (2) feed, as appropriate, into education, policy development; or (3) are used by other federal agencies or the private sector.|
|1999 Results: Successful. In FY 1999, 35 COV reports and 8 AC reports rated NSF for this goal. Of these, 34 COV reports and all AC reports rated NSF successful in achieving this goal.|
|America’s national security,
economic competitiveness, health, environment, quality of life, and
understanding of the world around us depend on taking advantage of discoveries.
Discoveries resulting from basic research and education lead to new
knowledge, which often cannot be identified at the start of a project.
Thus, the connections are not immediately apparent, and may only be
realized decades later. The new knowledge frequently leads to applications,
which can have a significant impact on society. NSF views the public
accessibility of NSF generated results as well as partnerships among
government, academia, and industry as critical components for the progress
of science and technological innovation.
NSF’s key strategy for success in achieving this goal is through use of the merit review process to make awards for research and education activities that focus on discovery and that create or have the potential for connections with use in service to society. Potential for use in service to society is an element in the merit review criteria established by NSF and used in the decision process leading to funding.
The following examples illustrate the impact and success of NSF’s programs in achieving important connections between discoveries in FY 1999:
Predicting Storms: Local high-impact weather causes economic losses in the United States that average $300 million per week, impacting over 10 percent of the U.S. economy each year. The mission of one NSF-funded center is to demonstrate the practicability of numerical weather prediction of storms and to develop, test, and validate a regional forecast system appropriate for operational, commercial, and research applications. The May 3, 1999 tornado outbreak in Central Oklahoma was used to test the storm model, and included networking, collecting, and processing data from several National Weather Service Doppler weather radars. While the storm-scale forecast was not perfect, improvement over current operational models was substantial—showing increased precision in predicting. It clearly demonstrated the value of using mesoscale radar data in storm prediction that can provide a two-hour forecast down to the scale of a county. The project was able to generate short-range high-resolution forecasts that dramatically out-performed the National Weather Service forecast during the tornado outbreak. As this forecasting capability is further developed, it will become a critical tool in determining which areas will be most severely hit by storms, thereby allowing sufficient and timely warnings to be issued to persons in affected areas. Who stands to benefit? The commercial airlines industry, power and communications industries, surface transportation, agriculture, defense and space flight, construction, insurance and recreation industries will clearly benefit, as well as the National Weather Service and the general public.
Oceanographic Research in Service of Fisheries Management: Over the past decade, there have been dramatic decreases in fish populations in the major fisheries of the United States and Canada including the Cod and Haddock fisheries of the eastern seaboard from the Grand Banks to Georges Bank. These decreases have had disastrous economic and societal consequences. Within the U.S., NSF spearheaded a research program whose goal is to predict changes in the distribution and abundance of marine species as a result of changes in their physical and biotic environments as well as to anticipate how their populations might respond to climate change. The program will provide information to fisheries organizations so they may better preserve stocks.
Practical Application of Digital Library: Research in digital libraries led to practical technology exploited in many different areas. The FBI applied digital library (DL) technology to establish an "electronic reading room" to comply with the Freedom of Information Act. The California Department of Transportation applied DL to roadside vegetation, quasi-real-time tracking of road conditions, and flood-related emergency services.
Global Change Research and Life in Extreme Environments—Sustainability Project: Results from a large, long-term interdisciplinary study involving eight natural and social science disciplines have provided a combined assessment of the effects of a predicted global warming, oil development, tourism, and government cutbacks on the sustainability of Arctic villages in the range of the porcupine caribou herd. The effects of global change on the tundra food sources for caribou on the Alaskan north slope and elsewhere are critically important to local native villages where a subsistence lifestyle is practiced either as a necessity for survival or as a cultural choice.
Bill Nye the Science Guy: This NSF-supported television series was awarded several Emmys in 1998-1999, including Outstanding Children’s Series, Outstanding Directing in a Children’s Series, Outstanding Achievement in Single-Camera Editing, and Outstanding Achievement in Sound Editing. The series resulted in increased comprehension and application of science facts and concepts among its child viewers, who were able to provide more complete and complex explanations of scientific concepts. "Bill Nye the Science Guy" is one of the most popular projects in NSF’s informal science education programming.
|OUTCOME GOAL 3: A diverse, globally-oriented workforce of scientists and engineers.|
|1999 Performance Goal: NSF is successful in meeting this goal when in the aggregate: (1) participants in NSF activities experience world-class professional practices in research and education, using modern technologies and incorporating international points of references; (2) academia, government, business, and industry recognize their quality; and (3) the science and engineering workforce shows increased participation of underrepresented groups.|
|1999 Results: Successful. In FY 1999 36 COV reports and 8 AC reports rated this goal. Of these, 33 COV reports and 5 AC reports rated NSF successful in achieving all or most areas of the goal. Four COV reports and 2 AC reports indicated that NSF should do more in the area of showing increased participation of underrepresented groups. A few reports noted that the data necessary to measure this goal were not always available, and more outreach by NSF is necessary. These issues are being addressed by NSF staff in the FY 2000 and FY 2001 performance goals and plans.|
Although NSF provides only a relatively small portion of the overall U.S. investment in the development of the science and engineering workforce through its programs, this investment is particularly important to the development of the workforce of the future. The quality of the future workforce is dependent on the investment being made now to educate and train students. A diverse science and engineering workforce that is representative of the American public and able to respond effectively to a global economy is vitally important to America. As a nation, we need new technical knowledge and people trained to use that knowledge. The competence and capabilities of the nation’s science and engineering workforce keep America at the forefront of innovation and technological progress.
One of NSF’s key strategies for success in achieving this goal is by providing opportunities for participation in integrative research and education experiences. To influence the development of integrated approaches, NSF has developed a number of Foundation-wide programs intended to facilitate the integration of research and education. Each of these programs relies on NSF’s close interaction with the academic science and engineering communities to draw research and education together. NSF works to achieve this goal by making awards for research and education activities that are intended to influence the development of the science and engineering workforce, and increase the participation of underrepresented groups.
The following examples illustrate the impact and success of NSF’s programs in achieving Goal 3 in FY 1999.
|OUTCOME GOAL 4: Improved achievement in mathematics and science skills needed by all Americans.|
|1999 Performance Goal: NSF is judged to be successful in meeting this goal when, in the aggregate, the results of NSF awards lead to: (1) the development, adoption, adaptation, and implementation of effective models, products, and practices that address the needs of all students; (2) well-trained teachers who implement standards-based approaches in their classrooms; and (3) improved student performance in participating schools and districts.|
|1999 Results: Successful. In FY 1999, 19 COV reports and 3 AC reports rated NSF for this goal. Of these, 15 COV reports and 3 AC reports rated NSF successful in achieving this goal in all or most areas of the goal. Three COV reports indicated NSF was not fully successful in some areas of this goal. This goal is viewed by many COVs and ACs as being more applicable to NSF’s educational activities, and therefore rated it only when applied to those programs.|
This goal addresses a need widely recognized by all Americans. Proficiency in essential skills and understanding of basic concepts in mathematics and science is critical to the earning power of individuals, to the nation’s economic competitiveness, and to the quality of life in the 21st century. NSF is the only agency that directly aims at developing such proficiencies at all levels of education.
NSF has established linkages with other agencies, and supports the development of prototypes for cooperative activities involving state and local educational agencies, and the private sector.
NSF supports a continuum of activities that enables improvement of mathematics and science skills for all Americans. These activities include educational reform at the K-12 levels and beyond; teacher education and professional development; research activities that use science and technology to inform better educational practice; and activities that bring science into the classroom and place students at the sites of exploration and discovery. Common themes that are emphasized across the Foundation include the implementation of high quality, standards-based instruction for all students; integration of research and education; and coordination of resources, policies, and practices to maximize the impact of educational investments. These activities benefit students, teachers, and the general public nationwide.
Investments in education are made to facilitate the development of essential skills in mathematics and science for all Americans through the promotion of broad-based or system-wide reforms in science, mathematics, engineering, and technology education that are based on national standards. Important for FY 1999 are projects that include systemic approaches, attention to teacher preparation and development, partnership with other agencies, and developing a strong research base for use by practitioners.
The following examples serve to illustrate the impact and success of some of NSF’s programs in achieving improvement in math and science skills, including areas of emphasis in FY 1999.
|OUTCOME GOAL 5: Timely and relevant information on the national and international science and engineering enterprise.|
|1999 Performance Goals: NSF is successful in meeting this goal when: (1) the average time interval between the reference period (the time to which the data refer) and the reporting of data decreases by 10%; and (2) customer satisfaction ratings are achieved with the relevance of products offered of at least 45% "excellent" and at least 90% "excellent" or "good."|
|1999 Results: Successful. Both the timeliness and the relevance goals were achieved.|
This goal addresses NSF’s legislative mandate to collect, interpret, and analyze data on scientific and engineering resources, and to provide a source of information for federal policy formulation. In a recent survey, a sample of the science and engineering policy community indicated that improving timeliness of data was high priority for them. The value of information on the science and engineering enterprise is highly dependent on its relevance to those who seek to use it in making policy decisions. Different users are interested in different aspects of the enterprise.
|previous section | Table of Contents | next section|