II. NATIONAL POLICY CONCERNS AND NEEDS

Decision makers in the Executive and Legislative branches of government are concerned about the management of Federal investments in research, which in the most recent budget had reached over $90 billion as measured by R&D. Articulating this concern, former OMB Director Franklin Raines in a 1998 Science article raised the following questions: How large a scientific enterprise does the United States need? How can we set priorities in the Nation's R&D enterprise? How can we measure the success of our Nation's research programs? How can we strengthen the government-university partnership? How do we engage the American people in the excitement and wonder of science?

Likewise, Chairman of the House Science Committee, Sherwood Boehlert, responded to the proposal to double Federal funding for research by questioning: "What are we going to get for that money? How will we know if we are under- or over-spending in any field?" He went on to warn: "I want the Committee, early on, to take a serious look at the balance within the federal research portfolio...You can ... count on me to ask tough and uncomfortable questions to ensure that the scientific community is acting in its and the nation's long term interest…We really need to push for more data". (1/21/2001).

How should the scientific community respond to these questions and expectations? How should it raise public awareness that the quality of life in the future will depend in large measure on the generation of new wealth, on safeguarding human health and the health of our planet, and on opportunities for enlightenment and individual development made possible by science and engineering discoveries? Will the response of the scientific community be effective against competing claims on the Federal budget?

These issues have prompted a vigorous policy debate over the last decade involving the Executive and the Legislature, the National Academies, and professional societies. Nonetheless, this debate has to this point generated no widely accepted process for the Federal government, with systematic input from the scientific community, to make priority decisions about the allocations in and across fields of research in support of Federal goals. The National Science Board has participated in this debate, issuing a series of policy statements, including an NSB working paper on Government Funding of Scientific Research in 1997. The NSB concluded in that paper that within the Federal budget there should be an overall strategy for research, with areas of increased and decreased emphasis and a level of funding adequate to both serve national priorities and to foster a world-class scientific and technical enterprise. To this end Congress and the Administration need to establish a process that examines the Federal research budget before the total Federal budget is disaggregated for consideration by Congressional committees. The Board further concluded in its 1998 Strategic Plan that a prerequisite for a coherent and comprehensive Federal allocation process for research is the development of an intellectually well founded and broadly accepted methodology for setting priorities across fields of science and engineering.

As followup to its earlier work, the Board undertook, beginning in March 1999, a focused examination of Federal priority setting for research in the U.S. at three levels: 1) in setting Federal goals, 2) in allocation decisions by Congress and the Administration that produce the Federal portfolio for research and 3) in Federal agencies and departments in achieving their missions in alignment with stated Federal priorities. The Board determined that the appropriate level for its focus is the second level, that is, the White House and Congressional processes that in the aggregate produce the Federal portfolio of investments in research.

Context for Federally Funded Research

The Federal role has always encompassed the missions of Federal agencies and departments and, beyond those missions, has helped to sustain a healthy national infrastructure for S&T. The Federal role today is especially critical for research that is high risk, requires long-term investment in the expectation of future high payoffs to society or that is unlikely to be funded by the private sector; for unique, costly, cutting edge research facilities and instrumentation; and for academic research that, as a primary purpose, supports the education of the future science and engineering workforce.

The national science and technology enterprise has grown and become more pervasive in both the private sector and in government, even as the Federal share of support to the enterprise has declined. Now, more than ever, achieving Federal goals for sustaining U.S. leadership in S&T demands partnerships and cooperation with other sectors. Understanding where Federal funding can be best employed and the level of investment required to assure the health of U.S. science and technology are critical to prudent management of the Federal portfolio. Commitment to an intellectually well founded, long-term strategy for Federal research must be an integral aspect of a sound fiscal policy, regardless of year-to-year fluctuations in available funds. The Federal budget process for research must assure sustained and sufficient support for a diverse, flexible, opportunistic portfolio of investments, emphasizing the long-term health of the knowledge base and infrastructure for research-including human resources.

Need for A Different Approach to Budget Coordination and Priority Setting

NSB discussions with spokespersons from Executive and Legislative branches and with experts on the budget, data and analytic methods, as well as reviews of the literature on budget coordination and priority setting identified the following needs.

Methodologies for:

• Determining the appropriate size of the enterprise;

• Determining the appropriate level of support to individual fields;

• Achieving balance in the portfolio;

• Setting priorities for the Nation's R&D enterprise;

• Achieving effective communication on scientific matters with the American people;

• Strengthening government-university partnerships.

Improved data, expert analyses, and scientific advice are needed to address these issues, including:

• A continuing mechanism for expert advice representing a broad cross-section of the science and engineering research and education community to support difficult decisions on research investments-especially in major infrastructure projects;

• Better quantitative data and methods of analysis adequate to measure the benefits of research;

• A mechanism to identify and track the relevant Federal funds for S&T through the budget process in the Administration and Congress.

The Current Federal System

The current Federal system for allocating funds for research is an incremental process that results in final allocation decisions based on input from a range of stakeholders, including the science and engineering communities. Ultimately, the Federal budget for research rests on aggregated political decisions in thirteen congressional appropriations subcommittees. There has been a host of critiques and suggestions for improving the process, many focused on the goals for research, but some suggesting changes to the process itself. The most frequent critique addresses a perceived lack of a clear methodology for priority setting and coordination. Several possible remedies have been suggested: structural changes to the process, alternative interpretations of the appropriate goals for Federal research, and new mechanisms for funding allocations and better management of the Federal research portfolio.

Since the late 1980s, and under both Republican and Democratic Administrations, there has been substantial attention devoted to developing better mechanisms for coordinating the Federal budget for research through the Office of Management and Budget (OMB) and the Office of Science and Technology Policy (OSTP) (Box 3).

The cabinet level National Science and Technology Council (NSTC) (Box 3) in the previous Administration and the earlier Federal Coordinating Council for Science, Engineering and Technology (FCCSET) (Box 3) provided mechanisms in OSTP for identifying major national initiatives that cut across agencies in designated priority areas (e.g., nanotechnology, global climate change, and information technology). Under the last Administration, the NSTC was established by Executive order as part of the OSTP science and technology policy apparatus. However, unlike FCCSET, OSTP and the Director of OSTP, which were established through legislation, the NSTC had no permanent status. Likewise, the President's Committee of Advisors on Science and Technology (PCAST), whose purpose was to provide "critical links to industry and academia," was established by Executive order.

Furthermore, in neither the Executive nor the Legislative branches is there a mechanism for evaluation that takes into account the breadth of Federal investments within the context of Federal goals for research. The Executive Branch, through OMB, OSTP and PCAST, made an effort to treat Federal funding of research as a portfolio, recently taking into account the issue of balance among fields of science in Federal support across all agencies and departments.

These steps have been in the right direction, but are only a preliminary effort. Congress (Box 4) also has directed attention to what might be done to improve its process but has not yet taken any action to implement formal Congressional mechanisms comparable to OSTP to coordinate functions across budget lines, agencies and departments, and committees.

To enhance the effectiveness of Federal investments in achieving long-term goals for research, a regular, credible process that relies in part on expert input from the science and engineering communities is essential for priority setting among competing investment choices. The Federal portfolio for research is an accounting device that aggregates the research portfolios of the individual departments and agencies funding S&T. It has not been managed as a portfolio. As a precondition for priority setting across the Federal research budget, coordination must be achieved among its diverse components. While efforts at better coordination through OSTP mechanisms have been useful in managing cross agency initiatives, coordinating mechanisms are also necessary for evaluating the performance of Federal research investments as a portfolio and for identifying gaps, overlaps, areas for decreased emphasis, and the top priorities for additional investments. Coordination and priority setting therefore must be intertwined in the Federal research budget process.

The Need for More and Credible Data and Analyses

No mechanism exists to provide strong quantitative input to justify a particular level of investment in Federal research based on expected benefits to society, due in part to the lack of data and methods to measure research benefits. Data on Federal research funding, especially at the field level, are often unavailable on a timely basis to inform budget allocation decisions, use outdated research field definitions, fail to capture important characteristics of research activities, and suffer from inconsistent applications of definitions across reporting units.

In spite of the need for more and better data on the Federal research enterprise, collecting such data requires the cooperation of a large number of Federal agencies and departments, many of which see no benefit to themselves from this effort. There are few resources available to address the major undertaking that would be required to generate reliable data tailored to the needs of budget decisions and outcomes for research funding allocations. It would require a concerted effort to define and obtain agreement among the many Federal units that would be involved, and would require support from OMB and Congress to assure collection of high quality, timely data.

Identifying the Composition of the Federal Research Portfolio

OMB requires agencies to report research and development (R&D) activities that they are funding for the annual budget process. Even if reliably measured across funding units, since "D" at about 55% of the total is larger than "R", reporting the sum of the two as the measure of Federal research investment results in an indicator that fails to reflect accurately the Federal funding to discovery and innovation. Also, significant fluctuations in support for "research" tend to be obscured when combined with the larger "development" category funds.

There have been several attempts to provide a better measure for the federally funded activities that contribute to national innovation. The National Academies proposed a coordinated "Federal Science and Technology Budget" (NRC, 1995), a subset of Federal R&D that constitutes "federal support for a national science and technology base." The FS&T budget would provide Congress with a tool for tracking the aggregated pool of Federal departmental and agency funds that support the science and technology base. OMB has employed over the last few budget cycles a similar mechanism for tracking the President's research priorities through the budget process. This mechanism comprised a collection of program budgets that are primarily research programs, but also includes non-research elements, such as the education and human resources component of the budget for NSF. OMB found this mechanism useful in highlighting Federal research investments and effective in supporting the President's priorities for research through the budget cycle.

The Board, for the purposes of this study, has focused on "S&T", in accord with the argument put forward in the NRC report and efforts by OMB to identify basic and applied research activities for tracking through the budget process. At the same time the Board recognizes the lack of consensus on federally funded activities that should comprise Federal "S&T". Criteria for inclusion of activities in a Federal budget for research for the purpose of monitoring and evaluating Federal activities as a portfolio will require further discussion and analysis. The important subset of research funding devoted to the long- term, high-risk basic research is especially vulnerable to becoming invisible in the larger budget for S&T. It is critical that this component-- which provides the long-term investment to produce as yet unforeseen major breakthroughs in knowledge--receive sustained public support.

Capturing the Character of Activities Supported

Within research, the character of research fields and activities has changed over time, resulting in definitions that no longer capture important distinctions in federally funded research activities. Special areas of weakness include multidisciplinary and cross disciplinary workgroups and teams, emerging areas, differences in interpretation across agencies' reporting units, and the evolving content of traditional research fields themselves. In addition, educational contributions of research--particularly in academic institutions for graduate education--are not captured in most agencies' databases.

Reliability and Timeliness

Differences in interpretation have resulted in wide discrepancies in research funding reported by performing and funding units--or even within the Federal government across agencies and programs-even though they ostensibly describe the same activities. In addition, timeliness, in most cases essential to budget allocation decisions, is not possible with Federal databases based on surveys. Much of the data measuring the Federal research portfolio with respect to programs funded, support for fields of science and engineering, and performing institutions are several years old at best. Timeliness will become increasingly more problematic as rapid changes in science and technology increase the need for current data to monitor Federal investments.

Assessing World Leadership of U.S. Science and Engineering

National capabilities in science and technology and the government role in enhancing these assets are a growing emphasis for governments around the world. As science and technology capabilities have become more broadly distributed, there is a need for the U.S. to monitor the U.S. enterprise against an international backdrop to detect declines in national capabilities in science and technology relative to other nations or to identify new opportunities for research investment that merit public support. The National Academies have urged regular international benchmarking at the field level to assess the health of individual fields of research in the U.S. in their 1993 report, Science, Technology and the Federal Government/National Goals for a New Era and their 1995 report, Allocating Federal Funds for Science and Technology. The use of international comparisons of the productivity of research fields and international expert participation in assessments of research programs are common in other countries.

The Board has noted the need for monitoring the relative health of U.S. science and technology as part of a continuing evaluation of the Federal portfolio, drawing on existing data and expert analyses, and continually improving data and methods for international comparisons that inform priority setting.

Understanding the Role of Federal Research in Producing Economic and Other Benefits

A large number of studies have attempted to elucidate, and in many cases measure quantitatively, the relationship between research and innovation and the benefits of research for society. Organizations like the Council on Competitiveness, the Science and Technology Policy Institute, RAND; OSTP, and NSF have explored issues and methods for analyzing the role of a range of factors in innovation--including Federally funded research--and resulting economic and social benefits. Academic programs for addressing these questions are inadequately funded. This is an area where additional research investment could improve both qualitative and quantitative data to inform budget allocation decisions, communicate the benefits of research to the public, and contribute to the effectiveness of Federal research investments.