Executive Summary

Context and Framework for Study

The Federal Government’s policy for investment in science and technology over the last 50 years has yielded enormous benefits to the economy, national security, and quality of life in the U.S. The Federal share of total national science and technology investment is critically focused in areas that would be inadequately funded or not supported by the private sector. These include research to support Federal missions; research that is high-risk or requires long-term investment in the expectation of future high payoffs to society; unique, costly, cutting-edge research facilities and instruments; and academic research that, as a primary purpose, supports the education of the future science and engineering workforce.

Over $90 billion (1) was allocated to Federal R&D in the most recent budget—representing a little more than a quarter of all national R&D. With such a large investment of public funds, policy makers in Congress and the Executive branch are asking for convincing evidence of the effectiveness of Federal investments in the form of hard data on benefits. There is general recognition among policy makers that outstanding opportunities for excellent research far exceed any reasonable level of funding by the Federal government. Choices must be made. Wise, well-informed choices among alternatives will sustain a strong, balanced research infrastructure to enable the discoveries that will be a foundation for future prosperity.

The current system for priority setting in the Federal research budget lacks a coherent, scientifically based process for systematic review and evaluation of the broad Federal investment portfolio for effectiveness in achieving national goals. Moreover available data and analyses are often ill suited for informing budget allocation decisions that affect U.S. research infrastructure.

Decision makers must rely on the scientific community to provide the best advice on the most promising research investment choices for the future. The form and timing of such advice are also important. Appropriate advice must include a reasonable estimate of the level of funding that would be required for adequate support of a new initiative over time, provide tradeoff options to enable funding for priorities, and be available on a schedule compatible with the Federal budget process.

No process now exists for weighing the available evidence on competing research investment opportunities across broad fields of research. It is critical that the choices among such opportunities be based on a process that is transparent and credible with the scientific communities and the general public and its representatives. Such a function requires an organizational home, appropriate expert resources, and adequate financial support.

Since the mid 1990s, the National Science Board has been actively engaged in issues of priority setting for the Federal research portfolio (2). In 1999, the Board charged its Ad Hoc Committee on Strategic Science and Engineering Policy Issues to undertake a study of research budget coordination and priority setting methodologies across fields of science and engineering in the U.S. and in other countries.

Conduct of Study

The study, Federal Research Resources: A Process for Setting Priorities (NSB 01-156), which follows on recommendations of the Board’s previous working paper on Government Funding of Scientific Research [NSB 97-186), responds to a request by the House Appropriations Committee (3) and the encouragement of the Office of Management and Budget. In its November 1998 Strategic Plan (NSB-98-215) the Board identified this effort as a high priority for national science policy.

The Committee on Strategic Science and Engineering Policy Issues commissioned reviews of the literature in two areas. (4) The first focused on Federal research budget coordination, priority setting across fields of science and engineering, and available data and analytical tools to support priority setting. A second study of the same subject reviewed international models of S&T budget coordination and priority setting. It also included a symposium with presentations by S&T officials from eight foreign governments.

In addition to these studies, the Committee heard presentations by invited experts who discussed a wide range of methodologies and data to support budget allocation decisions for research. It also received written comments on its draft recommendations by mail and through the National Science Board website, and heard presentations broadly representative of stakeholders in Federal research. Stakeholder input culminated with a Symposium on May 21-22, 2001 on the Board’s preliminary findings and recommendations, with more than 200 participants.

Principal Findings

• Federal priority setting for research occurs at three levels: 1) establishing Federal goals for research, 2) the budget allocation processes for research within the White House and Congress that in the aggregate produce the Federal research portfolio and 3) Federal agencies and departments in achieving their missions in accord with the President’s priorities for research. This report focuses on the second level, that is, the White House and Congressional processes that in the aggregate produce the Federal portfolio of investments in research.
• The allocation of funds to national research goals is ultimately a political process that should be informed by the best scientific advice and data available.
• A strengthened process for research allocation decisions is needed. Such allocations are based now primarily on faith in future payoffs justified by past success. They are difficult to defend against alternative claims on the budget that promise concrete, more easily measured results and are supported by large and vocal constituencies.
• The pluralistic framework for Federal research is a positive aspect of the system and increases possibilities for funding high-risk, high-payoff research. An improved process for budget coordination and priority setting should build on strengths of the current system and address weaknesses in data, analyses, and expert advice.
• There is a need for regular evaluation of Federal investments as a portfolio for success in achieving Federal goals for research, to identify areas of weakness in national infrastructure for S&T, and to identify a well-defined set of the top priorities for major new research investments.
• Additional resources are needed to provide both Congress and the Executive branch with data, analyses, and expert advice to inform their decisions on budget allocations for research.
  

Recommendations

Implementation of a broad-based, continuous capability for expert advice to both OMB and Congress during the budget process would yield immediate benefits to decision makers. There is also a long-term need for a regular, systematic evaluation of the effectiveness of Federal investments in achieving Federal goals for research through the Office of Science and Technology Policy, drawing broad-based input from scientific experts and organizations in all sectors. Complementing both would be improved analyses on research opportunities, needs, and benefits to society; and timely data that trace Federal research investments through the budget process and beyond.

Keystone Recommendation 1

The Federal Government, including the White House, Federal departments and agencies, and the Congress should cooperate in developing and supporting a more productive process for allocating and coordinating Federal research funding. The process must place a priority on investments in areas that advance important national goals, identify areas ready to benefit from greater investment, address long-term needs and opportunities for Federal missions and responsibilities, and ensure world class fundamental science and engineering capabilities across the frontiers of knowledge. It should incorporate input from the Federal departments and agencies, advisory mechanisms of the National Academies, scientific community organizations representing all sectors, and a global perspective on opportunities and needs for U.S. science and technology.

Research Community Input on Needs and Opportunities:

Presently there is no widely accepted and broadly applied way for the Federal Government to obtain systematic input from the science and engineering communities to inform budget choices on support for research and research infrastructure. The current system often fails to produce advice and information on a schedule useful to the budget process and responsive to needs for broad-based, informed assessments of the benefits and costs of alternative proposals for Federal support. A more effective system for managing the Federal research portfolio requires adequate funding, staffing and organizational continuity.

Recommendation 2

A process should be implemented that identifies priority needs and opportunities for research—encompassing all major areas of science and engineering—to inform Federal budget decisions. The process should include an evaluation of the current Federal portfolio for research in light of national goals, and draw on: systematic, independent expert advice from the external scientific communities; studies of the costs and benefits of research investments; and analyses of available data; and should include S&T priorities, advice, and analyses from Federal departments and agencies. The priorities identified would inform OMB in developing its guidance to Federal departments and agencies for the President’s budget submission, and the Congress in the budget development and appropriations processes.

Executive Branch Advisory Mechanism:

The Executive Branch should implement a more robust advisory mechanism, expanding on and enhancing current White House mechanisms for S&T budget coordination and priority setting in OSTP and OMB. It is particularly essential that the advisory mechanism include participants who are experienced in making choices among excellent opportunities or needs for research, for example, vice provosts for research in universities, active researchers with breadth of vision, and managers of major industrial research programs.

Recommendation 2a

An Executive Branch process for ongoing evaluation of outcomes of the Federal portfolio for research in light of Federal goals for S&T should be implemented on a five-year cycle. (5) A report to the President and Congress should be prepared including a well-defined set of the highest long-term priorities for Federal research investments. These priorities should include new national initiatives, unique and paradigm shifting instrumentation and facilities, unintended and unanticipated shifts in support among areas of research resulting in gaps in support to important research domains, and emerging fields. The report should also include potential trade-offs to provide greater funding for priority activities. The report should be updated on an annual basis as part of the budget process, and should employ the best available data and analyses as well as expert input. Resources available to OSTP, OMB and PCAST should be bolstered to support this function.

Congressional Advisory Mechanism:

There is no coherent congressional mechanism for considering allocation decisions for research within the framework of the broad Federal research portfolio. Though improvements in the White House process—particularly expansion of activities and resources available to OSTP—would benefit congressional allocation decisions, one or more congressional mechanisms to provide expert input to research allocation decisions are badly needed.

Recommendation 2b

Congress should develop appropriate mechanisms to provide it with independent expert S&T review, evaluation, and advice. These mechanisms should build on existing resources for budget and scientific analysis, such as the Congressional Budget Office, the Congressional Research Service, the Government Accounting Office, and the National Academies. A framework for considering the full Federal portfolio for science and technology might include hearings by the Budget Committees of both houses of Congress, or other such broadly based congressional forums.

Definitions, Data and Data Systems:

High quality data and data systems to monitor Federal investments in research would enhance the decision process. Such systems must be based on definitions of research activities that are consistently applied across departments and agencies and measured to capture the changing character of research and research needs. Improving data will require long-term commitment with input from potential users and contributors, and appropriate financial support.

Recommendation 3

A strategy for addressing data needs should be developed. Such a strategy supported by OMB and Congress and managed through OSTP and OMB would assure commitment by departments, agencies and programs to timely, accessible data that are reliable across reporting units and relevant to the needs for monitoring and evaluating Federal investments in research. Current data and data systems tracking federally funded research should be evaluated for utility to the research budget allocation process and employed as appropriate.

International Comparisons:

Both relative and absolute international statistical data and assessments should be a major component of the information base to support Executive Branch and Congressional research budget allocation decisions. International benchmarking of U.S. research performance and capabilities on a regular basis responds to the growing globalization of science and technology and the need for the U.S. to maintain a world-class science and engineering infrastructure.

Recommendation 4

Input to Federal allocation decisions should include comparisons of U.S. research resources and performance with those of other countries. National resources and performance should be benchmarked to evaluate the health and vigor of U.S. science and engineering for a range of macroeconomic indicators, using both absolute and relative measures, the latter to control in part for the difference in size and composition of economies. Over the long term, data sources should be expanded and quality improved.

Federal Research Benefits to the Economy and Society:

In addition to monitoring Federal expenditures for research, measuring the benefits to the public of funded research is essential for prudent management. Implementation of this recommendation should be coordinated with Recommendation 3 on definitions and data systems.

Recommendation 5

The Federal Government should invest in the research necessary to build deep understanding and the intellectual infrastructure to analyze substantive effects on the economy and quality of life of Federal support for science and technology. The research should include improvements to methods for measuring returns on public investments in research.

Conclusion

The Board’s recommendations provide a framework for improving the quality, content, and accessibility of science and engineering expert advice, data, and analyses to inform decisions on priorities in the White House and Congress for Federal investments across fields of research. We are aware that implementing these recommendations will be difficult and require long-term commitment and support. In the interest of science and the Nation, we urge that the Federal Government and its partners in the research community embrace this difficult task.

¹Executive Office of the President, Office of Management and Budget, “Analytical Perspectives,” Fiscal Year 2002 Budget of the United States Government, Table 7-2, Research and Development Spending.

² The National Science Board issued In Support of Basic Research—NSB 93-127; Federal Investments in Science and Engineering—NSB 95-254 and Statement on Federal R&D Budget Realignment—NSB 95-26, from 1993 to 1995, in addition to more recent papers.

³ House Appropriations Committee Report 105-610, 105th Congress, 2nd Session, U. S. House of Representatives. To accompany H.R. 4194.

⁴ Steven W. Popper, Caroline S. Wagner, Donna L. Fossum, William S. Stiles. Setting Priorities and Coordinating Federal R&D Across Fields of Science: A Literature Review (DRU-2286-NSF). Washington DC: RAND Science and Technology Policy Institute, April 2000; and H. Roberts Coward. Final Report: Symposium on International Models of Budget Coordination and Priority Setting for S&T. Washington DC: SRI International, August 2000.

⁵ The designation of a five-year cycle for evaluation of the Federal portfolio reflects both the scale of the effort, which would require a longer time than an annual process, and the increasingly rapid changes in science that demand a frequent reevaluation of needs and opportunities
for investments.