IV. CONCLUSIONS AND RECOMMENDATIONS

U.S. global leadership in science and technology has contributed enormously to national wealth and the quality of life. Today the capabilities for cutting edge science and engineering are becoming distributed more broadly throughout the industrialized and developing world. To maintain U.S. leadership, Federal funding is critical for long-term, high risk, and academic research, as well as unique research facilities and instrumentation. The White House and Congress must employ the best available information and scientific advice in research budget allocation decisions to assure the continued strength of the U.S. science and technology.

A deliberate, scientifically grounded process is essential for identifying opportunities and needs for research. Needs include human resources, instrumentation and facilities, alignment of the portfolio of Federal investments with national priorities for research, effective distribution of funding among research modes and performing organizations, closure of gaps in research resulting from changes in department and agency programs, and addressing patterns of underinvestment in vital areas of fundamental research.

Federally funded science and technology support the missions of every Federal department and agency, and have enormous long-term impacts on the economy and the quality of life of American citizens. The explosion of knowledge and new technologies over the last decade has expanded opportunities for breakthroughs in a broad range of science and engineering fields. The growth in opportunities for discovery and innovation requires the Federal government to make wise, well-informed choices for funding allocations that provide the greatest benefits over the long term to American society. Making those choices requires forward looking advice by experts, evaluation of the current Federal portfolio, and an information system to track investments and outcomes and to evaluate long-term opportunities as well as current needs for support in a global as well as domestic context.

The Board finds that the mechanisms that have evolved based on the OSTP Act of 1976 and the cooperation between OSTP and OMB represent valuable progress toward a more coherent and sophisticated system to inform major decisions on Federal research investments. Strengthening the OMB/OSTP/PCAST mechanisms would provide a strong information base for congressional decisions. Additional, complementary resources to provide timely expert and data input to the congressional budget processes are also needed.

RECOMMENDATION 1: KEYSTONE RECOMMENDATION

The Federal government, including the White House, Federal departments and agencies, and the Congress should cooperate in developing and supporting a more productive process for allocating and coordinating Federal research funding. The process must place a priority on investments in areas that advance important national goals, identify areas ready to benefit from greater investment, address long-term needs and opportunities for Federal missions and responsibilities, and ensure world class fundamental science and engineering capabilities across the frontiers of knowledge.

Research Community Input on Needs and Opportunities

Steps can be taken in the short term to improve the information base for Federal research investments. Effective funding decisions must be based on the best possible information on scientific opportunities and needs, and on national goals for science and engineering. A primary input to any process of priority setting for research is expert scientific advice on current and long-term opportunities and needs for research. Presently there is no widely accepted and broadly applied way for the Federal government to obtain systematic input from the science and engineering communities for making priority decisions about support for research and research infrastructure.

There is insufficient opportunity and capability within the framework of existing mechanisms for Federal research priority setting to undertake timely and broad-based assessments of the needs for Federal investments. A more effective system for managing the Federal research portfolio requires adequate funding, staffing and organizational continuity.

RECOMMENDATION 2:

A process should be implemented that identifies priority needs and opportunities for research--encompassing all major areas of science and engineering--to inform Federal budget decisions. The process should include an evaluation of the current Federal portfolio for research in light of national goals, and draw on systematic, independent expert advice, studies of the costs and benefits of research investments, and analyses of available data. The priorities identified would inform OMB in developing its guidance to Federal departments and agencies for the President's budget submission, and the Congress in the budget development and appropriations processes.

1. Executive Branch Advisory Mechanism

The Executive Branch should implement a more robust advisory mechanism, expanding on and enhancing current White House mechanisms for S&T budget coordination and priority setting. Resources available to OSTP, OMB and PCAST should be bolstered to permit more timely, broad-based data analysis to produce a comprehensive evaluation of the Federal portfolio for science and technology research on a five-year cycle¹, updated annually as input to the Federal budget process. Enhanced resources should include an adequate professional staff. A model to consider is the Council of Economic Advisors, which is supported by a rotating professional staff. It is particularly essential that the advisory mechanism include participants who are experienced in making choices among excellent opportunities or needs for research. (For example, vice provosts for research in universities, active researchers with breadth of vision, and managers of major industrial research programs would be appropriate in this role.)

RECOMMENDATION 2a:

An Executive Branch process for periodic evaluation of outcomes of the Federal portfolio for research in light of Federal goals for S&T should be implemented on a five-year cycle. A report to the President and Congress should be prepared including a list of the highest long-term priorities for Federal research investments. These priorities should include new national initiatives, unique and paradigm shifting instrumentation and facilities, unintended and unanticipated shifts in support among areas of research resulting in gaps in support to important research domains, and emerging fields. The report should also include potential trade-offs to provide greater funding for priority activities. The report should be updated on an annual basis as part of the budget process, and should employ the best available data and analyses as well as expert input.

2. Congressional Advisory Mechanism

Congress is inadequately supported in making allocation decisions for research. Analytical resources for science and technology policy tailored to Congressional needs have been eliminated or reduced in recent years, while the need for such resources is growing. There is no mechanism for considering allocation decisions for research within the framework of the broad Federal research portfolio. The current system splits areas of research among committees, each considering a limited portion of the portfolio. Though improvements in the White House process would benefit Congressional allocation decisions, a Congressional mechanism to provide expert input to research allocation decisions is badly needed.

RECOMMENDATION 2b:

Congress should develop an appropriate mechanism to provide it with independent expert S&T review, evaluation, and advice.

The advisory process should make use of the best available data and analyses in deriving its recommendations on Federal priorities and funding levels.

Long-Term Investments in Improved Data and Analyses

In addition to an enhanced process for expert advice and assessment, there is a long-term need to improve tools -- databases and analytic methods - for effective management of the Federal research portfolio.

1. Definitions, Data and Data Systems

High quality data and data systems to monitor Federal investments in research would enhance the decision process. Such systems must be based on definitions of research activities that are consistently applied across departments and agencies and measured to capture the changing character of research and research needs. Flexibility in defining categories of research for tracking purposes is especially important for monitoring emerging areas and addressing the range of modes for research - from the individual investigator to the major center or facility. Timely collection of data and ease of access are critical to be useful to the allocation decision process.

Improving data and data systems is a long-term objective, but one that is necessary and increasingly urgent for managing the large, diverse Federal research portfolio to serve the Nation. It will require long-term commitment to improve data systems, with input from potential users and contributors, and appropriate support.

RECOMMENDATION 3:

A strategy for addressing data needs should be developed. Such a strategy supported by OMB and Congress would assure commitment by departments, agencies and programs to timely, accessible data that are reliable across reporting units and relevant to the needs for monitoring and evaluating Federal investments in research. Current data and data systems tracking federally-funded research should be evaluated for utility to the research budget allocation process and employed as appropriate.

2. International Comparisons

Both relative and absolute international statistical data and assessments should be included as a major component of the information base to support Executive branch and Congressional research budget allocation decisions. International benchmarking of U.S. research performance and capabilities on a regular basis responds to the growing globalization of science and technology and the need for the U.S. to maintain a world class science and engineering infrastructure. Maintaining world class capabilities enables the Nation to take advantage of opportunities for rapid advancements in knowledge in targeted areas of research and to capitalize on breakthroughs wherever they occur worldwide. Although international data and methods of analysis are limited, they should be employed with sensitivity to those limitations and with a long-term commitment to developing better methods and data for monitoring U.S. performance and strength in science and technology.

International comparisons should include a range of measures of national research resources and performance to produce objective assessments of the relative strength of the U.S. in research areas important to national goals. For example, comparisons could include total national S&T investment as a share of Gross Domestic Product (GDP) or as a share of the high technology sector of the economy. Relative performance of individual fields important to national economic or defense priorities can be assessed using bibliometric methods and patent citations. Comparisons should be sensitive to the appropriate basis for comparing different economies, since the composition of the economy may be as important as its size as measured by GDP. For example, it might not be appropriate to compare S&T/GDP ratios for two economies that have very different manufacturing shares of total GDP. Of central importance is the comparison of human resources for research in priority areas in the U.S. and in other countries, including international migration of science and engineering personnel as well as participation by U.S. students in science and engineering studies in comparison with other nations.

Statistical trends are critical for evaluating the adequacy and direction of national research investments. Comparisons might include the following types of relative and absolute statistics:

• Total national S&T; Defense S&T; Civilian S&T; Basic (fundamental) research: National (US) and Federal;

• Civilian S&T by functional categories of: health, energy, environment and natural resources, space research and technology, general science, transportation, agriculture;

• Basic science investment categories, such as: engineering, natural sciences, social science, and mathematical sciences; and

• Human resources engaged in or available for research by field, degree attainment, gender and nationality.

RECOMMENDATION 4:

Input to Federal allocation decisions should include comparisons of U.S. research resources and performance with those of other countries. National resources and performance should be benchmarked to evaluate the health and vigor of U.S. science and engineering for a range of macroeconomic indicators, using both absolute and relative measures, the latter to control in part for the difference in size and composition of economies. Over the long term, data sources should be expanded and quality improved.

3. Federal Research Benefits to the Economy and Society

In addition to monitoring Federal expenditures for research, measuring the benefits to the public of funded research is essential for prudent management. Although there is an extensive literature on methods for measuring returns on research investments, usually in the private sector, these methods have not been widely applied in the Federal context for a number of reasons. With regard to economic methods, the difficulties include lack of sufficient data, questions of data quality, selection bias in case studies of specific industries and problems of time lags between research discoveries and their impacts on the economy. In the case of publicly supported research, many benefits cannot be expressed in terms of economic returns. Indicators and methods that have been used for measuring benefits of research include the following:

• Asset-oriented measures, which tally such system "assets" as research facilities and human resources for S&T resulting from Federal investments-for example, immigrant and native-born scientists and engineers, and graduate students supported on Federal research grants;

• Outputs measures, which track intellectual contributions and often employ bibliometric analysis-such as patent citations, publication counts, article citations, presentations at conferences--or honors received by researchers and research projects, e.g. Nobel prizes;

• Outcomes or results measures, including: (1) case studies and retrospective analyses, which are usually qualitative, tracing the inputs and the processes that produced an important innovation and (2) quantitative economic techniques such as production function analysis and surveys estimating economic impacts of public research within specific industries and enabling a better understanding of the channels and mechanisms whereby public research contributes to innovation.

Implementation of this recommendation should be coordinated with Recommendation 3 on definitions and data systems.

RECOMMENDATION 5:

The Federal government should invest in the research necessary to build the intellectual infrastructure in the higher education sector to analyze substantive effects on the economy and quality of life of Federal support for science and technology. The research should include improvements to methods for measuring returns on public investments in research.

Federal support for research has been highly successful in contributing to the quality of life that we enjoy in the U.S. today. Continued national commitment to publicly supported research offers the promise of even greater benefits in the future. The expanding frontiers of knowledge demand careful evaluation to identify the highest priorities for investment of Federal research funds. It is therefore essential that the processes by which allocation decisions are made rest on the best possible information base that high technology and well prepared minds can produce. The systematic participation of the scientific community in this process, bringing its vision and understanding of the needs and opportunities for research, is critical to its success. The Board's recommendations describe a strategy for improving the quality, content, and accessibility of science and engineering input to decisions on the allocation of Federal research funds. We are aware that implementing these recommendations will be difficult and require long-term commitment. In the interest of science and the Nation, we urge that the Federal government and its partners in the research community embrace this difficult task.

_____________________

¹ The designation of a five-year cycle for evaluation of the Federal portfolio reflects both the size of the effort, which would require more than an annual process, and the rapid changes in science, which demand a frequent reevaluation of needs and opportunities for investments.