Did You Know? A Review Analysis is Written for Every Proposal Processed at NSF
November 8, 2017
Carol Frost, Director
Division of Earth Sciences
Before I worked at the National Science Foundation (NSF), I did not realize that program directors write a Review Analysis for every proposal reviewed at NSF. This document includes a summary of the proposal, the review process and the rationale for the decision (Step 6 in the NSF Proposal and Award Process). The review analysis addresses both the intellectual merit and broader impacts of the proposed research, includes a summary of ad hoc and panel reviews; and, if needed, incorporates a discussion of any disparities between ad hoc and panel reviews. Through its merit review process, NSF ensures that proposals submitted are reviewed in a fair, competitive, transparent and in-depth manner.
As Division Director, I rely on the review analysis to provide a justification for the recommendations provided to me by program directors. The review analysis enables me to review a program director’s recommendation with confidence (Step 7 in the NSF Proposal and Award Process). It documents that the merit review process was conducted rigorously and fairly. The justification of the decision is shared with principal investigators as part of the "PO comments." Once the review process is finished, principal investigators have access to the ad hoc reviews, panel summary and PO comments. See NSF's Merit Review website for more details on the merit review process.
NSF Committees of Visitors (COV) also read review analyses to help assess the quality and integrity of NSF program operations, including the execution of the merit review process and how the outputs and outcomes generated by awardees contribute to attaining NSF’s mission and strategic goals. NSF convenes Committees of Visitors—committees of external experts—to examine the merit review process and make sure that NSF continues to meet high standards of program management. Each program at NSF must be reviewed by a COV at least once every four years.
The Division of Earth Sciences’ (EAR) COV met in June 2017 and studied the review record of a random selection of 406 proposals representing five percent of proposals submitted to programs in EAR between fiscal years 2013 and 2016. The COV looked at the quality of the ad hoc reviews, the panel summaries and the review analyses. The 17 COV members commented on the merit review process, selection of reviewers, management of the programs and the balance of awards across the division. In their report, they applauded the quality, consistency and integrity of EAR’s merit review process, program management, and the range and scope of EAR’s resulting portfolio of awards. They also had recommendations for making a good process even better.
EAR’s Committee of Visitors report and the division’s response are public documents. Feel free to take a look. I think you will be impressed by the thorough review undertaken by your geoscience colleagues on the Division of Earth Sciences’ COV.
The U.S. National Science Foundation propels the nation forward by advancing fundamental research in all fields of science and engineering. NSF supports research and people by providing facilities, instruments and funding to support their ingenuity and sustain the U.S. as a global leader in research and innovation. With a fiscal year 2023 budget of $9.5 billion, NSF funds reach all 50 states through grants to nearly 2,000 colleges, universities and institutions. Each year, NSF receives more than 40,000 competitive proposals and makes about 11,000 new awards. Those awards include support for cooperative research with industry, Arctic and Antarctic research and operations, and U.S. participation in international scientific efforts.