Dr. Cora Marrett
NSF Major Research Equipment and Facilities Management
March 8, 2012
Chairman Brooks, Ranking Member Lipinski, and distinguished Members of the Subcommittee, thank you for inviting me to participate in this hearing on “NSF Major Research Equipment and Facilities Management: Ensuring Fiscal Responsibility and Accountability."
I am pleased to have the opportunity to discuss the National Science Foundation's (NSF) large facility process with you.
As you well know, NSF supports research at the frontiers of knowledge across all fields of science and engineering (S&E) and all levels of S&E education. Its mission, vision and goals are designed to maintain and strengthen the vitality of the U.S. science and engineering enterprise.
Throughout its 60 year history, NSF has contributed to maintaining U.S. leadership in science and engineering research by enabling the creation of advanced instrumentation and world-class multi-user facilities for the science and engineering research community.
NSF's investments in multi-user facilities are designed to provide unique, transformational research capabilities at the frontiers of science and engineering. The NSF multi-user facility portfolio spans experimental disciplines from physics and astronomy, engineering, and materials science, to earth and ocean sciences, polar sciences and biological sciences. The newest facilities comprise distributed sensor arrays, extensive cyber-infrastructure, and streaming data networks on continental scales. In addition to enabling immense scientific return, multi-user facilities serve as platforms to train the next generation of scientists and engineers, and provide the high technology equipment and services necessary for economic growth and innovation.
The Major Research Equipment and Facilities Construction (MREFC) account was established in 1995 to support the acquisition, construction, and commissioning of large-scale facility projects. NSF requires that each MREFC candidate project represent an outstanding opportunity to enable breakthrough research and innovation, as well as education and broader impacts. Each should offer the possibility of transformative knowledge and the potential to shift existing paradigms in scientific understanding, engineering processes and/or infrastructure technology. And each must serve an urgent contemporary research need that will persist through the process of planning and development.
NSF takes its facility stewardship responsibilities very seriously. Implementation of the largest NSF multi-user facilities requires investments of hundreds of millions of dollars. To ensure success at this major scale of investment, NSF has strong processes in place for overseeing the planning, construction, and operations of its facilities, and for managing its overall facilities portfolio. NSF has recently taken steps to make these processes even stronger. I am pleased to describe the large facility process in detail below.
The NSF large facility process from initial planning to operation
NSF enables and oversees the creation and operation of major multi-user facilities through a defined set of funding mechanisms, stewardship policies, and management processes. A number of National Academies of Science reports on federal science facility project development1 provided the foundations for NSF’s current process.
As illustrated in the figure below, large facility projects under consideration for MREFC funding undergo a multi-stage development, review and approval process. This process is fully defined in NSF's guideline document, the Large Facilities Manual2. Note that MREFC funds support only the Construction Stage of an approved facility project; preconstruction planning and design of a potential facility project and post-construction operations and maintenance of the facility are funded through the Research and Related Activities (R&RA) budget of the sponsoring Directorate or Office.
Horizon Planning Stage: Ideas for potential large facilities originate in the scientific community and typically evolve conceptually over many years. Initially, NSF responds to opportunities articulated by the research community. These ideas are subjected to external merit review, and may receive funding by the cognizant Science and Engineering Directorate or Offices for further early- (pre-conceptual design) stage refinement. In parallel, the relevant scientific community may coalesce around a preferred concept or a prioritization of competing potential projects, and these preferences will be communicated to NSF via workshops, advisory committees, and authoritative reports from professional societies and the National Academies.
The sponsoring NSF Directorate or Office may determine that a proposed horizon stage facility addresses high priority science goals and deserves investment in conceptual design and development according to the guidelines of NSF's large facility planning process. As illustrated in the figure above, the Directorate or Office may at this point request internal NSF review of the proposed project by the senior-level MREFC Panel3, which in turn makes a recommendation to the NSF Director who may approve or decline the project's advancement to the Conceptual Design Stage. Key criteria are: the demonstration of a strong science case and readiness to undertake conceptual design activities defined by NSF guidelines. Approval only allows initiation of the first stage of formal preconstruction planning; it does not constitute a commitment by NSF to develop the project further.
Preconstruction Planning Stages: NSF preconstruction planning for potential NSF large facility projects comprises three stages: Conceptual Design, Preliminary Design and Final Design. The objective of preconstruction planning activities is to shepherd a project from the conceptual stage to construction readiness and approval. Activities include project planning and design by NSF awardees, and internal planning and oversight by NSF program management, NSF administrative units and NSF senior management. The sponsoring NSF program also establishes a strategy for supporting long-term facility operations, including sun setting, based on the operations plans and associated cost estimates developed by the project team.
As illustrated in the above figure, each preconstruction planning stage concludes with a corresponding formal review conducted by NSF utilizing an external independent panel of experts selected and charged by the responsible NSF Program Officer, in consultation with the NSF Large Facilities Office. The expert panel evaluates the scientific, technical, business, and post-construction operations plans, and the associated cost estimates for these according to NSF guidelines, and advises the Program Officer as to whether the project is ready to advance to next stage. The corresponding decision milestones comprise evaluation by the sponsoring Directorate or Office, review and recommendation by the MREFC Panel, and decision by the NSF Director to advance the project. The National Science Board also reviews and may endorse the Director’s advancement decisions, as noted further below. The decision milestones also constitute "off ramps" for terminating the project if progress is not deemed satisfactory or NSF's plans or priorities change.
Conceptual Design Stage: NSF requires the project team to develop conceptual designs that include the definition of science goals and objectives the proposed facility will address, and the respective science, technical and functional requirements that are essential to achieve the research objectives. The project team must also develop an initial top-down budget estimate; an assessment of major risks; an analysis of potential partnerships for facility development, construction and/or operation; and an initial estimate of the future budget needed to operate the proposed facility. The sponsoring NSF program also develops an Internal Management Plan that defines its stewardship strategy--including plans for oversight during the later more-intensive stages of pre-construction planning, funding the design activity and defining off-ramp criteria should project development not progress as planned.
At the appropriate time, NSF conducts a Conceptual Design Review (CDR) to determine if the project is ready to advance to the Preliminary Design Stage. Note again that the NSF Director's approval only allows investment in preliminary design activities; it does not constitute a commitment by NSF to develop the project further.
Preliminary Design Stage (also, Readiness Stage): If approved by the Director and the Board, and funds have been requested by NSF and appropriated by Congress, the sponsoring Program may fund the project team to develop a preliminary design, which comprises development of a detailed scope of work--via a technical design and development activities, development of a detailed Work Breakdown Structure (WBS), a resource-loaded project execution schedule, identification of risks and a comprehensive risk management plan, and a bottom-up "risk adjusted" cost estimate composed of the baseline estimate and a contingency budget added to the estimate for changes that experiences shows will likely be required, taking those risks into account.
At the appropriate time, NSF conducts a Preliminary Design Review (PDR) to determine if the project is ready to advance to the Final Design Stage. The review comprises evaluation of the technical and management plans and cost estimate; the credibility of the proposed project team and institutional partners to carry out the project; and the proponent's projection of future operating plans and associated cost estimates. The outcome of the PDR process establishes the project's baseline, which is used as the basis for the request to Congress for MREFC funds.
Advancement entails a request by the Director to the National Science Board to include the facility project in a future NSF Budget Request for MREFC Funds. Board approval effectively constitutes a commitment by NSF that it intends to fund the facility project. Following Board approval, the project is proposed to the Office of Management and Budget (OMB) for inclusion in a future NSF Budget Request to Congress, and is advanced to the Final Design Stage. Should OMB concur, NSF subsequently presents the project--including the overall plan, total cost, requested multi-year MREFC funding profile, and estimated out-year operations funding profile--in the President's NSF BudgetRequest to Congress.
Final Design Stage (also, Board Approved Stage): The period between Preliminary Design Review and appropriation of funds typically requires at least two years. During the Final Design Stage, the project team is funded to continue to refine cost estimates, recruit additional construction staff, finalize partnership commitments, and complete other preparatory work that must be accomplished prior to commencing construction. During this time, NSF conducts annual cost update reviews to ensure that the assumptions underlying the baseline definition continue to be valid.
Around the time appropriated funds become available, NSF conducts a Final Design Review (FDR) to verify that the project is fully prepared to undertake construction activity, and that the project cost estimate continues to be valid. If the review is successful, NSF requests approval from the National Science Board to obligate funds to commence construction.
Construction Stage: During this stage, NSF receives monthly technical and financial status reports, and performs comprehensive reviews and site visits of the project at least annually. NSF requires projects to report their project cost and schedule performance using earned value methodology, and to report all changes to the project baseline, including budget, schedule, and scope changes. NSF also approves requested changes and/or calls on budget contingency that exceed pre-determined thresholds. The National Science Board is notified if any requested changes exceed ten percent of the total project cost. If the project is judged not to be performing satisfactorily, NSF may require its project awardee to implement a Corrective Action Plan or perform rebaselining, or NSF may take other actions including termination.
Operations stage: Facility operational activity begins following construction and commissioning of the new facilities, or in many cases concurrently with the completion of those activities. Generally, the entity responsible for constructing and commissioning the facility also has responsibility for initial operation. NSF continually reviews facilities as they develop and during full operations to ensure that the activities comprising the operational stage fulfill scientific goals within the available funding. NSF also requires operating facilities to develop annual work plans that set annual performance goals, to measure their performance against those goals, and report the results to NSF each year. In keeping with National Science Board guidance, NSF also promotes excellence and efficiency in facility operation by encouraging full and open recompetition of the subsequent award for continued operation and maintenance.
NSF's capacity to support ongoing operation and research utilization of its facilities--including investments in advanced R&D to maintain the vitality of facilities--are the pacing factors in the ability of NSF to support new research infrastructure. NSF and the National Science Board continually assess current infrastructure to determine what should be maintained and where redirection towards new opportunities is appropriate. Blue Ribbon Panels, Advisory Committees, commissioning of external studies, and periodic facility reviews are all part of this process.
NSF staff roles in large facility stewardship: For each facility, a designated Program Officer within the sponsoring Directorate or Office executes stewardship responsibilities and serves as the NSF principal point of contact throughout the facility's life cycle. The Large Facilities Office, located within the Chief Financial Officer's front office and headed by the Deputy Director for Large Facility Projects, develops NSF's large facility process guidelines and assists program staff and senior management in project and facility portfolio planning and oversight. The Budget Division and Grants and Agreements Specialists in the CFO’s organization engage in budget development, and in pre-award planning and post-award oversight, respectively.
Role of the National Science Board4: The National Science Board provides oversight throughout the entire life-cycle process for planning, constructing, operating, and eventually terminating NSF support for large facilities. It prioritizes among competing projects in preconstruction planning and relative to other opportunities, endorses project advancement from one preconstruction planning stage to the next, approves NSF's request to OMB to include a request for construction funding within a future NSF Budget Request to Congress, and approves the obligation of funds to commence construction following a Congressional appropriation. The Board conducts annual portfolio reviews of all NSF major multi-user facility projects at post-Conceptual Design Review stages of planning and those in construction and operation, offering guidance to NSF on the balance between investments in research infrastructure and support for other activities.
Recent modifications to the large facilities process
With the Fiscal Year 2009 Budget Justification, NSF implemented a "no cost overrun policy" designed to increase rigor in the planning and execution of projects within approved cost estimates. As stated in the FY 2013 Budget Justification, this policy requires that "(1) the total cost estimate for each project at the preliminary design stage include adequate contingency to cover all foreseeable risks, and (2) any total project cost increases not covered by contingency be accommodated by reductions in scope, provided that the actual enacted funding levels have been consistent with the established project profiles.... If the total cost for a project is revised during construction for reasons other than inadequate funding, NSF will identify mechanisms for offsetting any cost increases in accordance with the no overrun policy."
In FY 2011, NSF implemented additional changes to its large facility process to improve handling of early-stage projects. The main objectives were to (1) clarify the distinction between "horizon" projects and projects in the formal "Conceptual Design Stage," (2) enhance the agency-wide strategy for enabling interdisciplinary planning at the early stage, and (3) fully align the facility planning and budget planning processes.
Previously, NSF did not make a strong distinction between what it called "horizon projects" and those in the defined preconstruction planning stages. While this practice maximized Directorate and Office flexibility to exploit facility opportunities in various stages of maturity, it was challenging for the agency to monitor the programmatic status of these early-stage concepts, and communicate these uniformly to stakeholders and potential partners.
As shown in the earlier figure, NSF now defines a "Horizon stage" prior to the "Conceptual Design Stage." Horizon projects are those priority activities under consideration within Directorates/Offices, but which are not yet ready for advancement through the MREFC preconstruction planning process. Projects in the Conceptual Design Stage follow Large Facilities Manual guidelines, and receive R&RA funding specific to developing the Conceptual Design of a potential future MREFC project.
Second, NSF now defines an agency-level transition milestone between the "Horizon Stage" and "Conceptual Design Stage," that is, an explicit entry point to the initial MREFC preconstruction planning stage that mirrors the transition review/approval process for all other stages. This comprises two steps: first, a formal review by the MREFC Panel of requests by Directorates or Offices to advance horizon projects to the Conceptual Design Stage; and second, a decision by the NSF Director--supported by the Panel's recommendation--on whether (and how) the project should advance. The Panel's review is conducted according to an established procedure and set of evaluation criteria.
These modifications enhance the roles of the Director and senior agency management in reviewing and advancing early-stage projects, and serve to ensure that adequate resources are invested at this early planning stage. In making these changes, NSF has also incorporated a requirement that sponsoring Directorates and Offices demonstrate increased early-stage planning and timeline development in order to strengthen alignment with the budget process; and engagement with the OMB and the Office of Science and Technology Policy (OSTP) as appropriate, particularly for interagency partnered projects.
It is important to emphasize again that this approval for advancement of a horizon project into the Conceptual Design Stage of planning does not imply NSF commitment to implement the project beyond that stage. As mentioned earlier, such NSF commitment does not come until after the successful conclusion of the Preliminary Design Stage and approval by the National Science Board.
Impacts on approved projects of underfunding of the MREFC construction account
NSF strives to make best use of the funds entrusted to it. To this end, NSF gives all projects in construction a higher priority than those proposed for future construction. NSF also develops multi-year project construction budgets that are "technically limited," that is, budgeted according to the optimum rate that work can be performed in order to obtain the lowest total project cost. The MREFC account has increased and decreased from year to year as a consequence of budgeting to meet project needs rather than planning the project activities according to a predetermined budget.
The success of this budgeting practice assumes that NSF will receive all of the MREFC funds requested from Congress for an approved project in a given year. When this does not happen, a project's plans must be adjusted accordingly, potentially leading to increased costs and reduced project scope and science capability.
If MREFC funding is less than requested, NSF gives highest priority to funding those projects farthest along in construction while deferring work on projects just getting underway, in order to avoid the added costs of suspending and restarting work in progress. NSF executed this policy in FY2011 when the enacted MREFC budget ($117 million) was less than requested ($165 million).
Selected Lessons Learned
Over many years of sponsoring the creation and operation of major multi-user research facilities, NSF has gained substantial insight into facility development best practices, and endeavors to adopt these uniformly as they are recognized. Here I note several "take-away" lessons that stand out in particular and that influence NSF's planning, investment, and oversight of major facility projects.
Experience at NSF and across the federal science facility enterprise confirms that adequate investment in preconstruction planning is essential to achieving a project's intended scope within its estimated budget. NSF experience is consistent with observations at other agencies that an amount equal to 10 to 25 percent of the total capital cost must be invested in preconstruction planning in order to assure that the project is well planned, risks are effectively understood and planned for, and that a credible project team is assembled and able to implement the proposed project.
Exceptional projects demand exceptional individuals in key positions of responsibility. Advisory and external review committees are very valuable as means to vet management capabilities. In addition, major projects must adequately invest in modern management tools. In particular, Project Management Control Systems are essential for determining the project's technically limited construction schedule and the associated funding profile, and so that, once in construction, the project manager can effectively ascertain technical and financial status, obtain a detailed picture of risks and contingency usage, and provide the necessary transparency to the agency needed to carry out an effective oversight role.
Finally, large facility projects often expend two-thirds, or more, of their total budget as subawards and subcontracts to other parties. Consequently, it is extremely important that during planning the project team develop effective plans for subawardee and subcontract monitoring and oversight, including Quality Assurance and Safety.
Mr. Chairman, the world-class equipment and facilities that NSF supports are essential to the task of discovery, and are vital to NSF accomplishing its mission of supporting fundamental U.S. science and engineering research. NSF will continue to enhance its policies and practices for stewardship of major research facilities and other infrastructure in concert with the evolving needs of the scientific community for these unique assets and capabilities.
I appreciate the opportunity to appear before the Subcommittee to speak to you on this important topic. I would be pleased to answer any questions that you may have.
1.Setting Priorities for Large Research Facility Projects Supported by the National Science Foundation, NRC 2004 (and see NSF-NSB responding report of the same title, 2004); Improving Project Management in the Department of Energy, NRC, 1999 (and see succeeding publications in this series through 2005).
Return to statement.
3. NSF's senior-level MREFC Panel is composed of the NSF Deputy Director, Directorate Assistant Directors and Office Heads, and also includes the heads of the Large Facilities Office and Budget Division. More details on the Panel's role and responsibilities can be found in the NSF Large Facilities Manual.
Return to statement.
4. Please refer to the testimony of Dr. José-Marie Griffiths at this Hearing for further details concerning the Board's role.
Return to statement.