Bypass Navigation


Performance Results

Compared with FY 1999, in FY 2000 NSF was much more rigorous in evaluating goal achievement. Options for grading were limited to either successful or not successful, and full justifications were required for successful grades to be counted for those goals that used qualitative measures. For the Outcome Goals, PricewaterhouseCoopers LLP verified and validated the goal achievement data tables. While NSF was successful in achieving 64% of its goals in FY 2000 as compared with achieving 78% for FY 1999, the results of the second year are very similar to the first. Positive trends were evident in some of the goals, indicating movement in the desirable direction. The areas identified as needing improvement continue to be: (1) use of both merit review criteria by reviewers and applicants; and (2) the customer service goals such as decreasing time to decision on proposals. Both these areas will be focal points in FY 2001.

line
FY 2000 Performance Results
  Number of Goals Achieved
line
Outcome Goals 6 out of 8 (75%)
line
Management Goals 5 out of 6 (83%)
line
Investment Process Goals 7 out of 14 (50%); one goal did not apply
line
Total 18 out of 28 (64%)


Results for NSF's Outcome Goals
Six of the eight Outcome Goals were achieved in FY 2000. In FY 1999, all Outcome Goals were achieved. Overall, results are similar to those obtained in FY 1999, with trends beginning to appear in this second year of assessment. Reports by external evaluators indicate that NSF successfully achieved the first two Outcome Goals (Goal 1 and Goal 2), and achieved with limited success the second two Outcome Goals (Goal 3 and Goal 4a). FY 2000 evaluators identified the same areas as having limited success and in need of improvement as in FY 1999. In general, programs are showing improvement over FY 1999 performance in the area of increasing diversity through increased participation of underrepresented groups, but reports indicate that the numbers are still lower than expected. The evaluators commented that increasing participation of underrepresented groups is an area needing more attention for NSF. Other areas needing further improvement include: (1) balance of portfolio by taking more risk; and (2) use of the NSF’s merit review criteria by reviewers and applicants. Several reports noted that there are clear indications that use of the merit review criteria is evident in making decisions to fund or not fund applications. Common issues identified in some reports that may result in negative impact on program performance, in general, include workload and delays in processing proposals.


Results for NSF's Management Goals
Five of NSF's six Management Goals were achieved in FY 2000, compared with three out of five in the prior year. Areas identified as improving include orientation and training of NSF staff using FastLane, NSF's electronic system for proposal submission, proposal review, and project reporting; and increasing the use of the new electronic Project Reporting System for project reporting by awardees. The one Management Goal that was not achieved involves the technological capability to submit proposals electronically. The difficulty encountered in FY 2000 which prevented this goal from being achieved was related to the establishing of protocols for electronic signature. NSF piloted two models for electronic certification of proposals and is currently assessing which model will best serve the agency.


Results for NSF's Investment Process Goals
Seven of NSF's Investment Process Goals were achieved in FY 2000; seven were not achieved and, as in FY 1999, one of the facilities management goals did not apply because there were no construction projects completed during the year. In FY 1999, nine Investment Process Goals were achieved, four were not achieved and as previously mentioned, one did not apply. Areas identified as needing improvement include use of the new merit review criteria in some programs; identifying best practices and training for improving customer service; allowing three months time to prepare proposals; decreasing the time to decision; increasing the percentage of awards to new investigators; maintaining facility upgrades and construction on schedule; and keeping operating time lost due to unscheduled downtime to less than 10% of the total scheduled operating time.


The following chart lists NSF’s FY 2000 GPRA goals and results. For a more detailed explanation of these goals and results, see the section on “Performance Results and Related Issues.”

Annual Performance Goals for Outcome Results
line
Outcome
FY2000 Annual
Performance Goals

NSF is judged successful when
Aggregated Results
line line line

Outcome Goal 1
Discoveries at and across the frontier of science and engineering

Performance Goal 1
NSF awards lead to important discoveries; new knowledge and techniques, both expected and unexpected, within and across traditional disciplinary boundaries; and high-potential links across these boundaries, as judged by independent external experts.

Baseline: Experiments using FY 1997 and FY 1998 information indicated successful achievement.
FY 1999: Goal achieved. Judged successful by external experts in all reports.
FY 2000: Goal achieved. Reports by external experts indicate NSF is successful in achieving this goal in the aggregate.

line

line

line

Outcome Goal 2
Connections between discoveries and their use in service to society

Performance Goal 2
The results of NSF awards are rapidly and readily available and feed, as appropriate, into education, policy development, or use by other federal agencies or the private sector, as judged by independent external experts.

Baseline: Experiments using FY 1997 and FY 1998 information indicated successful achievement.
FY 1999: Goal achieved. Judged successful in the aggregate by external experts who noted improvements could be made in some programs.
FY 2000: Goal achieved. Judged successful in the aggregate by external experts who noted improvements could be made in some programs, as in FY 1999.

line

line

line

Outcome Goal 3
A diverse, globally-oriented workforce of scientists and engineers

Performance Goal 3
Participants in NSF activities experience world-class professional practices in research and education, using modern technologies and incorporating international points of reference; when academia, government, business, and industry recognize their quality; and when the science and engineering workforce shows increased participation of underrepresented groups, as judged by independent external experts.

Baseline: Experiments using FY 1997 and FY 1998 information indicated successful achievement.
FY 1999: Goal achieved. Judged successful in most areas by external experts.
FY 2000: Goal judged successful in the aggregate by external experts with respect to achieving a globally oriented workforce, and not fully successful with respect to achieving diversity or increased participation of underrepresented groups, therefore goal is successful in a limited context but not fully achieved. FY 2000 results indicate improvements over FY 1999 performance, but improvements are still needed in the same areas identified in FY 1999.
For FY 2001, this goal has been incorporated into a broader goal that focuses on achieving NSF’s desired outcome of a diverse, internationally competitive and globally engaged workforce of scientists, engineers, and well-prepared citizens.

line

line

line

Outcome Goal 4
Improved achievement in mathematics and science skills needed by all Americans

Performance Goal 4a
NSF awards lead to the development, adoption, adaptation, and implementation of effective models, products, and practices that address the needs of all students; well-trained teachers who implement standards-based approaches in their classrooms; and improved student performance in participating schools and districts, as judged by independent external experts.

Baseline: Preliminary pilot efforts did not provide
sufficient information to yield a valid baseline.
FY 1999: Goal achieved. Judged successful in the aggregate by external experts for programs to which goal applies.
FY 2000: Goal judged successful in a limited context in the aggregate by external experts. Where programs did not have funds directed to these objectives, external evaluators were uncertain how to assess performance, resulting in an assessment of less than successful or no assessment. In FY 2001, performance measures/indicators for this goal will be better defined to eliminate confusion by evaluators.

line

line

Performance Goal 4b
Over 80% of schools participating in a systemic initiative program will: implement a standards-based curriculum in science and mathematics; further professional development of the instructional workforce; and improve student achievement on a selected battery of tests, after three years of NSF support.

FY 1999: Goal achieved.
FY 2000: Goal achieved.

line

line

Performance Goal 4c
Through systemic initiatives and related teacher enhancement programs, NSF will provide intensive professional development experiences annually for at least 65,000 pre-college teachers.

FY 1999: Goal achieved.
FY 2000: Goal achieved.

line

line

line

Outcome Goal 5
Timely and relevant information on the national and international science and engineering enterprise.

Performance Goal 5a
Maintain FY 1999 gains in timeliness for an average of 486 days the time interval between reference period (the time to which the data refer) and reporting of data.

FY 1995-96
Baseline 540 days

FY 1999-2000
Goal 486 days
Actual 461 days

FY 1999: Goal achieved.
FY 2000: Goal achieved.

line

line

Performance Goal 5b
Establish a standard set of data quality measures for reporting of Science Resource Studies (SRS) products. Prepare reports on these measures for all SRS surveys and publish them in electronic formats to inform users of SRS data quality. New in FY 2000, replacing the FY 1999 goal on relevance.

Baseline: None prior to goal setting.

New in FY 2000
FY 2000: Goal achieved.

line

line

line

 

Annual Performance Goals for Management
line
New and Emerging
Technologies
Critical Factors
for Success
Aggregated Results
line line line
Electronic proposal submission

Management Goal 1
NSF will receive at least 60% of full proposal submissions electronically through FastLane.

FY 1998 Baseline 17%
FY 1999 44%
FY 2000 Goal 60%
FY 2000 Result 81%

FY 1999: Goal achieved.
FY 2000: Goal achieved.

line

line

line

Electronic proposal processing Management Goal 2
By the end of FY 2000, NSF will have the technological capability to take competitive proposals submitted electronically through the entire proposal and award/declination process without generating paper within NSF.

New in FY 2000
FY 2000: Goal not achieved.

In FY 2001, NSF will be testing use of an electronic signature for funding approval, the one remaining barrier to a completely electronic processing of awards.

line

line

line

 

NSF Staff
Critical Factors
for Success
Aggregated Results
line line line
Diversity

Management Goal 3
In FY 2000, NSF will show an increase over 1997 in the total number of hires to science and engineering (S&E) positions from underrepresented groups. (Revised goal.)

FY 1997 Baseline: Of 54 S&E hires, 22% were female and 19% were from underrepresented minority groups.

FY 2000 Result: Of 113 S&E hires, 35 were female and 19 were from minority groups. Compared with FY 1997 baseline, this represents a 120% increase in female hires and a 27% increase in minority hires.

FY 1999: Goal achieved.
FY 2000: Goal achieved.

line

line

line

Capability in use of information technology Management Goal 4
By the end of FY 2000, all staff will receive an orientation to FastLane, and at least 80% of program and program support staff will receive practice in using its key modules.

Orientation
FY 1999 80%
FY 2000 Goal 100%
FY 2000 Result 100%

Training
FY 1999 43%
FY 2000 Goal 80%
FY 2000 Result 90%

FY 1999: Goal not achieved.
FY 2000: Goal achieved.

line

line

line

 

Implementation of
Management Reforms
Critical Factors
for Success
Aggregated Results
line line line

Year 2000 Compliance

Management Goal 5
NSF will complete all activities needed to address the Year 2000 problem for its information systems according to plan, on schedule and within budget. (Revised goal for FY 2000.)

FY 2000 Result: All activities needed to address the Year 2000 problem were completed according to plan, on schedule, and within budget.

FY 1999: Goal achieved.
FY 2000: Goal achieved.

line

line

line

Project Reporting System

Management Goal 6
During FY 2000, at least 85% of all project reports will be submitted through the new electronic Project Reporting System.

FY 1999 59%
FY 2000 Goal 85%
FY 2000 Result 92%

FY 1999: Goal achieved; target revised for FY 2000.
FY 2000: Goal achieved.

line

line

line

 

Annual Performance Goals for NSF’s Investment Process
line

Performance Area:
Proposal and
Award Processes

FY 2000 Annual Investment Process Performance Goals

Aggregated Results

line line line

Use of Merit Review

Investment Goal 1
At least 90% of NSF funds will be allocated to projects reviewed by appropriate peers external to NSF and selected through a merit-based competitive process.

FY 1998 (Baseline) 95%
FY 1999 95%
FY 2000 Goal 90%
FY 2000 Result 95%

During FY 2000, OMB redefined what constitutes a merit-reviewed project and established a new target goal of 70-90%.

Revised FY 2000 Goal 80% (est.)
FY 2000 Result 87%

FY 1999: Goal achieved.
FY 2000: Goal achieved.

line

line

line

Implementation of Merit Review Criteria

Investment Goal 2
NSF performance in implementation of the new merit review criteria is successful when reviewers address the elements of both generic review criteria appropriate to the proposal at hand and when program officers take the information provided into account in their decisions on awards, as judged by external independent experts.

Results: About one-third of evaluation reports rated NSF programs as successful in their use of the new merit review criteria. In most cases where NSF was rated not fully successful, reviewers and applicants were not fully addressing the second criterion regarding the broader impacts of the proposed activity.

FY 1999: Goal achieved.
FY 2000: Goal not achieved.

Full implementation of goal is a priority in FY 2001. A number of measures are being taken to ensure its achievement, e.g., different on-screen pages have been provided in FastLane so reviewers are guided to address each merit review criterion separately; performance data will be collected from the FastLane database; etc.

line

line

line

Customer Service: General

Investment Goal 3
Identify possible reasons for customer dissatisfaction with NSF’s merit review system and with NSF’s complaint system.

Results: NSF commissioned surveys in order to ascertain possible reasons for customer dissatisfaction.

New in FY 2000.
FY 2000: Goal achieved.

line

line

line

Customer Service: General

Investment Goal 4
Identify best practices and training necessary for NSF staff to conduct merit review and answer questions about the review criteria and process; identify best practices and training necessary for NSF staff to answer questions from the community and to deal with complaints in a forthright manner.

Results: Goal underway but not completed in FY 2000; plans to finalize implementation in FY 2001.

New in FY 2000.
FY 2000: Goal not achieved.

In FY 2001, staff will continue to develop models of best practices and staff training; NSF will pilot the best models at division level and provide specific customer service training to NSF staff.

line

line

line

Customer service: General

Investment Goal 5
Improve NSF's overall American Customer Satisfaction Index (ACSI) compared to the FY 1999 index of 57 (on a scale of 0 to 100).

FY 1999 57
FY 2000 Goal > 57
FY 2000 Result 58

New in FY 2000.
FY 2000: Goal achieved.

Ongoing commitment to improve results; see previous Goals 3&4

line

line

line

Customer service: Time to prepare proposals

Investment Goal 6
95% of program announcements and solicitations will be available at least three months prior to proposal deadlines or target dates.

FY 1998 Baseline 66%
FY 1999 75%
FY 2000 Goal 95%
FY 2000 Result 89%

Although this goal was not achieved, there is notable improvement from prior year. In FY 2000, 89% of program announcement/solicitations achieved goal; approximately 8% missed the 90-day time limit by fewer than 5 days.

FY 1999: Goal not achieved.
FY 2000: Goal not achieved.

In FY 2001, staff will limit number of special competitions requiring individual announcements; plan further in advance; initiate clearance process at least 6 months prior to anticipated deadlines; clearance procedures will be reviewed.

line

line

line

Customer service: Time to decision

Investment Goal 7
Maintain the FY 1999 goal to process 70% of proposals within six months of receipt, improving upon the FY 1998 baseline.

FY 1998 Baseline 59%
FY 1999 58%
FY 2000 Goal 70%
FY 2000 Result 54%

In FY 2000, 54% of proposals were processed within 6 months of receipt and an additional 35% were processed between 6 to 9 months of receipt.

FY 1999: Goal not achieved.
FY 2000: Goal not achieved.

In FY 2001, staff will make more effective use of electronic mechanisms in conducting reviews; more closely track processing; eliminate overloads/bottlenecks.

line

line

line

Maintaining Openness in the System Investment Goal 8
The percentage of competitive research grants going to new investigators will be at least 30%.

FY 1998 Baseline 27%
FY 1999 27%
FY 2000 Goal 30%
FY 2000 Result 28%

FY 1999: Goal not achieved.
FY 2000: Goal not achieved.

In FY 2001, NSF staff will pursue outreach efforts to promote awareness of NSF research opportunities; undertake analysis of trends (e.g., whether pool of new investigators is getting smaller, etc.) to determine whether goal needs to be modified.

line

line

line

 

Performance Area:
Integration of Research and Education

FY 2000 Annual Investment Process Performance Goals

Aggregated Results

line line line

In Proposals

Investment Goal 9
NSF will develop a plan and system to request that Principal Investigators (PIs) address the integration of research and education in their proposals, and develop and implement a system to verify that PIs have done so.
(Revised goal.) No baseline.

Result: In FY 2000, NSF implemented an electronic program announcement template clearance process (PAT) that is used by NSF staff to generate announcements and solicitations. Use of the PAT ensures that the integration of research and education is emphasized in all announcements and solicitations for PIs to address in their submissions.

New in FY 2000.
FY 2000: Goal achieved.

line

line

line

In Reviews

Investment Goal 10
NSF will develop and implement a system/mechanism to request and track reviewer comments tied to merit review criterion #2, “What are the broader impacts of the proposed activity?”
(Revised goal; no baseline.)

Result: In FY 2000, screens in FastLane were redesigned so that reviewers can address each merit-review criterion separately. The performance data will be collected from the FastLane database. This will be fully implemented in FY 2001.

New in FY 2000.
FY 2000: Goal achieved.

line

line

line

 

Performance Area:
Diversity

FY 2000 Annual Investment Process Performance Goals

Aggregated Results

line line line

NSF Applicants

Investment Goal 11
NSF will identify mechanisms to increase the number of women and underrepresented minorities in the proposal applicant pool, and will identify mechanisms to retain that pool.
(Revised goal; no baseline.)

Result: NSF identified and put into place mechanisms to increase the diversity of NSF applicants.

New in FY 2000.
FY 2000: Goal achieved.

line

line

line

 

Performance Area:
Facilities Oversight

FY 2000 Annual Investment Process Performance Goals

Aggregated Results

line line line

Construction and Upgrade

 

 

Investment Goal 12
Maintain 1999 goal to keep construction and upgrades within annual expenditure plan, not to exceed 110% of estimates.

FY 1999 Result: Majority of facilities were within 110% of annual spending estimates.

FY 2000 Result: Of the 11 construction and upgrade projects supported by NSF, all were within annual expenditure plans; most were under budget.

FY 1999: Goal achieved.
FY 2000: Goal achieved.

line

line

Investment Goal 13
Maintain 1999 goal to keep construction and upgrades within annual schedule, total time required for major components of the project not to exceed 110% of estimates.

FY 1999 Result: Majority of facilities on schedule.

FY 2000 Result: Of the 11 construction/upgrade projects supported by NSF, seven were within the annual schedule goal.

FY 1999: Goal achieved.
FY 2000: Goal not achieved.

NSF program managers will work more closely with project managers to ensure compliance in FY 2001.

line

line

Investment Goal 14
For all construction and upgrade projects initiated after 1996, keep total cost within 110% of estimates made at the initiation of construction.

FY 1999 and FY 2000:
There were no completed projects, therefore, this goal did not apply.

line

line

line

Operations

Investment Goal 15
Maintain 1999 goal to keep operating time lost due to unscheduled downtime to less than 10% of the total scheduled operating time.

FY 1999 Result: Reporting database under development.

FY 2000 Result: Of the 26 reporting facilities, 22 met the goal of keeping unscheduled downtime to below 10% of the total scheduled operating time.

FY 1999: Inconclusive.
FY 2000: Goal not achieved.

NSF program managers will work more closely with project managers to ensure compliance in FY 2001.

 



 

 

Policies and Important Links

|

Privacy | FOIA | Help | Contact NSF | Contact Web Master | SiteMap  

National Science Foundation

The National Science Foundation, 4201 Wilson Boulevard, Arlington, Virginia 22230, USA
Tel: (703) 292-5111, FIRS: (800) 877-8339 | TDD: (800) 281-8749

Last Updated:
01/24/13
Text Only