This document has been archived.
T he mission of the Instrumentation and Laboratory Improvement program is to increase the range and quality of modern laboratory equipment used in undergraduate education. The program addresses its mission by supporting innovative curriculum development projects that show promise for improving the quality of undergraduate education in science, mathematics, engineering, and technology. From 1985 through 1994, the program has supported more than 4,500 such projects.
The results of this evaluation show that overall, the ILI program has largely achieved its mission; although the types and strength of impacts vary for different categories of undergraduate institutions, for different SMET fields, and for different target audiences. To return to the metaphor that guided our description of program impacts, the seeds have been widely sown, the seedlings have taken hold and grown, and the greenery has spread beyond the initial acreage.
Nonetheless, as in any innovative adventure, be it a laboratory experiment or a new teaching practice, not all ILI projects are equally successful, and a similar investment of time and money has quite different results depending on the context in which projects operate. This pattern of outcomes includes both merits and limitations that NSF may want to consider in shaping the program for the future. Specifically:
The ILI program has reached a wide range of institutions and fields. Not only have a majority of four-year institutions received one or more ILI awards during the program's first 10 years, but the awards have been equitably distributed across individuals and SMET fields. In fact, fewer than 10 percent of ILI grantees have won more than one award.
However, two-year institutions-institutions with programs that reach more students and especially minority students-have relatively lower application rates and, despite their comparable proposal funding rates, are not as strongly represented. Only 40 percent of the two-year colleges that award associate degrees in SMET fields had submitted one or more proposals to ILI by 1994, and less than 20 percent had received one or more awards.
Further, despite the fact that in each year the ILI program announcement has made a special point of encouraging proposals involving consortia, there has been relatively little interest in equipment-sharing proposals or consortia, judging by the small number of proposals submitted and eventually funded.
The laboratory equipment and curricular innovations supported by ILI grants serve large numbers of undergraduate students in the grantee institutions through improved learning- opportunities. This impact is perhaps the strongest and most consistent finding across the full range of ILI projects. Nearly all 1990 and 1992 PIs reported using their project to reconfigure, expand, and update existing laboratory courses, and almost half use the equipment as a base for creating new courses. Consequently, approximately 140 under-graduates per project receive direct, hands-on experience with ILI equipment each year in their coursework during a project's first four years. Some of the most widespread impacts were found at institutions receiving multiple awards.
In addition, ILI grants were found to stimulate extensive resource leveraging, nearly five times the original grant amount, at the grantee institutions. This finding confirms that the grantee institutions and other organizations and foundations that provide the matching and leveraging funds for ILI projects recognize the value and importance of the laboratory improvement projects.
However, mixed impacts were reported on the faculty reward system. Project PIs spend a large amount of time developing their projects, approximately 430 hours per project on average. And although some PIs reported improved prospects for tenure, promotion, and salary increases, in most cases PIs reported that their efforts went unrecognized and unrewarded by administrators at their institutions.
Further, the "snowball" effect of receiving multiple awards has been limited to a few institutions. Thus, the opportunity for widespread impacts similar to those reported at institutions such as Rochester Institute of Technology and Calvin College has been relatively rare.
PIs receiving ILI awards spread the word about their projects through a wide range of dissemination activities, including presentations of papers at regional or national meetings, publications in journals and other professional media, presentations or colloquia at other institutions, and workshops or short courses to acquaint others with project-developed approaches and materials.
Although our tracer study examination of exemplary dissemination efforts showed instances of instructional innovations being adopted by workshop attendees and other identifiable second-stage users of project-generated products throughout the nation, most ILI PIs are unable to confirm whether and to what extent their disseminated information and materials have been adopted at institutions beyond the host institution.
The impacts of the program on unsuccessful applicants were also examined. More than three-quarters of the curricular improvement projects that were submitted as proposals to ILI in 1990 and 1992 ultimately became implemented to some degree, either with ILI funding support or with support from other sources. However, unsuccessful applicants who reapplied to the program were often critical of the feedback they had received from proposal reviewers and the inconsistency of selection criteria from year to year.
While there is little that NSF can do to address some of these limitations, there are clear actions to be considered. This evaluation identified four areas where the program fell short of its goals:
Clearly program decisions that seek to capitalize on these findings will require establishing priorities and tradeoffs. Thus, two-year institutions could be encouraged more actively to submit applications. Aside from resulting in a more equitable coverage of eligible institutions, the findings of this evaluation
suggest that such an action would probably also result in
An alternate model for strengthening the capacity of the ILI program to achieve broader systemic impacts would involve greater coordination among programs in NSF's Division of Undergraduate Education, and the provision of incentives for undergraduate faculty to seek multiple grants from complementary programs. This would allow ILI to cultivate many more of the multiplicative effects observed in those tracer studies and exemplary project site visits in which ILI grants were supplemented with grants from other DUE programs serving similar target audiences. The tradeoff here would involve ILI and other DUE programs sacrificing a certain degree of autonomy within programs.
Similarly, if the review process placed a higher priority on proposals involving increased research opportunities, shared-equipment consortia, or community outreach-areas this evaluation found to be relatively weak-quite different program outcomes could be expected.
There are probably a number
of additional tradeoffs that will be suggested by the study findings to
program officials and other stakeholders. Whatever changes are ultimately
made, this evaluation provides evidence that the program's first decade
of activity has been rich and fertile, and that the seed money provided
by NSF has yielded value well beyond its initial investments.
| Table of Contents | Start of Chapter 1 |
| Start of Chapter 2 | Start of Chapter 3 |