Academic Research and Development Expenditures: Fiscal Year 2009
Appendix A. Technical Notes
During the production of this report, the America COMPETES Reauthorization Act of 2010 was signed into law. Section 505 of the bill renames the Division of Science Resources Statistics as the National Center for Science and Engineering Statistics (NCSES). The Center retains its reporting line to the Directorate for Social, Behavioral and Economic Sciences within the National Science Foundation. The new name signals the central role of NCSES in the collection, interpretation, analysis, and dissemination of objective data on the science and engineering enterprise.
The National Science Foundation (NSF) Survey of Research and Development Expenditures at Universities and Colleges (Academic R&D Expenditures Survey) is the primary source of information on separately budgeted R&D expenditures within academic institutions in the United States and outlying areas. Conducted annually since FY 1972, the survey collects information on R&D expenditures by academic field as well as by source of funds. The results of the survey are primarily used to assess trends in R&D expenditures across the fields of science and engineering (S&E). This information is vital for federal, state, and academic planners who make decisions on future R&D funding priorities. Unless otherwise noted, expenditures analyzed in this report refer to S&E R&D activities only.
Scope of the Survey
The FY 2009 Academic R&D Expenditures Survey was sent to 722 institutions of higher education in the United States, Guam, Puerto Rico, and the U.S. Virgin Islands. These institutions granted a bachelor's degree or higher in S&E fields and expended at least $150,000 in separately budgeted S&E R&D in FY 2009.
The survey population was reviewed before data collection began to ensure that each institutional classification was accurate. Characteristics of the schools were reviewed before and during the course of the survey to determine whether changes had occurred (e.g., highest degree granted; school openings, closings, or mergers).
After data collection closed, institutions were reviewed to verify that only those reporting at least $150,000 in separately budgeted R&D were included in the population. Of the 722 institutions surveyed, 11 reported total R&D expenditures of less than $150,000. These 11 institutions were excluded from the population, and their data are not included in the FY 2009 detailed statistical tables. The total and federally financed R&D expenditures for these 11 institutions are listed in table A-1.
FY 2009 Survey Frame Design
The Academic R&D Expenditures Survey is a census of the full population of eligible academic institutions. Prior to FY 1998 a census of eligible institutions was conducted approximately every 5 years; during intervening years eligible institutions were sampled. Since then a census has been conducted annually. NSF also conducts a population review each year to ensure that all institutions that meet the inclusion criteria are surveyed. In FY 2004 the inclusion criteria were changed, and since then the survey population has included all institutions that grant a bachelor's degree or higher in S&E fields and that reported at least $150,000 in separately budgeted S&E R&D expenditures in the previous fiscal year. Each year NSF contacts institutions that meet the degree-granting criterion but were not in the population of the previous census to determine whether they meet the $150,000 expenditure criterion. Institutions with a minimum of $150,000 are added to or retained in the survey population. As a result of this step, the overall number of academic institutions in the population increased from 690 in FY 2008 to 711 in FY 2009 (table A-2).
Most major R&D-performing institutions have incorporated the data that are needed to complete this survey into their record-keeping systems, thereby ensuring a consistent format from one year to the next. Such consistency yields the most useful statistics for time series. As a rule, information to complete this survey is found in the institutions' year-end accounting records.
Item 1 is a request for total current expenditures for separately budgeted S&E R&D expenditures by source of funds. Institutions are also asked to provide the percentage of total expenditures and federally financed expenditures that are considered basic research. Also included are research funds for which an outside organization, educational or other, is a subrecipient.
Care should be taken when interpreting data on source of funds. Industry R&D support is limited to grants and contracts for R&D activities from profit-making organizations. Total industry funds exclude research funded through unrestricted accounts and from corporate foundations, endowments, and fellowships to students; those funds would be included in an institution's own funding totals. In addition, public institutions may have differing policies for determining whether general state appropriations are reported as state or institutional funding.
Item 1A is a request for total and federally financed current fund expenditures for separately budgeted S&E R&D passed through the institution to subrecipients.
Item 1B is a request for total and federally financed current fund expenditures for separately budgeted S&E R&D received by the institution as a subrecipient.
Item 2 is a request for total and federally financed current fund expenditures for separately budgeted R&D activities by detailed S&E field. When interpreting these data at the detailed discipline level, users should keep in mind that there is considerable interdisciplinary and multidisciplinary activity.
Item 2A is a request for total and federally financed current fund expenditures for separately budgeted non-S&E R&D activities by detailed non-S&E field. This item was originally added to the survey as an optional item in FY 1997 and was collected as a core item beginning in FY 2003.
Item 2B is a request for federally financed current fund expenditures for separately budgeted R&D activities by S&E field and federal agency. This item was originally added to the survey as an optional item in FY 1998 and was collected as a core item beginning in FY 2003.
Item 3 is a request for the portions of total and federally financed expenditures reported in items 1 and 2 that were used for the purchase of research equipment. This portion includes all research equipment purchased under sponsored research project awards and disbursed in the same detailed disciplines as in item 2. These data are of special interest to federal and institutional policymakers in determining current funding levels for scientific research instrumentation.
Item 4 is new to the FY 2009 questionnaire and is a request for the number and amount of R&D projects awarded by source of funds. This item includes awards for research in all S&E and non-S&E fields and the total amount awarded in FY 2009, even if the funds will be spent over multiple years. Source of funding categories are (1) federal stimulus (American Recovery and Reinvestment Act of 2009 [ARRA]) funds, (2) other federal funds, and (3) nonfederal funds. Because of inconsistencies across institutions in reporting total award amount, these data are not included in this report.
Forty surveyed institutions (table A-3) did not complete the Academic R&D Expenditures Survey instrument, but instead they responded to a pilot version of a new survey instrument, the Higher Education Research and Development (HERD) Survey. The HERD Survey results from an effort by NSF to redesign the current survey. Much of the data requested as part of the HERD Survey were the same as those requested on the Academic R&D Expenditures Survey. Expenditures collected as part of the HERD pilot are included in the detailed statistical tables (DSTs). When necessary, pilot institutions were asked to provide additional information in order to have complete data in the DSTs. Institutions included in the HERD pilot were not included in the data collection procedures that follow. For more information about the HERD Survey instrument and data collection procedures, contact the NSF project manager.
The FY 2009 survey questionnaires were sent by e-mail in November 2009. Respondents could choose to submit an Adobe Portable Document Format questionnaire downloaded from the Web or use a Web-based data collection system to respond to the survey. Every effort was made to maintain close contact with respondents in order to preserve both the consistency and continuity of the resulting data. Questionnaires were carefully examined for completeness upon receipt. Computerized facsimiles of the survey data were then prepared for each institution; these compared the current and 2 prior years of data and noted any substantive disparities. Respondents were sent personalized e-mail messages asking them to provide any necessary revisions before the final processing and tabulation of data. These e-mail messages included a link to the Academic R&D Expenditures Survey Web-based data collection system, allowing respondents to view and correct their data online.
Respondents were asked to explain significant differences between current-year reporting and established patterns of reporting verified for prior years. They were encouraged to correct prior-year data if needed. When respondents updated or amended figures from past years, NSF made corresponding changes to trend data in this report and to the underlying microdata database. Similarly, if a respondent institution underwent an organizational change, such as a merger, NSF incorporated the effects of the change into prior years' data.
By the survey's closing date in April 2010, forms had been received from 694 universities and colleges out of the academic population of 711, a response rate of 97.6%. Responses were received from 98.3% of all doctorate-granting institutions. The R&D expenditures reported by these doctoral institutions constituted 98.3% of the estimated national R&D expenditures in S&E fields for FY 2009. Table A-4 displays a detailed breakdown of response rates by highest degree granted, and table A-5 displays a breakdown of response rates by survey item for the academic population.
National Total and Imputation
Two statistical procedures were used to provide a national estimate for all universities and colleges performing R&D in FY 2009. First, for the 14 institutions that had not responded by the closing date of the survey and had been included in the FY 2008 population, data for items 1, 2, and 3 were estimated by imputation using techniques that have been used consistently since FY 1976. These techniques use prior years' figures derived from the data of respondent institutions with similar characteristics, including highest degree granted and type of institutional control (public or private). Data were not imputed for three institutions that did not respond. These institutions were not part of the FY 2008 population, and there were no previous year's data to impute from. These institutions were included in population counts but do not appear in DSTs. Second, data for items 1, 2, and 3 were imputed for universities and colleges that submitted only partial responses. This imputation was performed using prior years' figures for each institution to allocate that institution's FY 2009 total R&D expenditures reported within the various categories. The imputed total R&D was $71 million, or 0.1%, of the $55 billion in all separately budgeted R&D expenditures (table A-6).
Tables A-7 and A-8 present imputed amounts for total and federally financed R&D expenditures, respectively, by S&E field. The dollar amount imputed is displayed along with the percentage it represents of the national estimate for universities and colleges in a particular field. The amount imputed is similarly broken down by source of funds in table A-9.
A number of surveyed institutions have responded intermittently in past years. For years in which no response was received, data have been imputed as previously described. Although the imputation algorithm accurately reflects national trends, it cannot account for specific trends at individual institutions. For this reason, a separate estimation of institutional data for prior years is also performed, following current-year imputation.
For each institution, formerly imputed key variables for items 1–3 were recomputed to ensure that the imputed data accurately represent the growth patterns shown by reported data. If, for example, data were reported for FY 2005 and FY 2009 but not for the intervening years, the difference between the reported figures for each item total was calculated and evenly distributed across the intervening years (2006–08). The new figures were spread across disciplines (items 2 and 3) or sources of support (item 1) on the basis of the most recent reporting pattern. These procedures result in much more consistent reporting trends for individual institutions but have little effect on aggregate figures reflecting national totals.
Changes in Basic Research Totals
The Division of Science Resources Statistics at NSF regularly reviews the methodologies used in the Academic R&D Expenditures Survey with the goal of producing the most accurate statistics possible for researchers and policymakers. Reviews of responses to item 1, question 2, were conducted in FY 2001, 2002, 2003, and 2004. These reviews determined that the aggregate statistics on basic research could be improved by refining the imputation methodology for that item. Some universities and colleges are either unable or unwilling to respond to this question, and values must be imputed in order to present aggregate statistics.
In the past, if a respondent did not reply to the basic research question, the prior year's basic research share (whether reported by or imputed for the respondent) was carried forward. After interviews with respondents revealed that abnormal or erroneous values were sometimes carried forward for several years, a revised imputation methodology began being used in FY 2004. The revised imputation methodology uses an econometric model to impute the amount of total and federal basic research. The model accounts for differences between public and private institutions and nonfederal sources of R&D funding.
Item 1A Analysis
Item 1A, R&D passed through the institution to subrecipients, was completed by 96.3% of the respondents from academic institutions (table A-10). The total R&D expenditures passed through to subrecipients, $3.8 billion, represents 8.0% of item 1A respondents' total R&D expenditures ($47.6 billion) and 6.9% of all separately budgeted R&D ($54.9 billion) in FY 2009 (see table 39). Doctorate-granting institutions reported a slightly higher percentage of pass-through funds in item 1A, 8.0% ($3.7 billion), than did nondoctorate-granting institutions, 6.5% ($56.9 million). Item 1A respondents from private institutions reported a higher percentage of pass-through funds than did those from public institutions, 9.3% ($1.6 billion) compared with 7.2% ($2.2 billion).
Academic respondents to item 1A reported $3.2 billion in federally financed R&D expenditures passed through to subrecipients (see table 39). This amount represents 11.3% of the federal support reported by item 1A respondents ($28.7 billion) and 9.9% of the $32.6 billion in federal support to all respondents.
Item 1B Analysis
Item 1B, R&D received by the institution as a subrecipient, was completed by 96.3% of the respondents from academic institutions (table A-11). The total R&D expenditures received as subrecipients, $4.1 billion, represents 8.5% of item 1B respondents' total R&D expenditures ($48.5 billion) and 7.5% of all separately budgeted R&D ($54.9 billion) in FY 2009 (see table 40). Doctorate-granting institutions reported a lower percentage of funds received as subrecipients in item 1B, 8.5% ($4.0 billion), than did nondoctorate-granting institutions, 11.2% ($95.0 million). Item 1B respondents from public institutions reported a similar percentage of funds received as subrecipients (8.6% or $2.7 billion) compared to those from private institutions (8.4% or $1.5 billion).
Academic respondents to item 1B reported $3.7 billion in federally financed R&D expenditures received as subrecipients (see table 40). This amount represents 12.7% of the federal support reported by item 1B respondents ($29.2 billion) and 11.4% of the $32.6 billion in federal support to all respondents.
Item 2A Analysis
Item 2A, R&D in non-S&E fields, was completed by 97.4% of the respondents from academic institutions (table A-12). The total for non-S&E R&D expenditures, $2.4 billion, represents 4.2% of item 2A respondents' total S&E and non-S&E R&D expenditures ($56.5 billion) in FY 2009 (see table 77). Nondoctorate-granting institutions reported a higher percentage of non-S&E R&D expenditures in item 2A, 21.3% ($236.6 million), than did doctorate-granting institutions, 3.9% ($2.1 billion). Item 2A respondents from public institutions reported a higher percentage of non-S&E R&D expenditures, 4.7% ($1.8 billion), than did those from private institutions, 3.2% ($563.1 million).
Academic respondents to item 2A reported $867.2 million in federal non-S&E R&D. This amount represents 2.6% of the federal S&E and non-S&E support reported by item 2A respondents ($32.9 billion).
Tables 78–80 show the total and federal non-S&E R&D expenditures for all item 2A respondents.
Item 2B Analysis
Item 2B, federal sources of R&D by S&E field, was completed by 98.7% of the respondents from academic institutions. Total federally financed R&D expenditures reported by item 2B respondents, $32.5 billion, represented 99.8% of all reported federally financed R&D expenditures ($32.6 billion) in FY 2009 (table A-13).
Academic respondents to item 2B reported the highest percentage of their federally financed R&D expenditures from the Department of Health and Human Services, 55.6%, or $18.1 billion (see table 64). NSF accounted for 12.2% ($3.9 billion) of item 2B respondents' R&D from federal sources; the Department of Defense, 10.4% ($3.4 billion); the Department of Energy, 3.8% ($1.2 billion); the National Aeronautics and Space Administration, 3.4% ($1.1 billion); and the U.S. Department of Agriculture, 2.8% ($900.6 million).
Tables 64–73 show federally financed R&D expenditures for item 2B respondents.
Item 4 Analysis
Item 4 (R&D projects awarded to the institution) was completed by 97.6% of the respondents from academic institutions (table A-14). Academic respondents to item 4 reported that the highest percentage of their total R&D project awards (205,536 awards) were from federal sources, 50.6% (104,037 awards), with 1.7% (3,492 awards) from ARRA and 48.9% (100,545 awards) from other federal sources (table A-15). Awards from nonfederal sources accounted for 49.4% (101,499 awards) of item 4 respondents' total project awards in R&D. The majority of these awards went to doctorate-granting institutions (193,442 awards) over nondoctorate-granting institutions (12,094 awards) and to public institutions (162,357 awards) over private institutions (43,179 awards).
NSF discovered as part of the pilot survey follow-up that this question resulted in a major inconsistency across institutions when reporting total award amounts. Although the question had instructions to include the total awarded amount even if it crossed multiple years, many institutions were not able to track awards in this manner. This resulted in a mix of reporting patterns. For example, some institutions reported multiple years of promised incremental funding at the beginning of the award, whereas others only reported the funds released to them in that fiscal year. Because of this inconsistency, those data are not published.
Within the University of California (UC) system, UC campuses manage nearly all R&D activities; however, some R&D expenditures are attributable to the UC Office of the President (UCOP) rather than to a particular campus. The reporting of UCOP R&D expenditures has changed over time. In FY 2004 R&D expenditures began being reported separately for UCOP. In FY 2004, when UCOP was reported separately from UC Los Angeles (UCLA), total R&D at UCLA decreased by $77 million, whereas UCOP showed expenditures of $90 million. Thus, some of the trend changes shown in tables 27–30 for UCLA and UCOP can be attributed to changes in the location of reporting UCOP R&D expenditures.
The Scripps Research Institute (TSRI) in La Jolla, California, was added to the FY 2005 survey population after being mistakenly omitted for several years. During the FY 2005 data collection TSRI reported R&D expenditures for FY 2005 and FY 2004.
Due to the devastation caused by Hurricane Katrina and the subsequent staff turnovers, records damage, and shortage of resources, there was a higher rate of nonresponse in FY 2005 for universities and colleges in New Orleans, Louisiana. Data were imputed for institutions that were unable to commit the resources necessary to complete the FY 2005 survey. Institutions also reported reduced R&D expenditures in FY 2006 as a result of facility loss and faculty turnovers.
Analysts should be cautious when interpreting the FY 2009 data of institutions participating in the pilot test of the HERD Survey (table A-3). Although much of the data requested as part of the HERD Survey were the same as those requested on the Academic R&D Expenditures Survey, revisions to definitions produced sizable trend changes in some cases. For example, expenditures for clinical trials and research training grants were explicitly included on the HERD Survey but not the Academic R&D Expenditures Survey. For more information about the HERD Survey pilot study, contact the NSF project manager.
Data from this and other NSF academic S&E surveys (Survey of Graduate Students and Postdoctorates in Science and Engineering; Survey of Federal Science and Engineering Support to Universities, Colleges, and Nonprofit Institutions; and the Survey of Earned Doctorates) are contained in academic institutional profiles, available at http://www.nsf.gov/statistics/profiles/. The profiles are linked to the corresponding ranking table for each survey.
Data from these and other surveys are also available at http://www.nsf.gov/statistics/ and through the WebCASPAR database system at http://ncsesdata.nsf.gov/webcaspar/. Data are available at the institution, state, and national levels and as longitudinal data. WebCASPAR also includes data from the National Center for Education Statistics surveys of universities and colleges. These data include information on earned degrees, opening fall enrollment, tuition, faculty salaries, tenure and fringe benefits, and financial statistics.