Contents

---

General Notes

Data Tables

Appendix A. Technical Notes

Appendix B. Survey Instrument

Suggested Citation, Acknowledgments

 

Appendix A. Technical Notes

The National Science Foundation (NSF) Higher Education Research and Development (HERD) Survey is the primary source of information on separately budgeted research and development expenditures within higher education institutions in the United States and outlying areas. In 2010, the HERD survey replaced a previous annual collection, the Survey of Research and Development Expenditures at Universities and Colleges (Academic R&D Expenditures Survey), which was conducted from FY 1972 through FY 2009. The revised survey continues to collect information on R&D expenditures by field of research and source of funds, but additional questions about funding sources, types of research and expenses, and R&D personnel have been included in the new survey. Survey questions now request data across all academic disciplines, not just science and engineering (S&E) disciplines. Unless otherwise noted, expenditures analyzed in this report refer to R&D activities across all fields of R&D. The results of the survey are primarily used to assess trends in R&D expenditures across academic field. This information is vital for federal, state, and academic planners who make decisions on future R&D funding priorities.

Scope of the Survey

The FY 2011 HERD Survey was sent to 954 institutions of higher education in the United States, Guam, Puerto Rico, and the U.S. Virgin Islands. These institutions granted a bachelor's degree or higher, expended at least $150,000 in separately budgeted R&D in FY 2011, and were geographically separate campuses headed by a president or chancellor.

The survey population was reviewed before data collection began to ensure that each institutional classification was accurate. Characteristics of the schools were reviewed before and during the course of the survey to determine whether changes had occurred (e.g., name; highest degree granted; school openings, closings, or mergers). Table A-1 shows all institution name changes between the FY 2010 and FY 2011 surveys. An additional category for highest degree granted (HDG) was included on the FY 2011 tables. In addition to bachelor's, master's, and doctorate, an HDG category labeled "professional" was added (see Table A-4). Institutions with a professional HDG do not grant a research-focused doctorate degree (PhD), but do grant a professional doctorate—e.g., Doctor of Osteopathic Medicine (DO), Doctor of Dental Surgery (DDS). Institutions in this category would have previously been classified as bachelor's or master's based on the highest non-professional degree granted.

After data collection closed, institutions were reviewed to verify that only those reporting at least $150,000 in separately budgeted R&D were included in the population. Of the 954 institutions surveyed, 42 reported total R&D expenditures of less than $150,000. These 42 institutions were excluded from the population, and their data are not included in the FY 2011 detailed statistical tables. The total and federally financed R&D expenditures for these 42 institutions are listed in table A-2.

 Back to Top

FY 2011 Survey Frame Design

The HERD Survey is a census of the full population of eligible academic institutions. The population of the Academic R&D Expenditures Survey, the predecessor to the HERD Survey, included all institutions that granted a bachelor's degree or higher in S&E fields and had at least $150,000 in separately budgeted R&D expenditures during the targeted fiscal year. This had been the population criteria since the FY 2004 collection. Beginning in FY 2010, the first year the HERD Survey was used, the survey population was revised to include institutions that granted a bachelor's degree or higher in any field of study. It was further required that within a university system each campus headed by a campus-level president or chancellor complete a separate survey rather than combining their response with other campuses in the university system. Most university systems and member campuses were identified and asked to report separately during the FY 2010 collection; however, additional systems were split into separate campuses during the FY 2011 collection. Table A-1 shows the separate campuses that were identified for two institutions that previously reported in the aggregate. For data users wanting to compare HERD data across university systems by aggregating member campuses, Table A-3 shows all institutions in the FY 2011 population by state, institutional control, and system.

Each year the NSF contacts institutions that meet the degree-granting criterion but were not in the population of the previous survey to determine whether they meet the $150,000 expenditure criterion. Institutions with a minimum of $150,000 are added to or retained in the survey population. The overall number of academic institutions in the population increased from 744 in FY 2010 to 912 in FY 2011 (Table A-4). The institutions added to the population were identified for inclusion in the population through two sources—the FY 2010 HERD Short Form and the FY 2011 population review.

During the FY 2010 collection cycle, NSF conducted the FY 2010 HERD Short Form Survey. This survey was an abbreviated version of the FY 2010 HERD Survey and was conducted simultaneously with the FY 2010 HERD data collection. The survey was administered to all academic institutions in U.S. states, territories, and outlying areas that granted a 4-year degree or higher and were not already included in the HERD Survey population. Institutions that reported a minimum of $150,000 on the Short Form Survey were included in the FY 2011 HERD population. Institutions were also identified during a population review. Prior to the FY 2011 collection, the institutions that did not respond to the FY 2010 Short Form and institutions that reported more than $0 but less than $150,000 on the Short Form were contacted to determine whether they meet the $150,000 expenditure criterion for FY 2011.

 Back to Top

Survey Instrument

The HERD Survey continues to collect much of the same data requested as part of the Academic R&D Expenditures Survey; however, several additional questions have been included in the survey, and some of the definitions applicable to the previous survey have been revised. The following section describes the content of the FY 2011 HERD Survey instrument while highlighting how the current survey differs from the Academic R&D Expenditures Survey.

General Changes

The survey still asks for current fund expenditures for separately budged R&D, but total R&D expenditures now includes both S&E fields and non-S&E fields such as humanities, education, law, and the arts. Although there was one item in the previous survey concerning non-S&E expenditures, most items requested information only on S&E expenditures. All HERD Survey questions ask about R&D across all fields. Additionally, clinical trials and research training grants are now explicitly included in the definition of R&D.

Analysts should be cautious when examining trend data from the Academic R&D Expenditures Survey and HERD. Although many variables are similar across the two surveys, because of changes to the definition of R&D and the inclusion of non-S&E expenditures, exact comparisons may be misleading.

HERD Questions

Question 1 is a request for R&D expenditures by source of funds. This question is similar to Item 1 from the Academic R&D Expenditures Survey, but there are some important differences. In addition to the inclusion of non-S&E expenditures, a separate category has been created for nonprofit organizations. Prior to FY 2010 these expenditures were reported under the category for all other sources. The "Industry" category has been renamed "Business."

Care should be taken when interpreting data on source of funds. Business R&D support is limited to grants and contracts for R&D activities from profit-making organizations. Total business funds exclude research funded through unrestricted accounts and from corporate foundations, endowments, and fellowships to students; those funds would be included in an institution's own funding totals.

Question 2 is a request for the amount of R&D expenditures funded by foreign sources.

Question 3 requests externally funded R&D expenditures by type of agreement (i.e., contract vs. grant).

Question 4 is a request for the amount of R&D expenditures for projects in the institution's medical school, if applicable. For the purposes of this survey, a medical school was defined as a school granting an MD or DO degree.

Question 5 asks for federally and nonfederally financed R&D expenditures for Phase I, Phase II, and Phase III clinical trials.

Question 6 asks respondents to characterize federally and nonfederally financed R&D expenditures by character of work: basic research, applied research, or development. In the Academic R&D Expenditures Survey, institutions were asked to report the percentage of R&D expenditures that would be considered basic research. HERD asks for amounts rather than percentages and was expanded to separately cover applied research and development.

Question 7 is a request for federally and nonfederally financed expenditures for R&D from funds received by the institution as a subrecipient from higher education institutions, businesses, nonprofit organizations, and other sources. This question was expanded from the FY 2010 version, which asked only for expenditures from higher education and other sources.

Question 8 is a request for federally and nonfederally financed funds for R&D passed through the institution to a higher education institution, business, nonprofit organization, or other subrecipient. This question was expanded from the FY 2010 version, which only asked for pass-through expenditures to higher education and other sources.

Question 9 is a request for federally financed expenditures for R&D by field and federal agency. Again, this is similar to items from the Academic R&D Expenditures Survey, but non-S&E expenditures are included. Although institutions did report federally financed R&D expenditures in non-S&E fields in the past, they were not asked to report expenditures by agency. When analyzing these data at the detailed discipline level, users should keep in mind that there is considerable interdisciplinary and multidisciplinary activity.

Question 10 asks for the agency name and corresponding expenditure amount for all federal agencies and subagencies included under "Other" federal agency on question 9.

Question 11 requests the amount for federally financed R&D expenditures funded by the American Recovery and Reinvestment Act (ARRA).

Question 12 is a request for nonfederally financed expenditures for separately budgeted R&D activities by detailed R&D field and source. In past years institutions have reported nonfederal R&D expenditures by field, but prior to FY 2010, they were not asked to report those expenditures by detailed source (e.g., nonprofit organizations, institutional funds). As with Question 9, when interpreting these data at the detailed discipline level, users should keep in mind that there is considerable interdisciplinary and multidisciplinary activity.

Question 13 is a request for total expenditures for R&D by type of cost. This question includes data for salaries, equipment, other direct costs, and indirect costs.

Question 14 is a request for the capitalization thresholds for software and equipment.

Question 15 is a request for the amount of federally and nonfederally financed expenditures that were used for the purchase of research equipment. These data are of special interest to federal and institutional policymakers in determining current funding levels for scientific research instrumentation.

Question 16 asks for a headcount of principal investigators and other personnel that were paid from R&D salaries reported in question 13.

Question 17 is a request for the number of personnel reported in Question 16 that were categorized as postdocs. For the purposes of this survey, NSF defines a postdoc as someone who (1) holds a recent doctoral degree, generally awarded within the past 5 years, and (2) has a limited-term appointment, generally no more than 5–7 years.

Question 18 is a request for the respondent's contact information, the institution's fiscal year, and the time it took for the institution to complete the survey.

 Back to Top

Data Collection

The FY 2011 survey questionnaires were sent by e-mail in December 2011. Respondents could choose to submit an Adobe PDF questionnaire downloaded from the Web or use a Web-based data collection system to respond to the survey. Every effort was made to maintain close contact with respondents in order to preserve both the consistency and continuity of the resulting data. Questionnaires were carefully examined for completeness upon receipt. Computerized facsimiles of the survey data were then prepared for each institution; these compared the current and 2 prior years of data and noted any substantive disparities. Respondents were sent personalized e-mail messages asking them to provide any necessary revisions before the final processing and tabulation of data. These e-mail messages included a link to the HERD Survey Web-based data collection system, allowing respondents to view and correct their data online.

Respondents were asked to explain significant differences between current-year reporting and established patterns of reporting verified for prior years. They were encouraged to correct prior-year data if needed. When respondents updated or amended figures from past years, NSF made corresponding changes to trend data in this report and to the underlying microdata.

 Back to Top

Response Rate

By the survey's closing date in June 2012, forms had been received from 875 universities and colleges out of the population of 912, a response rate of 95.9%. Responses were received from 98.2% of all doctorate-granting institutions. The R&D expenditures reported by these doctoral institutions constituted 98.1% of the estimated national R&D expenditures for FY 2011. Table A-5 displays a detailed breakdown of response rates by highest degree granted, and table A-6 displays a breakdown of response rates by survey item for the academic population.

 Back to Top

National Total and Imputation

Missing values from questions that were part of the previous version of the survey—R&D expenditures by source of funds and R&D field, R&D expenditures passed through to subrecipients, R&D expenditures received as a subrecipient, and R&D equipment expenditures—were imputed based on the previous year's data and the reported data of peer institutions in the current cycle. For the 37 institutions that had not responded by the closing date of the survey and had been included in the FY 2010 HERD or Short Form population, R&D expenditures were imputed by applying inflator/deflator factors to the prior years' figures. Imputation factors were ratios derived from the 2-year-trend data of responding institutions with similar characteristics, including controlling for highest degree granted and type of institutional control (public or private). For the 15 institutions that were in the Short Form population in FY 2010, there was only previous year data for values in Question 1 of the survey. For these institutions, the average proportions of the data of peer institutions with similar characteristics was used to direct the allocation of expenditures across Questions 6, 7, 8, 9, 12, and 15. Data for partial nonresponse was imputed using similar techniques.

Questions that were new to the redesigned survey were imputed using a set of core predictors that were highly correlated with most of the survey questions. These include type of institutional control, highest degree granted, and expenditure size. Several models also included the existence of a medical school in the set of core predictors. In most cases, an imputation class was identified based on predictors (e.g., public, doctoral-granting institutions with total R&D expenditures in the upper 25%) and missing values in that class were imputed as the average reported values in that class.

R&D expenditures from ARRA, capitalization thresholds for software and equipment, and headcounts of personnel were not imputed.

The imputed total R&D was $259 million, or 0.4%, of the $65 billion in all separately budgeted R&D expenditures (table A-7). Tables A-7 through A-18 present imputed amounts for each applicable survey variable. The dollar amount imputed is displayed along with the percentage it represents of the national estimate for universities and colleges in a particular field.

A number of surveyed institutions have responded intermittently in past years. For years in which no response was received, data have been imputed as previously described. Although the imputation algorithm accurately reflects national trends, it cannot account for specific trends at individual institutions. For this reason, a separate imputation of institutional data for prior years is also performed, following current-year imputation.

For each institution, formerly imputed values from the Academic R&D Expenditures Survey and the FY 2010 HERD survey were recomputed to ensure that the imputed data accurately represent the growth patterns shown by reported data of the FY 2011 HERD Survey. If, for example, data were reported for FY 2006 and FY 2011 but not for the intervening years, the difference between the reported total for each question was calculated and evenly distributed across the intervening years (2007–09). The new totals were then spread across disciplines or sources of support on the basis of the most recent reporting pattern. These procedures result in much more consistent reporting trends for individual institutions but have little effect on aggregate figures reflecting national totals.

 Back to Top

Question 10 Analysis

The portion of federally financed R&D expenditures financed by federal agencies not listed in Question 9, $3.5 billion, represents 8.8% of all federally financed R&D expenditures. The majority of these expenditures were financed by the Department of Education ($715 million), the Department of Commerce ($526 million), and the Department of Transportation ($344 million) (table A-19).

 Back to Top

Question 11 Analysis

Question 11, federally financed R&D expenditures from ARRA, was completed by 98.2% of respondents (table A-20). The portion of federally financed R&D expenditures received from ARRA funding, $4.2 billion (see table 10), represents 10.2% of all survey respondents' federally funded higher education R&D expenditures ($40.8 billion) in FY 2011. Doctorate-granting institutions reported a higher portion of funds received from ARRA funding (10.3% or $4.1 billion), than did nondoctorate-granting institutions (7.4% or $59 million). Public institutions reported a slightly lower portion of funds received from ARRA funding (10.0% or $2.5 billion) compared to those from private institutions (10.5% or $1.6 billion).

 Back to Top

Question 14 Analysis

Question 14, capitalization thresholds for software and equipment, was completed by 88.9% of respondents (table A-21). Of these respondents, 11 reported that they do not capitalize equipment and 118 reported that they do not capitalize software. The remaining respondents reported a mean software threshold of $66,000 and a mean equipment threshold of $6,000. Doctorate-granting institutions reported a higher capitalization threshold for software ($87,000) and equipment ($7,000) compared to nondoctorate-granting institutions ($40,000 and $6,000, respectively). Public institutions reported higher thresholds for software ($105,000) compared to private institutions ($14,000), but both groups reported thresholds for equipment of $6,000.

 Back to Top

Data Anomalies

As mentioned above, analysts should be cautious when comparing FY 2010 and 2011 data to data from previous years. In prior years, the Academic R&D Expenditures Survey collected expenditures for S&E and non-S&E fields separately. Institutions were not always able to provide non-S&E expenditures, and those data were not previously imputed. Also, revisions to the definition of R&D in 2010 may have influenced reported values to varying degrees depending on the amounts of clinical trials and training grants at a particular institution.

It should also be noted that because institutions are asked to include funds passed through to higher education institutions as well as subrecipient funding from higher education institutions, there is double counting included in national and group totals. For example, Institution A's survey will include the $2 million passed through to Institution B and Institution B's survey will also include that $2 million in subrecipient funding. Overall, institutions reported $2.9 billion in expenditures from subrecipient funding received from other universities and $3.1 billion in funds passed through to higher education subrecipients.

Washington State University (WSU) is a multi-campus university system that continues to report as one unit of the HERD Survey. In addition to the main campus in Pullman, WSU Spokane, WSU Tri-Cities, and WSU Vancouver are separate campuses that grant bachelor's degrees and are headed by chancellors. The institution has been asked to report expenditures separately for each campus but the records needed to comply are not yet available. Representatives of the university have indicated that R&D expenditures are concentrated at the Pullman and Spokane campuses. Since the expenditures reported represent all campuses, WSU's ranking in expenditure tables is higher than would be expected if the campuses reported separately.

 Back to Top

Data Availability

Data from this and other NSF academic surveys (Survey of Graduate Students and Postdoctorates in Science and Engineering; Survey of Federal Science and Engineering Support to Universities, Colleges, and Nonprofit Institutions; and Survey of Earned Doctorates) are contained in academic institutional profiles, available at http://www.nsf.gov/statistics/profiles/. The profiles are linked to the corresponding ranking table for each survey.

Data from these and other surveys are also available at http://www.nsf.gov/statistics/ and through the WebCASPAR database system at http://ncsesdata.nsf.gov/webcaspar/. Longitudinal data are available at the institution, state, and national levels. WebCASPAR also includes data from the National Center for Education Statistics surveys of universities and colleges. These data include information on earned degrees, opening fall enrollment, tuition, faculty salaries, tenure and fringe benefits, and financial statistics.

 Back to Top

Technical Tables