Key Survey Information
Data Collection and Processing
Survey Quality Measures
Data Availability and Comparability
1. Survey Overview (FYs 2013–15 survey cycle; volume 63)
- Purpose: The annual Survey of Federal Funds for Research and Development (Federal Funds Survey) is the primary source of information about federal funding for R&D in the United States. The results of the survey are also used for budget purposes for four federal programs: the Federal Laboratory Consortium for Technology Transfer, Small Business Innovation Research, Small Business Technology Transfer, and Experimental Program to Stimulate Competitive Research.
- Data collection authority: The information is solicited under the authority of the National Science Foundation Act of 1950, as amended.
- Major changes to recent survey cycle: None.
2. Key Survey Information
- Frequency: Annual.
- Initial survey year: 1951.
- Reference period: FYs 2013-15.
- Response unit: Establishment.
- Sample or census: Census.
- Population size: Not applicable.
- Sample size: 15 federal departments, their 54 subagencies, and the 13 independent agencies.
- Key variables: Key variables of interest are listed below.
- The survey provides data on federal obligations by the following key variables:
- Character of work
- Basic research
- Applied research
- Federal agency
- Federally funded research and development centers (FFRDCs)
- Field of science and engineering
- Geographic location (within the United States and by foreign country or economy)
- Performer (type of organization doing the work)
- R&D plant
The survey provides data on federal outlays by the following key variables:
Note that the variables "R&D," "character of work," and "R&D plant" in this survey use definitions comparable to those used by the Office of Management and Budget. The variables "geographic location" and "field of science and engineering" are comparable in the Federal Funds Survey and the related National Science Foundation (NSF) Survey of Federal Science and Engineering Support to Universities, Colleges, and Nonprofit Institutions.
3. Survey Design
- Target population: The population consists of the federal agencies that conduct R&D programs. In the survey cycle for data collection on FYs 2013–15, a total of 28 federal agencies (15 federal departments and 13 independent agencies) reported R&D data. Because multiple subdivisions of a federal department were in some cases requested to complete the survey, there were 72 individual respondents (5 federal departments, 54 department subdivisions, and 13 independent agencies).
- Sample frame: The sample frame is obtained from information in the president's budget submitted to Congress. The Analytical Perspectives volume and the "Detailed Budget Estimates by Agency" section of the appendix to the president's budget identify agencies that receive research and development funding. In addition, the Federal Yellow Book and the U.S. Government Manual are consulted.
- Sample design: Five of the 15 federal departments were surveyed at the department level, and 10 were surveyed at the subdivision level, with 54 subdivisions surveyed. All 13 independent agencies were surveyed at the agency level.
4. Data Collection and Processing
- Data collection: Synectics for Management Decisions, Inc. (Synectics) performed the data collection for volume 63 (FYs 2013–15) under contract with NSF's National Center for Science and Engineering Statistics (NCSES). Agencies were initially contacted by phone to verify the name, e-mail address, and phone number of each survey respondent. A Web-based data collection system was used to collect data for the Federal Funds Survey.
Federal funds data, as collected, span 3 government fiscal years: the fiscal year just completed, the current fiscal year, and the next fiscal year. Actual data are collected for the year just completed; estimates are obtained for the current and next fiscal year.
For volume 63, actual data (representing completed transactions) were collected for FY 2013 and estimated data were collected for FYs 2014 and 2015. Estimated data do not represent final actions. Data collection for the Federal Funds Survey began in February 2014 and continued until August 2014. The amounts reported for FY 2014 reflect congressional appropriation actions as of that period, as well as apportionment and reprogramming decisions as of that time. Data for FY 2015 represent as-yet unimplemented administration budget proposals. Authorization, appropriation, deferral, and apportionment actions completed after these data were collected will be reflected in later surveys in this series.
- Data processing: A Web-based data collection system is used to collect and manage data for the Federal Funds Survey. This Web-based system was designed to help improve survey reporting and reduce data collection and processing costs by offering respondents direct online reporting and editing.
All data collection efforts, data imports, data editing, and trend checking are accomplished using the Web-based data collection system. The Web-based data collection system has a component that allows survey respondents to enter their data online; it also has a component that allows the contractor to monitor support requests, data entry, and data issues.
- Estimation techniques: There is no known item nonresponse, so no weighting or imputation methods are used.
5. Survey Quality Measures
- Sampling error: Because all eligible agencies are included, there is no sampling error.
- Coverage error: Because identifying relevant federal agencies is a straightforward task, coverage is assumed to be complete except for activities of the Central Intelligence Agency.
- Nonresponse error: Item nonresponse. Agencies are encouraged to estimate informally when actual data are unavailable. The survey instrument allows respondents to enter data or skip data fields. NSF assumes a blank field is zero for estimation purposes. If a significant number of blank fields are incorrectly skipped by respondents, bias could result.
- Measurement error: Some measurement problems are known to exist in the data. More specifically, some agencies are not able to report the full costs of R&D. For example, the Department of Defense (DOD) does not include headquarters costs of planning and administering R&D programs, which are estimated at a fraction of 1% of its total cost; DOD has stated that identification of amounts at this level is impracticable.
The National Institutes of Health (NIH) in the Department of Health and Human Services currently has many of its awards in its financial system without any field of science code. Therefore, NIH uses an alternate source to estimate its research dollars by field of science. NIH uses scientific class codes (based upon past history of grant, content of the title, and the name of the awarding institute or center) as an approximation for field of science codes.
The National Aeronautics and Space Administration (NASA) does not include any field of science codes in its financial database, and consequently, NASA must estimate what percentage of the agency's research dollars are allocated into the fields of science.
Also, agencies are required to report the ultimate performer of R&D. However, through past workshops, NSF has learned that some agencies do not always track their R&D dollars to the ultimate performer of R&D. This leads to some degree of misclassification of performers of R&D, but NSF has not determined the extent of the errors in performer misclassification by the reporting agencies.
R&D plant data are underreported to some extent because of the difficulty some agencies, particularly DOD and NASA, encounter in identifying and reporting these data. DOD's respondents report obligations for R&D plant funded under the agency's appropriation for construction, but they are able to identify only a small portion of the R&D plant support that is within R&D contracts funded from DOD's appropriation for research, development, testing, and evaluation. Similarly, NASA respondents cannot separately identify the portions of industrial R&D contracts that apply to R&D plant, since these data are subsumed in the R&D data covering industrial performance. NASA R&D plant data for other performing sectors are reported separately.
6. Data Availability and Comparability
- Data availability: Annual data are available for FYs 1951–2015.
- Data comparability: The information included in this survey has been stable since FY 1973, when federal obligations for research to universities and colleges by agency and detailed science and engineering field were added to the survey. Many of the other variables are available from the early 1950s on. However, analysts studying trends are encouraged to obtain up-to-date data from the NCSES website, because agencies reclassify their responses for prior years as additional budget data become available.
7. Data Products
- Publications: NCSES publishes data from this survey annually in detailed statistical tables in the series Federal Funds for Research and Development and the Science and Engineering State Profiles series. Historical information is also available from NSF's Integrated Science and Engineering Resources Data System (WebCASPAR).
8. Contact Information
For additional information about this survey, or the methodology report, contact the Project Officer listed below:
Research and Development Statistics Program
National Center for Science and Engineering Statistics
National Science Foundation
4201 Wilson Boulevard, Suite 965
Arlington, VA 22230
Phone: (703) 292-7815