Key Survey Information
Data Collection and Processing
Survey Quality Measures
Data Availability and Comparability
1. Survey Overview (FYs 2014–16 survey cycle; volume 64)
- Purpose: The annual Survey of Federal Funds for Research and Development (Federal Funds Survey) is the primary source of information about federal funding for R&D in the United States. The results of the survey are also used for budget purposes for four federal programs: the Federal Laboratory Consortium for Technology Transfer, Small Business Innovation Research, Small Business Technology Transfer, and Experimental Program to Stimulate Competitive Research. The survey is sponsored by the National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation (NSF).
- Data collection authority: The information is solicited under the authority of the National Science Foundation Act of 1950, as amended, and the America COMPETES Reauthorization Act of 2010.
- Major changes to recent survey cycle: None.
2. Key Survey Information
- Frequency: Annual.
- Initial survey year: 1951.
- Reference period: FYs 2014–16.
- Response unit: Federal agencies.
- Sample or census: Census.
- Population size: 28 federal agencies (5 federal departments surveyed at the agency level; 10 federal departments surveyed at the subdivision level, with a total 54 subdivisions; and 13 independent agencies).
- Sample size: Not applicable; the survey is a census of all federal agencies that conduct R&D programs, except the Central Intelligence Agency.
- Key variables: Key variables of interest are listed below.
- The survey provides data on federal obligations by the following key variables:
- Type of R&D
- Basic research
- Applied research
- Federal agency
- Federally funded research and development centers (FFRDCs)
- Field of science and engineering
- Geographic location (within the United States and by foreign country or economy)
- Performer (type of organization doing the work)
- R&D plant
The survey provides data on federal outlays by the following key variables:
Note that the variables "R&D," "type of R&D," and "R&D plant" in this survey use definitions comparable to those used by the Office of Management and Budget. The variables "geographic location" and "field of science and engineering" used in this survey are comparable with those in the related Survey of Federal Science and Engineering Support to Universities, Colleges, and Nonprofit Institutions.
3. Survey Design
- Target population: The population consists of the federal agencies that conduct R&D programs. In the survey cycle for data collection on FYs 2014–16, a total of 28 federal agencies (15 federal departments and 13 independent agencies) reported R&D data. Because multiple subdivisions of a federal department were in some cases requested to complete the survey, there were 72 individual respondents. (Five of the 15 federal departments were surveyed at the department level, and 10 were surveyed at the subdivision level, with 54 subdivisions surveyed. All 13 independent agencies were surveyed at the agency level.)
- Sample frame: The survey is a census of all federal agencies that conduct R&D programs, which are identified from information in the president’s budget submitted to Congress. The CIA is excluded. The Analytical Perspectives volume and the “Detailed Budget Estimates by Agency” section of the appendix to the president’s budget identify agencies that receive research and development funding. In addition, the Federal Yellow Book and the U.S. Government Manual are consulted.
- Sample design: Not applicable.
4. Data Collection and Processing
- Data collection: Synectics for Management Decisions, Inc. (Synectics) performed the data collection for volume 64 (FYs 2014–16) under contract with NCSES. Agencies were initially contacted by e-mail to verify the contact information of each survey respondent. A Web-based data collection system is used to collect data for the Federal Funds Survey.
Federal funds data, as collected, span 3 government fiscal years: the fiscal year just completed, the current fiscal year, and the next fiscal year. Actual data are collected for the year just completed; estimates are obtained for the current and next fiscal year.
For volume 64, actual data (representing completed transactions) were collected for FY 2014 and estimated data were collected for FYs 2015 and 2016. Estimated data do not represent final actions. Data collection for the Federal Funds Survey began in February 2015 and continued until August 2015. The amounts reported for FY 2015 reflect congressional appropriation actions, as well as apportionment and reprogramming decisions, as of that time. Data for FY 2016 represent as-yet unimplemented administration budget proposals. Authorization, appropriation, deferral, and apportionment actions completed after data collection concluded will be reflected in later surveys in this series.
- Data processing: A Web-based data collection system is used to collect and manage data for the Federal Funds Survey. This Web-based system was designed to help improve survey reporting and reduce data collection and processing costs by offering respondents direct online reporting and editing.
All data collection efforts, data imports, data editing, and trend checking are accomplished using the Web-based data collection system. The Web-based data collection system has a component that allows survey respondents to enter their data online; it also has a component that allows the contractor to monitor support requests, data entry, and data issues.
- Estimation techniques: There is no known unit or item nonresponse, so no weighting or imputation methods are used; NCSES assumes a blank field is zero for estimation purposes.
5. Survey Quality Measures
- Sampling error: Because all eligible agencies are included, there is no sampling error.
- Coverage error: Because identifying relevant federal agencies is a straightforward task, coverage is assumed to be complete except for activities of the Central Intelligence Agency.
- Nonresponse error: Agencies are encouraged to estimate informally when actual data are unavailable. The survey instrument allows respondents to enter data or skip data fields. NCSES assumes a blank field is zero for estimation purposes. If a significant number of blank fields are incorrectly skipped by respondents, bias could result.
- Measurement error: Some measurement problems are known to exist in the data. More specifically, some agencies are not able to report the full costs of R&D.
For example, the Department of Defense (DOD) does not include headquarters costs of planning and administering R&D programs, which are estimated at a fraction of 1% of its total cost; DOD has stated that identification of amounts at this level is impracticable.
The National Institutes of Health (NIH) in the Department of Health and Human Services currently has many of its awards in its financial system without any field of science code. Therefore, NIH uses an alternate source to estimate its research dollars by field of science. NIH uses scientific class codes (based upon past history of grant, content of the title, and the name of the awarding institute or center) as an approximation for field of science codes.
The National Aeronautics and Space Administration (NASA) does not include any field of science codes in its financial database. Consequently, NASA must estimate what percentage of the agency’s research dollars are allocated into the fields of science.
Also, agencies are required to report the ultimate performer of R&D. However, through past workshops, NCSES has learned that some agencies do not always track their R&D dollars to the ultimate performer of R&D. This leads to some degree of misclassification of performers of R&D, but NCSES has not determined the extent of the errors in performer misclassification by the reporting agencies.
R&D plant data are underreported to some extent because of the difficulty some agencies, particularly DOD and NASA, encounter in identifying and reporting these data. DOD’s respondents report obligations for R&D plant funded under the agency’s appropriation for construction, but they are able to identify only a small portion of the R&D plant support that is within R&D contracts funded from DOD’s appropriation for research, development, testing, and evaluation. Similarly, NASA respondents cannot separately identify the portions of industrial R&D contracts that apply to R&D plant, since these data are subsumed in the R&D data covering industrial performance. NASA R&D plant data for other performing sectors are reported separately.
6. Data Availability and Comparability
- Data availability: Annual data are available for FYs 1951–2016.
- Data comparability: The information included in this survey has been stable since FY 1973, when federal obligations for research to universities and colleges by agency and detailed science and engineering field were added to the survey. Many of the other variables are available from the early 1950s on. However, analysts studying trends are encouraged to obtain up-to-date data from the NCSES Web site, because agencies reclassify their responses for prior years as additional budget data become available.
7. Data Products
- Publications: NCSES publishes data from this survey annually in the detailed tabular data series Federal Funds for Research and Development and the Science and Engineering State Profiles series. Historical information is also available from NSF's Integrated Science and Engineering Resources Data System (WebCASPAR).
8. Contact Information
For additional information about this survey, or the methodology report, contact the Project Officer:
Research and Development Statistics Program
National Center for Science and Engineering Statistics
National Science Foundation
4201 Wilson Boulevard, Suite 965
Arlington, VA 22230
Phone: (703) 292-7815