Key Survey Information
Data Collection and Processing
Survey Quality Measures
Data Availability and Comparability
1. Survey Overview (FY 2014 survey cycle)
- Purpose: The Federally Funded Research and Development Centers (FFRDC) R&D Survey is the primary source of information on separately accounted for R&D expenditures at FFRDCs in the United States. The survey is conducted by National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation (NSF).
- Data collection authority: The information is solicited under the authority of the National Science Foundation Act of 1950, as amended, and the America COMPETES Reauthorization Act of 2010. The Office of Management and Budget control number is 3145-0100 and expires on 30 September 2016.
- Major changes to recent survey cycle: None.
2. Key Survey Information
- Frequency: Annual.
- Initial survey year: 2001.
- Reference period: FY 2014.
- Response unit: Establishment.
- Sample or census: Census.
- Population size: 41.
- Sample size: The survey is a census of all known eligible FFRDCs.
- Key variables: Key variables of interest are listed below.
- R&D expenditures by source of funds (federal, state and local, business, nonprofit or other)
- R&D expenditures by character of work (basic research, applied research, or development)
- R&D expenditures funded by the American Recovery and Reinvestment Act of 2009 (ARRA)
- R&D expenditures by type of costs (salaries, software, equipment, subcontracts, other direct costs, and indirect costs)
- Total operating budget
3. Survey Design
- Target population: All FFRDCs.
- Sample frame: The survey is a census of all known eligible FFRDCs, which are identified through the NSF Master Government List of FFRDCs. NSF is responsible for maintaining this list and queries all federal agencies annually to determine any changes to, additions to, or deletions from the list.
- Sample design: The survey is a census of all known eligible FFRDCs.
4. Data Collection and Data Processing
- Data collection: The FY 2014 survey was administered by ICF under contract to NCSES. Surveys were distributed to designated contacts at each institution. The data collection period was from November 2014 through May 2015. Respondents could choose to submit an Adobe PDF questionnaire downloaded from the Web or use a Web-based data collection system. For both methods, telephone and e-mail follow-up was used.
- Data processing: Respondents were contacted and asked to resolve possible self-reporting issues themselves. Questionnaires were carefully examined by survey staff upon receipt. Reviews focused on unexplained missing data and explanations provided for changes in reporting patterns. If additional explanations or data revisions were needed, respondents were sent personalized e-mail messages asking them to provide any necessary revisions before the final processing and tabulation of data.
- Estimation techniques: Missing data are rarely imputed or estimated for the FFRDC R&D Survey. For the FY 2014 survey, the Center for Naval Analyses (CNA) was unable to report expenditures by character of work (Question 3). All CNA expenditures, $80.3 million, were imputed as applied research, which was 1.2% of all FFRDC expenditures for applied research.
5. Survey Quality Measures
- Sampling error: Not applicable.
- Coverage error: Because of the limited size of the FFRDC universe, there is little to no coverage error for this survey. FFRDCs are identified through the NSF Master Government List of FFRDCs. NSF is responsible for maintaining the master list and queries all federal agencies annually to determine changes to, additions to, or deletions from the list.
- Nonresponse error: Most FFRDCs have incorporated the data needed to complete most of the survey questions into their record-keeping systems. Many FFRDCs choose not to complete Question 4 of the survey, which asks for expenditures by type of cost. These FFRDCs are managed by private companies for whom salary information is considered proprietary. Thirteen FFRDCs did not respond to Question 4. Four other FFRDCs could not provide software expenditures, and four could not provide equipment expenditures. One FFRDC did not report its operating budget (Question 5).
6. Measurement error
NSF discovered during the FY 2011 survey cycle that seven FFRDCs were including capital project expenditures in the R&D totals reported on the survey. Corrections made for the FY 2011 survey cycle lowered total expenditures by $468 million. However, the data from previous years still include an unknown amount of capital expenditures in the total. The amount is estimated to be less than $500 million per year.
Prior to the FY 2011 survey, the five FFRDCs administered by the MITRE Corporation had reported only internally funded R&D expenditures. After discussions with NSF, these five FFRDCs agreed to report all FY 2011 operating expenditures for R&D and to revise their data for FYs 2008–10.
NSF discovered during the FY 2013 survey cycle that Los Alamos National Laboratory (LANL) was reporting some expenditures that were not for R&D as defined by this survey. Corrections made for the FY 2013 survey cycle lowered the laboratory’s total expenditures by $349 million. LANL was also incorrectly reporting that all expenditures were for basic research. In corrections made for FY 2013, LANL reported that $1,554 million (91%) of its total research expenditures were for applied research. LANL data from previous years still include an unknown amount of expenditures that were not for R&D and categorize all expenditures as basic research.
Prior to FY 2014, the Aerospace FFRDC reported only expenditures on internal R&D projects. After discussions with NSF, the Aerospace Corporation agreed to report all R&D expenditures for FY 2014 and provide revised data to include all R&D expenditures for FYs 2010–13. R&D expenditures increased by more than $800 million dollars each year.
During the FY 2014 survey, NSF discovered that the National Optical Astronomy Observatory had been including data for the National Solar Observatory since FY 2010. The Association of Universities for Research in Astronomy, the administrator of both FFRDCs, provided revised data for both FFRDCs for FYs 2010–13.
7. Data Availability and Comparability
- Data availability: Annual data are available for all FFRDCs for FYs 2001–2014.
- Data comparability: When the review for consistency between each year’s data and submissions in prior years reveals discrepancies, it is sometimes necessary to modify prior years’ data. For accurate historical data, use only the most recently released data tables. Individuals wishing to analyze trends other than those published in the most recent publication are encouraged to contact the Project Officer for more information about comparability of data over time.
8. Data Products
- Publications: The data from this survey are published annually in data tables in the series R&D Expenditures at Federally Funded R&D Centers. The most recent report in this series is available at http://www.nsf.gov/statistics/ffrdc/. Information from this survey is also included in Science and Engineering Indicators.
- Electronic access: Public use files are available at http://www.nsf.gov/statistics/ffrdc/pub_data.cfm.
9. Contact Information
For additional information about this survey, or the methodology report, contact the Project Officer:
Research and Development Statistics Program
National Center for Science and Engineering Statistics
National Science Foundation
4201 Wilson Boulevard, Suite 965
Arlington, VA 22230
Phone: (703) 292-7765