Key Survey Information
Data Collection and Processing
Survey Quality Measures
Data Availability and Comparability
1. Survey Overview (2013 survey cycle)
- Purpose: The Business Research and Development and Innovation Survey (BRDIS) is the primary source of information on R&D expenditures and the R&D employees of for-profit, nonfarm businesses with five or more employees operating in the United States.
- Data collection authority: National Science Foundation Act of 1950, as amended, and the America COMPETES Reauthorization Act of 2010; collected under Office of Management and Budget control number 0607-0912, expiring 31 January 2015.
- Major changes to recent survey cycle: None.
2. Key Survey Information
- Frequency: Annual.
- Initial survey year: 2008. The predecessor to BRDIS, the Survey of Industrial R&D (SIRD), began in 1953.
- Reference period: CY 2012
- Response unit: Companies. Companies with known R&D activity (~5,000) are sent the standard survey (BRDI-1), and all other companies (~40,000) are sent a short survey screener (BRD-1(S)).
- Sample or census: Sample survey of for-profit companies with a U.S. presence and five or more employees engaged in the mining, utilities, construction, manufacturing, wholesale trade, retail trade, or services industries.
- Population size: Approximately 2,000,000 companies.
- Sample size: Approximately 45,000 companies.
- Key variables: Key variables of interest are listed below.
- R&D performance (domestic and foreign R&D for U.S.-based companies)
- Total and R&D employment
- Sources of R&D funding
- Character of R&D work (basic research, applied research, and development)
- Type of R&D cost (e.g., salaries and fringe benefits)
- R&D capital expenditures
- R&D application and technology focus areas
- Business codes
- Geographic location of domestic and foreign R&D performance of U.S.-based companies
- Patenting, licensing, and technology transfer activities
3. Survey Design
- Target population: The target population consists of all for-profit nonfarm companies that have five or more paid employees in the United States and have at least one establishment that is classified in an in-scope sector based on the North American Industry Classification System (NAICS), is in business during the survey year, and is physically located in the United States.
- Sample frame: The Business Register, maintained by the Census Bureau, is the source used to create the sample frame for BRDIS.
- Sample design: BRDIS has a stratified probability sampling design that uses both simple random sampling and probability proportional to size (PPS) sampling within strata. Stratification is based on R&D activity and a NAICS-based industry code. For companies with known R&D activity, PPS sampling is based on R&D performance. For companies with unknown R&D activity, PPS sampling is based on annual payroll. Companies known to perform large amounts of R&D and companies with large amounts of payroll are selected with certainty.
4. Data Collection and Processing
- Data collection: BRDIS uses multimode data collection by paper booklet or Web reporting instruments. Respondents report both by mail (20%) and on the web (80%) about equally.
- Data processing: All data submitted by respondent companies are reviewed to ensure that data fields are complete and that data are internally consistent. Given the size and complexity of the BRDIS, many survey responses include errors that require correction or unusual patterns that require validation. Several hundred automated edit checks are applied to improve the efficiency of analyst data review and correction. Approximately two-thirds of these edit checks are designed to catch arithmetic errors and logically inconsistent responses (balance edits). The remaining edit checks are designed to flag outliers for further analyst review (analytical edits). During editing, if additional information or data corrections are needed, respondents are contacted and are asked to provide additional information or to correct data. If additional information or corrected data cannot be obtained from respondents, data are imputed.
- Estimation techniques: The general methodology used to produce estimates from the BRDIS involves sums of weighted data (reported or imputed), in which the weights are the product of the sampling weight and the nonresponse adjustment factor. However, there are some exceptions, which are described in detail in the annual detailed statistical tables reports.
5. Survey Quality Measures
- Sampling error: Estimates based on the total sample have relatively small sampling errors. Estimates of sampling errors associated with each cell in the detailed statistical tables are available by request.
- Coverage error: Coverage error is minimal because the Business Register, the source for BRDIS, is continually updated and coverage is comprehensive of all domestic businesses.
- Nonresponse error: Except for estimates of counts, patents, patent licensing agreements, product or process innovation, and intellectual property protection, unit nonresponse is done by adjusting weighted reported and imputed data by multiplying each company's sampling weight by a nonresponse adjustment factor. For estimates of counts, patents, patent licensing agreements, and product or process innovation, other adjustments for nonresponse are made. For count estimates for the BRDIS checkbox items that involve intellectual property protection, both unit and item nonresponse are handled using a nonresponse weight adjustment based on R&D groups, industry sampling strata, and the presence or absence of R&D activity. Detailed descriptions of the adjustments for nonresponse are available in the annual detailed statistical tables reports.
- Measurement error: Known sources of measurement error include differences in respondent interpretations of the definitions of R&D activities; differences in accounting procedures, specifically, the characterization and reporting of R&D activities by large defense contractors funded by the U.S. federal government; the reporting of R&D activities by companies classified in the R&D services industry, NAICS 5417; and differences in how companies count and report numbers of employees in various categories, including whether they work on R&D full-time or part-time on R&D. No quantitative metrics of measurement error are produced, but ongoing efforts to minimize measurement error include questionnaire pretesting, improvement of questionnaire wording and format, inclusion of more cues and examples in the questionnaire instructions, in-person and telephone interviews and consultations with respondents, and post-survey evaluations.
6. Data Availability and Comparability
- Data availability: Data produced from the SIRD for 1991–2007 and data produced from the BRDIS for 2008 to the current survey year are available at
- Data comparability: BRDIS is a cross-sectional survey designed to produce annual estimates of R&D performance and related statistics, as was its predecessor, SIRD. However, many of the companies that perform large amounts of R&D are included in the survey each year. Thus, there is a significant longitudinal component to the survey. Because of this and the generally low sampling variability of the annual level estimates, estimates of year-to-year changes are generally precise. Estimates for changes covering a longer time span will generally be less precise.
There is no conclusive evidence that the redesign of SIRD to create BRDIS caused breaks in the series for the items common to both surveys, because no substantial changes in scope and methodology were introduced. Significant efforts were made to minimize the effects of changes in the assignment of companies to industry strata, the inclusion of data on worldwide activities, changes in the measurement of employment, and the use of a modular survey questionnaire and to preserve the comparability of the data series. Nonetheless, an unanticipated drop in the number of full-time equivalent scientists and engineers occurred between the last cycle of SIRD (2007) and the first cycle of BRDIS (2008).
7. Data Products
- Publications: BRDIS data are published in NCSES InfoBriefs and Detailed Statistical Tables Reports in the Business and Industrial R&D series, available at http://www.nsf.gov/statistics/industry/. Data from BRDIS are also used in the biennial report Science and Engineering Indicators, available at http://www.nsf.gov/statistics/seind/.
- Electronic access: The BRDIS contains confidential data that are protected under Title 13 and Title 26 of the U.S. Code. Restricted microdata can be accessed at any of the 15 secure Research Data Centers administered by the Center for Economic Studies (CES) at the Census Bureau. Researchers interested in analyzing microdata can apply for data use by submitting a proposal to the CES, which evaluates proposals based on their benefit to the Census Bureau, scientific merit, feasibility, and risk of disclosure. To learn more about the Research Data Centers and how to apply, please visit the CES page on research opportunities. For additional information about the application process, including how to initiate a project, please contact the administrator at the primary site where the research will be conducted.
8. Contact Information
For additional information about this survey, please contact the Project Officer.
Raymond M. Wolfe
Research and Development Statistics Program
National Center for Science and Engineering Statistics
National Science Foundation
4201 Wilson Boulevard, Suite 965
Arlington, VA 22230
Phone: (703) 292-7789