Separator.

The BERD Survey is the primary source of information on research and development (R&D) expenditures and R&D employees of for-profit, publicly or privately held, nonfarm businesses with 10 or more employees in the United States that performed or funded R&D domestically or abroad.

Survey Info

Summary

The BERD Survey and its predecessors are the primary sources of information on research and development (R&D) performed or funded by businesses within the United States since 1953. The BERD Survey was developed and is cosponsored by the National Center for Science and Engineering Statistics within the National Science Foundation and by the Census Bureau, which collects and tabulates data for the survey in accordance with an agreement between the two agencies. Results are used to assess trends in the performance and funding of business R&D. The annual survey examines a nationally representative sample of companies in manufacturing and nonmanufacturing industries. Its predecessors were the Survey of Industrial Research and Development (SIRD) (1953–2007), the Business R&D and Innovation Survey (BRDIS) (2008–16), and the Business Research and Development Survey (BRDS) (2017–18). Note that SIRD and BRDIS collected statistics for businesses with five or more employees. BRDS produced (and the BERD Survey produces) statistics for businesses with 10 or more employees. Beginning in survey year 2018, companies that performed or funded less than $50,000 of R&D were excluded from tabulation. In prior years, companies that performed or funded any amount of R&D were tabulated. This change has affected the comparability of these estimates to those published in prior years, although it is estimated that companies that performed or funded less than $50,000 of R&D accounted for a very small percentage of total domestic R&D. SIRD, BRDIS, and BRDS all collected data on the full range of R&D activities, but only BRDIS collected data on business innovation activities. Data on business innovation activities have been collected by the Annual Business Survey (ABS) since 2017. Statistics on the R&D activities of businesses with one to nine employees are also collected by the ABS.

Areas of Interest

Survey Administration

The BERD Survey is cosponsored by the National Center for Science and Engineering Statistics and the Census Bureau.

Survey Details

Status Active
Frequency Annual
Reference Period Calendar year 2021
Next Release Date September 2024

Methodology

Survey Description

Survey Overview (2021 Survey Cycle)

Purpose

The Business Enterprise Research and Development (BERD) Survey and its immediate predecessors, the Business R&D and Innovation Survey (BRDIS) and the Business Research and Development Survey (BRDS), are collectively referred to as the BERD Survey in this overview. The BERD Survey is the primary source of information on research and development (R&D) expenditures and R&D employees of for-profit, publicly or privately held, nonfarm businesses with 10 or more employees in the United States that performed or funded R&D either domestically or abroad.

Data collection authority

The National Science Foundation Act of 1950, as amended, and the America COMPETES Reauthorization Act of 2010, collected under Office of Management and Budget control number 0607-0912.

Major changes to recent survey cycle

To minimize reporting burden especially for smaller companies, survey items are rotated on and off the survey on an odd- and even-numbered year schedule. Statistics on patents, intellectual property, and technology transfer activities were rotated off the survey for 2021. Items rotated on the survey for 2021 include questions on R&D performed by others by type of performer, federal R&D by government agency, and R&D by application area.

Key Survey Information

Frequency

Annual.

Initial survey year

BERD Survey data collection began for 2019. BRDS collected data for 2017–18, BRDIS collected data for 2008–16, and the Survey of Industrial R&D (SIRD) collected data for 1953–2007.

Reference period

Calendar year 2021.

Response unit

Companies with known positive R&D activity (approximately 21,000) and with unknown R&D activity (approximately 26,500).

Sample or census

Sample survey representing for-profit, publicly or privately held companies with 10 or more employees in the United States that performed or funded R&D either domestically or abroad in the manufacturing, mining, utilities, wholesale trade, transportation, information, or services industries.

Population size

A total of 1,137,000 companies.

Sample size

A total of 47,500 companies prior to data collection. The actual number of companies that remained within the scope of the survey between sample selection and tabulation was 44,000.

Key variables

Key variables of interest are listed below.

  • Business activity codes based on the North American Industry Classification System (NAICS)
  • Geographic location of domestic and foreign R&D performance of U.S.-based companies
  • Industry coding based on NAICS
  • R&D capital expenditures
  • R&D performance (domestic and foreign R&D for U.S.-based companies)
  • Sales
  • Sources of R&D funding
  • Technology focus areas
  • Total and R&D employment
  • Type of R&D cost (e.g., salaries and fringe benefits)
  • Type of R&D work (basic research, applied research, and development)
  • Activities with academia (data collected for odd-numbered years)
  • Federal R&D by funding government agency (odd years)
  • R&D by application area (odd years)
  • R&D performed by others by type of performer (odd years)
  • Intellectual property (even years)
  • Patents (even years)
  • Technology transfer activities (even years)

Survey Design

Target population

The target population consists of all for-profit nonfarm companies that are publicly or privately held, have 10 or more paid employees in the United States, and have at least one establishment that is classified in an in-scope sector based on NAICS, is in business during the survey year, and is physically located in the United States.

Sampling frame

The Business Register (BR) serves as the primary input to the sampling frame. The BR is the Census Bureau’s comprehensive database of U.S. businesses. BR data are compiled from a combination of business tax returns, data collected from the economic census, and data from other Census Bureau surveys.

Sample design

The BERD Survey has a stratified sample design that uses both simple random sampling and probability proportional to size (PPS) sampling within strata. Stratification is based on R&D activity and a NAICS-based industry code. For companies with known R&D activity, PPS sampling is based on historical R&D performance. For companies with unknown R&D activity, PPS sampling is based on annual payroll. Companies known to perform large amounts of R&D and companies with large amounts of payroll are selected with certainty.

Data Collection and Processing

Data collection

The BERD Survey uses Web reporting via the Census Bureau’s Centurion system with embedded electronic worksheets designed to ease respondent burden. Respondents have the option of downloading a PDF version of the questionnaire, but the overwhelming majority report via the online system (≥ 99%) and not by mail (< 1%).

Data processing

All data submitted by respondent companies are reviewed to ensure that data fields are complete and that data are internally consistent. Given the size and complexity of the BERD Survey, many survey responses contain errors that require correction or unusual patterns that require validation. Several hundred automated edit checks are applied to improve the efficiency of analyst data review and correction. Approximately two-thirds of these edit checks are designed to catch arithmetic errors and logically inconsistent responses (balance edits). The remaining edit checks are designed to flag outliers for further analyst review (analytical edits). During editing, if additional information or data corrections are needed, respondents are contacted. If additional information or corrected data cannot be obtained from respondents, data are imputed.

Estimation techniques

The general methodology used to produce estimates from the BERD Survey involves sums of weighted data (reported or imputed), in which the weights are the product of the sampling weight and the nonresponse adjustment factor. However, there are some exceptions, which are described in the technical notes in the annual reports for BRDIS, BRDS, and the BERD Survey (https://www.nsf.gov/statistics/srvyberd/).

Survey Quality Measures

Sampling error

Estimates based on the total sample have small relative standard errors (RSEs). An RSE is the standard error of the survey estimate divided by the survey estimate and then multiplied by 100. For 2021, RSEs for domestic R&D performance paid for by the company, paid for by others, and total were 0.20%, 0.48%, and 0.18%, respectively. Estimates of sampling errors associated with each cell in the detailed statistical tables are available by request.

Coverage error

Coverage error is minimal because the BR, the source for the BERD Survey, is continually updated and contains comprehensive coverage of all domestic businesses.

Nonresponse error

The unit response rate was 69% for 2020. Except for estimates of company counts, unit nonresponse is handled by adjusting weighted reported and imputed data by multiplying each company's sampling weight by a nonresponse adjustment factor. For estimates of company counts, other adjustments for nonresponse are made. Detailed descriptions of the adjustments for nonresponse are available in the annual reports containing detailed statistical tables.

Measurement error

Known sources of measurement error include differences in respondent interpretations of the definitions of R&D activities; differences in accounting procedures, specifically, the characterization and reporting of R&D activities by large defense contractors funded by the U.S. federal government; the reporting of R&D activities by companies classified in the scientific research and development services industry, NAICS 5417; and differences in how companies count and report numbers of employees in various categories, including whether they work on R&D full time or part time. No quantitative metrics of measurement error are produced, but ongoing efforts to minimize measurement error include questionnaire pretesting, improvement of questionnaire wording and format, inclusion of more cues and examples in the questionnaire instructions, in-person and telephone interviews and consultations with respondents, and post-survey evaluations.

Data Availability and Comparability

Data availability

Statistics from the BERD Survey for 2019–21 are available at https://ncses.nsf.gov/surveys/business-enterprise-research-development/. Statistics produced from BRDS for 2017 and 2018, BRDIS for 2008–16, and SIRD for 1991–2007 are available at https://www.nsf.gov/statistics/industry/. Statistics from SIRD dating to 1953 are available at https://www.nsf.gov/statistics/iris/.

Data comparability

The BERD Survey is a cross-sectional survey designed to produce annual estimates of R&D performance and related statistics, as were its predecessors, BRDS, BRDIS, and SIRD. However, many of the companies that perform large amounts of R&D are included in the survey each year. Thus, there is a longitudinal aspect to the survey. Because of this and the generally low sampling variability of the annual level estimates, estimates of year-to-year changes are generally precise. Estimates for changes covering a longer time span generally will be less precise.

Beginning in survey year 2018, companies that performed or funded less than $50,000 of R&D were excluded from tabulation. In prior years, companies that performed or funded any amount of R&D were tabulated. This change has affected the comparability of these estimates to those published for years prior to 2018. These companies in aggregate represented a very small share of total R&D expenditures in prior years, but they accounted for a larger share of count estimates. Had the companies under this threshold been included in the 2019 estimates, they would have contributed approximately $90 million to overall R&D expenditures and would have added around 7,500 to the estimated number of U.S. companies with R&D expenditures. It is assumed that this group of companies would have contributed similar levels of R&D and number of companies to the 2020 estimates.

In survey year 2017, the employment threshold for inclusion in the target population, described above, was increased from 5 employees to the current threshold of 10 employees for international comparability.

Except for the discontinuance of the collection of business innovation data by BRDIS and the transfer of the production of business innovation statistics to the Annual Business Survey beginning with the 2017 cycles of both, the transition from BRDIS to BRDS to the BERD Survey produced no breaks in the series for the items common to all surveys.

There is no conclusive evidence that the redesign of SIRD to create BRDIS caused breaks in the series for the items common to both surveys because no substantial changes in scope and methodology were introduced. Significant efforts were made to preserve the comparability of the data series and to minimize the effects of (1) changes in the assignment of companies to industry strata, (2) the inclusion of data on worldwide activities, (3) changes in the measurement of employment, and (4) changes because of the use of a modular survey questionnaire. Nonetheless, possibly because of improved reporting instructions, an unanticipated drop in the number of full-time equivalent scientists and engineers was reported between the last cycle of SIRD (2007) and the first cycle of BRDIS (2008).

Data Products

Publications

BERD Survey data are published in NCSES InfoBriefs and reports containing detailed statistical tables in the following series: Business Research and Development, Business R&D and Innovation, and Industrial R&D. Data from the BERD Survey are also used in the National Science Board’s congressionally mandated report Science and Engineering Indicators and in National Patterns of R&D Resources.

Electronic access

Results from SIRD are available at NCSES' Industrial Research and Development Information System historical data website (https://www.nsf.gov/statistics/iris/).

The BERD Survey contains confidential data that are protected under Title 13 and Title 26 of the U.S. Code. Restricted microdata can be accessed at the secure Federal Statistical Research Data Centers (FSRDCs) administered by the Census Bureau. FSRDCs are partnerships between federal statistical agencies and leading research institutions. FSRDCs provide secure environments supporting qualified researchers using restricted-access data while protecting respondent confidentiality. Researchers interested in using the microdata can submit a proposal to the Census Bureau, which evaluates proposals based on their benefit to Census, scientific merit, feasibility, and risk of disclosure. To learn more about the FSRDCs and how to apply, please visit https://www.census.gov/about/adrm/fsrdc.html.

Survey Overview (2021 Survey Cycle)

Purpose. The Business Enterprise Research and Development (BERD) Survey, successor to the Business Research and Development Survey (BRDS) (2017–18), the Business R&D and Innovation Survey (BRDIS) (2008–16), and the Survey of Industrial Research and Development (SIRD) (1953–2007), is the primary source of information on research and development (R&D) expenditures and R&D employees of for-profit, publicly or privately held, nonfarm businesses with 10 or more employees in the United States that performed or funded R&D either domestically (in the 50 states and the District of Columbia) or abroad.

Data collection authority. The information collected by the BERD Survey is solicited under the authority of the National Science Foundation (NSF) Act of 1950, as amended, and the America COMPETES Reauthorization Act of 2010. Response to this annual survey is mandatory and confidential; the Census Bureau collects the data under the authority of Title 13, Section 8, of the U.S. Code. The Office of Management and Budget (OMB) control number is 0607-0912.

Survey sponsors. The BERD Survey is cosponsored by the National Center for Science and Engineering Statistics (NCSES) within NSF and the Census Bureau.

Survey collection and tabulation agent. The survey is conducted annually by the Census Bureau in accordance with an interagency agreement with NCSES.

Key Survey Information

Frequency. Annual.

Initial survey year. BERD Survey data collection began in 2019. BRDS collected data for 2017–18, BRDIS collected data for 2008–16, and SIRD collected data for 1953–2007.

Reference period. Calendar year 2021.

Response unit. Company.

Sample or census. Sample.

Population size. 1,137,000 companies.

Sample size. 47,500 companies; the actual number of companies that remained within the scope of the survey between sample selection and tabulation was 44,000.

Survey Design

The survey is administered both to companies known to have performed R&D and to companies with no known history of R&D activity. The BERD Survey has been designed to provide detailed statistics on global and domestic R&D expenditures of companies located in the United States and on these companies’ R&D employees.

The survey is sent to a single coordinator within each company, but it is organized into sections that help the coordinator collect specific types of information from different experts in the company (human resources, accounting, R&D managers, etc.). Foreign-owned companies are instructed to report only for company operations owned by the U.S. subsidiary and, for purposes of the survey, to treat the U.S. subsidiary’s foreign owners as if they were unrelated third parties.

Target Population

The target population for the BERD Survey consists of all for-profit companies that have 10 or more paid employees in the United States, that have at least one establishment located in the United States that is in business during the survey year, and that are classified in certain industries based on the 2017 North American Industry Classification System (NAICS), with a particular focus on those companies that perform R&D in the United States.

The Business Register, a Census Bureau compilation that contains information on more than 7.3 million establishments with paid employees, serves as the primary input to the sampling frame from which the BERD Survey sample is selected. For a given company with more than one establishment, the prior year’s annual payroll and employment data for its active establishments are summed to the company level.

Sampling Frame

The scope of the 2021 BERD Survey is limited to companies that (1) are in business primarily to make a profit; (2) are classified within a specific set of NAICS industries; (3) have 10 or more paid employees in the United States, based on employment on 12 March 2020; (4) have at least one establishment that is physically located in the United States and is in business at the end of calendar year 2021 (the time at which the Census Bureau finished the 2020 Business Register Processing); and (5) are not federally funded R&D centers.

Single-unit company records were extracted from the 2020 Business Register if the company had at least 10 paid employees in 2020 or, when employment information was unavailable, if the company’s 2020 payroll was greater than or equal to $500,000. Companies were removed from the sampling frame if their NAICS codes were designated as agriculture, forestry, fishing, and hunting (NAICS 11); postal service (NAICS 491); educational services (NAICS 61); private households (NAICS 814); or public administration (NAICS 92); or if they were no longer in business or were nonprofits. Companies were also removed from the sampling frame if they were not located in the 50 U.S. states or the District of Columbia.

Records for active establishments from multiunit companies were extracted from the 2020 Business Register if the given establishment’s 2020 payroll was greater than zero or if the establishment employed at least one person in 2020. Prior to creating records for multiunit companies from these establishments, establishments classified as postal service (NAICS 491), private households (NAICS 814), or public administration (NAICS 92) were removed, as were those that were not physically located in the 50 U.S. states or the District of Columbia. Unlike single-unit companies, establishments classified as agriculture, forestry, fishing, and hunting (NAICS 11) or educational services (NAICS 61) were not initially removed during the construction of multiunit company records. Establishments classified in NAICS 11 or NAICS 61 contributed to the classification of the multiunit companies, and their payroll could have been included in determining the company’s measure of size. From the resulting set of multiunit companies, companies were removed from the sampling frame if they had fewer than 10 paid employees or if the payroll associated with their nonprofit establishments was greater than the payroll of their for-profit establishments.

For each company on the sampling frame, a measure of size was assigned. The measure of size for a given company was based on R&D, if R&D data from the last 5 years were available from (1) the BERD Survey, (2) online financial databases, (3) the Report of Organization conducted as part of the Company Organization Survey (in years not ending in “2” or “7”) or as a supplement to the Economic Census (in years ending in “2” or “7”), or (4) qualified R&D expenses from the Internal Revenue Service (IRS). For all other companies, the measure of size was based on total annual payroll for 2020 from the Business Register.

Industry Classification for Sampling

Each company was assigned to 1 of 60 industry sampling strata based on the reported business segment in which the company performed the largest amount of total domestic R&D, as reported within the previous 5 years (2016 BRDIS, 2017–18 BRDS, and 2019–20 BERD Survey), if available. If these business segment data were not reported for a given company, assignment was based on the NAICS codes of its establishments in the Business Register using the following method, with some adjustments made to account for vertical integration of related business activities within the company. The company was first assigned to the economic sector, defined by a two-digit NAICS code that accounted for the highest percentage of its aggregated annual payroll. Then the company was assigned to a subsector, defined by a three-digit NAICS code that accounted for the highest percentage of its annual payroll within the economic sector. Next, the company was assigned a four-digit NAICS code within the subsector, again based on the highest percentage of its aggregated annual payroll within the subsector. Finally, the company was assigned a six-digit NAICS code within the four-digit NAICS code, based on the highest percentage of its aggregated annual payroll within the four-digit NAICS code. Companies with an assigned six-digit NAICS code beginning with 11 or 61 were removed from the sampling frame at this point, and their R&D and associated data are not included in the tabulations. The industry used for sampling purposes was not necessarily the same code used for publication; see section “Postsampling Industry Classification.”

Stratification of the Sampling Frame

Each company in an industry sampling stratum was further assigned to one of two R&D groups based on information about its prior domestic R&D activity: (1) companies with a positive value for the measure of size based on R&D (known positive R&D group), or (2) companies with unknown R&D activity (unknown R&D group). For the 2021 BERD Survey, there were 31,500 companies in the first group and 1,106,000 companies in the second group, for a total of 1,137,500 companies (table A-1).

Sample Selection

In the known positive R&D group, Pareto probability-proportional-to-size (PPS) sampling was used within each noncertainty industry stratum, where the probability of selection was proportional to the company’s measure of size. In the unknown R&D group, Pareto PPS sampling was used within most industry strata; simple random sampling (SRS) was used instead of Pareto PPS for industries in which the number of companies in the sampling frame was high and the likelihood of R&D was low. For both the known positive and unknown groups, a separate stratum was created for companies with fewer than 25 employees, regardless of industry. Pareto PPS sampling was used within these strata. Each sampling stratum had a certainty and noncertainty portion (table A-2). Across both groups (known positive and unknown), companies in select industries and companies that exhibited characteristics of having large R&D amounts, including those with the largest annual payroll, were selected for the sample with certainty (i.e., the probability of selection was equal to 1). The probability of selection for other companies in the known positive R&D and unknown R&D groups depended on their size, the number of companies selected, and the total size or number of companies in their strata. The number of companies selected was based on a coefficient of variation constraint on the estimated sample total for the stratum and was adjusted, if necessary, to ensure that the minimum probability of selection is 0.05 for the known positive R&D group and one of six values for the unknown R&D group: 0.004, 0.01, or 0.02 for nonmanufacturing industries (NAICS other than 31–33) and incomplete manufacturers (incomplete NAICS beginning with 3), depending on the population size and likelihood of R&D; 0.04 for printing and related support activities manufacturing (NAICS 323), furniture and related product manufacturing (NAICS 337), and computer systems design and related services (NAICS 5415); 0.02 for the remaining manufacturing (NAICS 31–33) industries; and 0.05 for the companies with fewer than 25 employees. Within the unknown R&D group, a total of 33,500 companies had a known zero value for the measure of size based on R&D from prior survey response. The probability of selection for those companies was set to zero.

Once selected, each company was assigned a sampling weight equal to the reciprocal of its probability of selection for the sample. Companies that were selected for the sample with certainty were assigned sampling weights equal to 1, and companies that were selected using SRS or Pareto PPS sampling were assigned weights ranging from 1 to 250. A complete and detailed description of the sample design and estimation methodology is given in the annual BERD Survey methodology report available from the NCSES Survey Manager.

Sample Size

Using the sample design parameters detailed above, a total of 47,500 companies were selected, of which 21,000 companies were in the known positive R&D group, and 26,500 companies were in the unknown R&D group (table A-3 and table A-4).

During the survey’s annual contact update procedures, 150 large R&D performers from the 2020 sample were found that were not included on the 2021 sampling frame. The primary reasons for this were mergers, acquisitions, or other company structure changes that cause a difference between a company’s current Business Register ID and its prior year survey ID—in this case, its ID in the 2020 BERD Survey file. To follow up with these companies, they were added to the 2021 sample with certainty, and 150 companies were mailed a survey letter. Because it was expected that many of these records would not contribute to 2021 BERD Survey tabulations due to changes in company structure, these companies are not included in sampling frame counts or sample sizes (table A-5). A complete and detailed description of the selection of additional sampling units is given in the annual BERD Survey methodology report available from the NCSES Survey Manager.

Data Collection and Processing Methods

Starting in the 2017 cycle, an initial letter was mailed to all companies instructing them to report via the Census Bureau’s Centurion data collection instrument. Paper questionnaires were mailed by respondent request only and during a later nonresponse follow-up for delinquent companies. For paper versus electronic response rates, see section “Response by Mode.”

Questionnaires

Beginning in the 2017 cycle, all companies completed the same questionnaire, BRD-1, which combined the standard form BRDI-1 and the abbreviated form BRDI-1(S) used in prior cycles (table A-6). Companies that reported less than $1 million in R&D paid for by the company or less than $1 million in R&D funded by others completed abbreviated worksheets in lieu of more detailed questions asked of companies with $1 million or more in R&D. As was the case in previous cycles, some companies with decentralized records requested a special reporting arrangement with the Census Bureau. Under this arrangement, the survey form was mailed to multiple operating units within the same company, and each Alternate Reporting Unit (ARU) completed the survey individually. The data for these ARUs were also tabulated separately in published estimates and were not consolidated at the company level (table A-7).

Because of the potential compartmentalization of organizational knowledge within companies (particularly in larger companies), the BERD Survey questionnaire was organized into sections based on the subject matter of the questions. These sections included the following:

Section 1. Company information. Includes basic questions about company ownership, lines of business, sales, and employee data.

Section 2. R&D paid for by your company. Includes geographic location and accounting questions about the company’s R&D expenses.

Section 3. Costs paid for by others. Includes accounting questions about R&D paid for by others, such as the company’s customers or grant-giving organizations.

Section 4. Assets. Includes questions on total assets, total capital expenditures, and capital expenditures for R&D operations.

Section 5. Management and strategy of R&D. Includes questions related to the characteristics and focus of the company’s R&D. This section was targeted toward company employees responsible for managing R&D departments or programs.

Section 6. Human resources. Includes questions related to the human resources involved in the company’s R&D activities.

Section 7. Tax topics. Includes questions on filing federal and state tax credits for research activities.

For specific differences among the BERD Survey questionnaires, see section “Data Comparability (Changes).”

Response Rates
Unit Response Rates

Of the companies surveyed for the 2021 survey, 30% did not submit any response, and an additional 1% did not provide enough information to be treated as responses. Nonresponse studies are conducted periodically to assess reasons for nonresponse and possible nonresponse bias. Three metrics used by NCSES and the Census Bureau to measure unit response to the BERD Survey were check-in rates, unit response rates (URRs), and coverage rates.

Check-in rate. The check-in rate is defined as the unweighted number of surveys that were either submitted online or mailed in by in-scope companies, divided by the unweighted total number of all in-scope companies in the sample. Response to individual questions did not factor into this metric.

Coverage rate. BERD Survey Managers track a coverage rate that is a weighted measure of survey response based on the measure of size at the time of sample selection. The coverage rate measures how much of the weighted measure of size for in-scope companies in the sample is accounted for by respondents to the survey.

Unit response rate (URR). The URR is the unweighted number of responding companies with positive data for at least one of the survey’s key items (i.e., worldwide R&D expenses or R&D costs funded by others, worldwide or domestic sales, or worldwide or domestic employees), divided by the unweighted total number of in-scope companies in the sample.

For the 2021 BERD Survey, the check-in rate was 70% and the URR was 69%. The coverage rate for the 2021 BERD Survey was 92% for the known positive R&D group and 69% for the unknown R&D group (table A-8 and table A-9).

Item Response Rates

The BERD Survey collects data for over 500 variables, and the distribution of values reported by sample companies is highly skewed. Thus, rather than report unweighted item response rates, total quantity response rates are calculated, which are based on weighted data.

Total quantity response rate (TQRR). For a given published estimate other than count or ratio estimates, TQRR is the percentage of the weighted estimate based on data that were reported by units in the sample or on data that were obtained from other sources and were determined to be equivalent in quality to reported data. The TQRR for total R&D performed in the United States in 2021 was 70%.

Total quantity nonresponse rate (TQNR). For a given published estimate, TQNR, defined as 100% minus TQRR, is calculated for each tabulation cell from the BERD Survey, except for cells that contain count or ratio estimates. TQNR measures the combined effect of the procedures used to handle unit and item nonresponse on the weighted BERD Survey estimate. TQNR tables giving imputation rates corresponding to each data table are available from the NCSES Survey Manager. Also see section “Item Nonresponse.”

Response by Mode

Overall, more than 99% of checked-in cases responded to the BERD Survey by using the online version of the survey and less than 1% responded by mailing in the paper form. Lastly, more than 97% of checked-in companies with account managers—that is, the top R&D companies (based on prior year reported or imputed data) that were assigned an analyst to act as a single point of contact for all communications—responded using the online version of the survey.

Data Editing

Given the size and complexity of the BERD Survey, many survey responses included errors that required correction or had unusual patterns that required validation. Several hundred automated edit checks were programmed to improve the efficiency of analyst data review and correction (table A-10).

Approximately two-thirds of these edit checks were designed to catch arithmetic errors and logically inconsistent responses (balance edits). The remaining edit checks were designed to flag outliers for further analyst review (analytical edits). Descriptions of the data edits and edit failure rates are in annual methodology reports that are available from the NCSES Survey Manager.

During the editing and review process, several cases were identified where companies reported zero R&D or a relatively small amount of R&D, even though subject-matter experts expected large amounts of R&D to be reported. Some of these companies were contract research organizations or federal contractors that did not account for the costs they incurred while conducting customer-sponsored research as R&D; instead, they accounted for these as the cost of sales. The largest of these companies were contacted by analysts and asked to resubmit their surveys. In rare cases, if no response could be elicited from a company and public information was available related to costs for customer-sponsored R&D, those data were used to impute an R&D estimate for that company.

Techniques for Handling Unit and Item Nonresponse

For various reasons, many firms chose to return the survey questionnaire with one or more blank items. For some firms, internal accounting systems and procedures may not have allowed quantification of specific expenditures. Others may have refused to answer any questions as a matter of company policy. Weighted estimates produced from the BERD Survey include adjustments to account for companies that did not respond to the survey (unit nonresponse) and for companies that did respond but left some questions blank (item nonresponse).

Unit Nonresponse

The BERD Survey accounts for unit nonresponse by multiplying each company’s sampling weight by a nonresponse adjustment factor. To calculate the adjustment factors, each company in the sample that is eligible for tabulation is assigned to one (and only one) adjustment cell. The adjustment cells are based on the two R&D groups, which are subdivided according to R&D size and certainty status and also on the industry sampling strata described in the section “Stratification of the Sampling Frame” that are updated using information on industry classification reported in the BERD Survey. For a given adjustment cell, the nonresponse adjustment factor is the ratio of the sum of the weighted measure of size for all companies in the cell to the sum of the weighted measure of size for all companies in the cell with reported or imputed data. The measure of size used to select the sample for the 2021 BERD Survey (see section “Sampling Frame”) was also used to adjust for unit nonresponse. For companies in the known positive R&D stratum, the measure of size was based on R&D in the United States. For companies in the unknown R&D stratum, the measure of size was based on total annual payroll in the United States. Beginning in survey year 2018, this unit nonresponse adjustment was also applied to counts, patents, patent licensing agreements, and intellectual property protection. In previous survey years, no such adjustment was applied to those items.

R&D estimates for geographic areas outside the largest countries, states, and core-based statistical areas (CBSAs) that are dominated by a relatively small number of companies are particularly sensitive to changes in survey response because these areas may not be similarly represented by companies in the same adjustment cell.

Item Nonresponse

Item nonresponse for a given company is handled by item imputation. For account manager companies, large companies, and special cases, analysts impute these data using direct substitution of available company data (e.g., data from the company’s website, annual Form 10-K report, or administrative sources) or other methods determined appropriate by NCSES’s and the Census Bureau’s subject-matter experts. For other cases, including cases where analysts were unable to provide a superior estimate, data are imputed by programmed item imputation procedures. Depending on the particular item being imputed for a company, these procedures are based on a combination of (1) direct substitution of available company data, (2) ratio imputation using the company’s survey data for both the current and prior year, and (3) ratio imputation using survey data from both the company and other similar companies, which reported both the survey item being imputed for the company and the other survey item used in the ratio. Tables of imputation rates corresponding to each data table are available from the NCSES Survey Manager.

Estimation

The general methodology used to produce estimates from the BERD Survey involves sums of weighted data (reported or imputed) in which the weights are the product of the sampling weight and the nonresponse adjustment factor. However, there are some exceptions, which are described below.

Weighting

Estimates published for the BERD Survey are computed as sums of weighted data for sample companies that reported to the survey or for sample companies for which data could be reliably imputed based on prior reports or other information. Two types of weights are used for estimates of R&D: sampling weights and nonresponse adjustment factors. The sampling weight for a given company is calculated as the reciprocal of the company’s probability of inclusion in the sample. Nonresponse adjustment factors are used to represent companies in the sample that did not provide sufficient response data to be directly tabulated and whose data could not be imputed. For information on the calculation of the nonresponse adjustment factors, see section “Unit Nonresponse.”

Each value that contributes to a given BERD Survey estimate is multiplied by both its sampling weight and its nonresponse adjustment factor, and these weighted values are then summed to create the estimate. Beginning in survey year 2018, the nonresponse adjustment factor was also applied to estimates of counts, patents, and patent licensing agreements. In prior survey years, the adjustment was not applied to those items.

Postsampling Industry Classification

As mentioned in the section “Industry Classification for Sampling,” the industry classification assigned to companies for sampling was based on either reported BERD Survey business segment data from prior years or annual payroll. To produce more accurate estimates for the current survey year, a company’s reported business code, if available for the current survey year, was used to assign an updated industry code for tabulations. The company’s response to the domestic R&D performance questions from the current survey year was used to classify each company into the business code that accounted for the largest amount of total domestic R&D performance. The business codes reported by companies with large amounts of R&D were validated, and in some cases corrected, by survey staff. If no business code data were available for a company’s domestic R&D performance, the industry code used for sampling was also used for tabulations.

R&D, by Core-Based Statistical Area (CBSA) or State

CBSA-level estimates of business R&D are modeled using BERD Survey data reported by companies on the address of each of their largest R&D locations and administrative data from the Census Business Register on the payroll for each business establishment in the United States. The weighted amount of company R&D is apportioned to CBSAs within states using the same hybrid estimator method used for state estimates (below). Current BERD Survey methodology cannot attribute all R&D to a specific location. For example, R&D reported by companies that did not respond to geographic questions and for whom no method was available to impute locations is not assigned to a location. This R&D was performed in one of the 50 U.S. states or the District of Columbia, but the specific location is not estimated and is reported as “undistributed” in BERD Survey tables.

The estimation methodology for state estimates takes the form of a hybrid estimator, combining the unweighted reported amount, by state, with a weighted amount apportioned (or raked) across states with relevant industrial activity. The hybrid estimator smooths the estimate over states with R&D activity, by industry, and accounts for real observed change within a state.

R&D estimates for CBSAs and states that are dominated by a relatively small number of companies are particularly sensitive to changes in both unit and item response. Because companies rarely publish detailed information about the geographic location of their R&D operations, it is difficult to identify and correct erroneous responses to these questions. And, as noted earlier, the method used to correct for unit nonresponse relies on data from companies in the same industry or size category. These peer companies may report relatively less (or no) R&D in the geographic areas where a nonresponsive company performs R&D.

R&D, by Business Segment Code

To provide increased granularity on R&D activities, the BERD Survey includes questions asking companies to report data for business units below the company level. To support subcompany reporting, a list of business codes based on NAICS was provided in the BERD Survey for companies to use to categorize their business operations. The list of business codes for the 2021 cycle of the BERD Survey was based on the 2017 NAICS. To assist companies in selecting appropriate business codes, likely business codes were provided to respondents by prepopulating them on the online version of the survey and by printing them on the forms mailed to companies. For companies that reported to the 2017 BRDIS or 2018 BRDS, the most recent business codes reported by the company were used to provide the business codes. For companies that did not report to the 2017 BRDIS or 2018 BRDS, establishment payroll data from the Business Register were used to provide the business codes.

Company Counts

The BERD Survey is designed primarily to produce valid, useful estimates of money amounts related to the performance and funding of R&D. Because this is the survey’s focus, estimation of meaningful company counts is difficult. Consequently, throughout the history of the survey there have been various efforts to calculate frequencies more precisely. Following is a summary of the results of recent efforts.

Beginning in survey year 2019, company count estimates reflect a change in rounding methodology. Beginning in survey year 2018, a weight adjustment to account for unit nonresponse was implemented for company counts; as a result, company count estimates are not comparable with survey years prior to 2018.

The company count estimates for 2017 are not comparable with those for 2016 for two reasons. First, the target population for 2017 was restricted to companies with 10 or more domestic employees, whereas the 2016 survey population included companies with 5 or more employees. Second, no adjustment is applied to count estimates to correct for unit nonresponse, and overall response to the survey declined from 2016 to 2017.

The company count estimates for 2014 and onward are not comparable with estimates published for previous years. BRDIS review and correction procedures were amended beginning in survey year 2014 to include a more systematic analysis of companies reporting small amounts of R&D (less than $10,000) and no R&D employees. Many of these responses were determined to be reporting errors (false-positive R&D), and their R&D was set to zero. These corrections had a negligible impact on BRDIS R&D estimates compared to earlier years. But because these companies tended to have high sample weights, these corrections had a large impact on the estimate of R&D-active companies compared to prior years (when many fewer such corrections were made).

Revision of Capital Expenditure Estimates

Review of BERD Survey capital expenditures data in calendar year 2021 uncovered cases where reported or imputed R&D capital expenditures (CAPEX) were unrealistically high, resulting in inflated overall survey estimates. In addition to an overall escalation in the priority placed on the review of these data by survey staff, the following specific procedures were implemented to improve the quality of BERD Survey estimates of CAPEX and total capital expenditures (TCPEX):

  1. Use of total domestic capital expenditures data from the Census Bureau’s Annual Capital Expenditures Survey (ACES) when companies report TCPEX amounts significantly smaller than estimated in ACES.
  2. Imputing values of CAPEX for respondents that report the same value for CAPEX as they do for either domestic R&D expenditures (DOEXP) or domestic R&D performance.
  3. Flagging cases where reported CAPEX exceeds DOEXP for additional subject-matter expert review. Erroneous CAPEX values will be replaced with imputed values.
  4. Excluding influential outliers from the imputation bases for CAPEX. Imputation for CAPEX relies on industry ratios of reported CAPEX to reported TCPEX. These ratios can be highly influenced by outliers.
Tabulation Eligibility

Beginning in survey year 2018, companies that performed or funded less than $50,000 of R&D were excluded from tabulation. In prior years, companies that performed or funded any amount of R&D were tabulated. This change has affected the comparability of these estimates to those published for years prior to 2018. These companies in aggregate represented a very small share of total R&D expenditures in prior years, but they accounted for a larger share of count estimates. Had the companies under this threshold been included in the 2021 estimates, they would have contributed approximately $110 million to overall R&D expenditures and would have added approximately 7,800 companies to the estimated count of U.S. companies with R&D expenditures.

Survey Quality Measures

Sampling and Nonsampling Errors

The estimates produced from the BERD Survey are subject to both sampling and nonsampling errors. Sampling error is the difference between estimates obtained from the sample and results theoretically obtainable from a comparable complete enumeration of the sampling frame. This error results because only a subset of the sampling frame is measured in a sample survey. For published estimates from the BERD Survey, standard errors are produced for estimated percentages, while relative standard errors (RSEs) are produced for all other estimates. Tables of the estimated measures of sampling variability corresponding to each data table are available from the NCSES Survey Manager.

Beginning in survey year 2021, a data flag denoted by “r” was added to the data tables to denote any data estimates that had RSEs greater than 50%. If estimates required both an “r” and “i” flag (where an “i” flag indicates that more than 50% of the estimate is a combination of imputation and reweighting to account for nonresponse), the estimate “r” flag was assigned. This is because the “r” flag indicates the data is of lower quality than the “i” flag.

Standard errors may be used to define confidence intervals about the corresponding estimates with a desired level of confidence. If a confidence interval was constructed for each possible sample that could be selected using the same sample design, then it would be expected that the percentage of confidence intervals containing the result of a complete enumeration of the sampling frame would equal the percentage of the level of confidence. For example, the interval defined by a margin of error of 2 standard errors yields a confidence interval of approximately 95%.

Because relatively few companies perform R&D in the United States, and because the amount of R&D they perform is quite variable, it is difficult to achieve control over the sampling error of survey estimates produced from the BERD Survey. This depends on the correlation between the measure of size on the sampling frame that was used to assign the selection probabilities and the actual data that are collected in the BERD Survey, which cannot be predicted accurately for all companies when the sample is designed. However, the companies known to perform the greatest amounts of R&D are included in the sample with certainty so that these companies will not contribute to the sampling error of the resulting estimates produced from the BERD Survey.

The sample size is sufficiently large so that estimates based on the total sample are subject to low sampling error. However, because priority in designing the sample was given to industries that were identified in previous surveys as conducting large amounts of R&D expenditures, the sampling error may be larger for estimates for the lower-priority industries. The RSE for the estimate of total domestic R&D performed by the company was 0.18% in 2021.

Potential nonsampling errors include coverage error and various response and operational errors, such as errors during data collection, reporting errors, transcription errors, and bias due to nonresponse. These are errors that could also occur if a complete enumeration of the sampling frame had been conducted under the same conditions as the sample survey. Most of the important operational errors were detected and corrected during the course of reviewing the data for reasonableness and consistency. Although nonsampling error is not measured directly, quality control procedures were employed throughout the survey process to minimize this type of error.

Measurement Error

Variations in respondent interpretations of the definitions of R&D activities and variations in accounting procedures are of particular concern—specifically, the characterization and reporting of R&D activities by large defense contractors funded by the U.S. federal government, the reporting of R&D activities by companies classified in the scientific R&D services industry (NAICS 5417), and the method used by companies, in general, to count and report numbers of employees in various categories, such as the number of employees who work full time versus part time on R&D. The sophistication and comprehensiveness of a company’s accounting and personnel tracking systems often depend on its size and activities and on its willingness to accommodate government-sponsored surveys. Although no metric of measurement error is produced, ongoing efforts to minimize measurement error include questionnaire pretesting, improvement of questionnaire wording and format, inclusion of more cues and examples in the questionnaire instructions, in-person and telephone interviews and consultations with respondents, and post-survey evaluations.

Data Comparability (Changes)

The statistics resulting from this survey are intended primarily as indicators of absolute levels of R&D spending and personnel. Nevertheless, the statistics are often taken to be a continuous time series prepared using the same collection, processing, and tabulation methods. Although estimating the business portion of the nation’s R&D enterprise accurately has always been the goal, such strict uniformity has not been the case. Since the survey was first fielded, improvements have been made to increase the reliability of the statistics and to make the survey results more useful. To that end, past practices have been changed and new procedures instituted. Preservation of the comparability of the statistics has, however, been an important consideration in making these improvements. Nonetheless, changes to survey coverage and definitions, the industry classification system, estimation methods, and the procedure used to assign industry codes to multi-establishment companies, which are documented in the annual reports, may have affected the comparability of the statistics. Among the most demonstrable changes are those made to the questionnaires, and those changes are summarized below.

Differences between the 2020 and 2021 BERD Survey Questionnaires

The following changes were made to the 2021 BERD Survey from the 2020 BERD Survey:

  • Question on details of R&D performed by others was rotated back on.
  • Question on activities with academia was rotated back on.
  • Question on business code or codes in which the company performed R&D and was paid for by others was rotated back on.
  • Question on federal R&D by government agency was rotated back on.
  • Question on federal R&D by types of agreements was rotated back on.
  • Questions on assets were added to section 4. The capital expenditures questions were expanded, and the section was also renamed from “Capital Expenditures” to “Assets.”
  • Questions on R&D application areas were rotated back on.
  • Questions on patents, intellectual property, and technology transfer were cycled off.
Differences between the 2019 and 2020 BERD Survey Questionnaires

The following changes were made to the 2020 BERD Survey from the 2019 BERD Survey:

  • Questions on the contact information of the person primarily responsible for completing each section were added to the end of each section.
  • A follow-up question was added to the ownership question asking whether the new owner of the sampled company is a holding company or a private equity firm.
  • Question on details of R&D performed by others was cycled off.
  • Question on activities with academia was cycled off.
  • Question on business code or codes in which the company performed R&D and was paid for by others was cycled off.
  • Question on federal R&D by government agency was cycled off.
  • Question on federal R&D by types of agreements was cycled off.
  • Question on source of funds for domestic R&D funded by others was added to the section 3 worksheet.
  • Questions on R&D application areas were cycled off.
  • Questions on patents, intellectual property, and technology transfer were rotated back on.
Differences between the 2018 BRDS and 2019 BERD Survey Questionnaires

The following changes were made to the 2019 BERD Survey from the 2018 BRDS:

  • Questions on patents, intellectual property, and technology transfer were cycled off.
  • A question on filing for state tax credit for research activities in 2019 was added.
Differences between the 2017 and 2018 BRDS Questionnaires

The following changes were made to the 2018 BRDS from the 2017 BRDS:

  • A question on land acquisition capital expenditures for R&D operations was added.
  • Questions related to R&D on artificial intelligence (AI) were added.
  • Questions asking for country and state location of employees and R&D employees were added.
  • A question asking for counts of temporary and leased employees working on R&D was added.
Differences between the 2016 BRDIS and 2017 BRDS Questionnaires

The following changes were made to the 2017 BRDS from the 2016 BRDIS:

  • Questions on product and process innovation were removed.
  • A question on whether R&D paid for by the company included customer-sponsored R&D was added.
  • Questions on R&D funded by foreign governments or agencies and R&D funded by universities and colleges located outside the United States were added.
  • A question on whether any capital expenditures for R&D were funded, paid for, or reimbursed by others not owned by your company was added.
Differences between the 2015 and 2016 BRDIS Questionnaires

The following changes were made to the 2016 BRDIS from the 2015 BRDIS:

  • Questions on royalties and licensing fees were added to both the R&D paid for by the company and the R&D paid for by others type-of-cost sections.
  • Questions on the type of worldwide R&D paid for by the company were added to Form BRDI-1(S).
  • Questions on the type of worldwide R&D paid for by others were added to Form BRDI-1(S).
  • A question on all other contract R&D services paid for by customers was added to Form BRDI-1(S).
  • A question on R&D performed by universities, colleges, and academic researchers located inside the United States was added, while a question on R&D performed by foreign governments was removed.
  • A question on R&D funded by universities, colleges, and academic researchers located inside the United States was added, while a question on R&D funded by foreign governments was removed.
  • A question on whether utility patent sales and licensing revenues were received from companies or organizations outside the United States was added to Form BRDI-1.
  • A question on whether utility patent purchases and licensing payments were made to companies or organizations outside the United States was added to Form BRDI-1.
  • A question on whether the company filed for the tax credit for increasing research activities (IRS Form 6765) in tax year 2016 was added to Form BRDI-1.
  • Questions on the importance of intellectual property protections were removed.
Differences between the 2014 and 2015 BRDIS Questionnaires

The following changes were made to the 2015 BRDIS from the 2014 BRDIS:

  • A question on phase IV clinical trials was added to Form BRDI-1.
  • A question on R&D performed by others that was paid for by U.S. federal government agencies or laboratories was added to Form BRDI-1.
  • Questions on the type of foreign R&D paid for and performed by the company were added to Form BRDI-1.
  • Questions on the type of foreign R&D performed by the company but paid for by others were added to Form BRDI-1.
  • A question on nonprovisional utility patents was added to Form BRDI-1.
  • Questions related to R&D in photonics and optics application areas were removed from Form BRDI-1.
Differences between the 2013 and 2014 BRDIS Questionnaires

The following changes were made to the 2014 BRDIS from the 2013 BRDIS:

  • A question on monetary gifts to universities or colleges restricted to supporting R&D was added to Form BRDI-1.
  • A question on revenue received from patent licensing was added to Form BRDI-1.
  • A question on purchasing patents from others was added to Form BRDI-1.
  • A question on licensing patents from others was added to Form BRDI-1.
  • One business code was added to the list of business codes collected on the survey: 33333, digital cameras manufacturing. In prior years, this line of business was included in the business code 33412, computers and peripheral equipment manufacturing and magnetic and optical media.
  • Questions related to patenting were removed from Form BRDI-1(S).
  • Questions related to innovation were added to Form BRDI-1(S).
Differences between the 2012 and 2013 BRDIS Questionnaires

The following changes were made to the 2013 BRDIS from the 2012 BRDIS:

  • The list of countries for which foreign R&D performance data were collected was expanded by three: Hungary, Luxembourg, and Norway.
  • A question was restored asking the amount of R&D that the company plans to recoup through indirect charges on U.S. federal government contracts (independent R&D). This question was last asked on the 2010 BRDIS.
  • Questions on R&D for software products and R&D for embedded software were combined.
  • A question on R&D for software products and embedded software paid for by the federal government was added.
  • A question related to the educational attainment of scientists and engineers was restored. This question was last asked on the 2010 BRDIS.
  • Two business codes were added to the list of business codes collected on the survey: 32542, biotechnology-based pharmaceutical and biological products (except diagnostics), and 51801, cloud computing applications and Internet-based software services.
  • Delinquent companies in the known positive R&D stratum for the past two survey cycles were sent Form BRD-1(S) to see if they would report at least the high-level numbers.
Differences between the 2011 and 2012 BRDIS Questionnaires

For 2012, a much shorter (8-page) version of the short form, BRD-1(S), was implemented. The form included 19 high-level-detail items on worldwide sales; domestic sales; R&D expenses funded both by the company and by others; employment both worldwide and domestic, including R&D employment; and patents applied for and issued. Companies that reported $1 million or more of domestic R&D performance were then sent the long form (BRDI-1) for additional details. The BRD-1(S) form was sent to companies in the unknown and known zero R&D strata. In section 2, the questionnaire collected the additional detail categories for capital expenditures. In section 3, four agencies were added to the type of agency question so as to reduce the amount reported in the “all other” category. In section 4, the percentage of R&D that was directed toward business areas or product lines new to the respondent’s company and also the percentages that pertain to defense applications, health or medical applications, or agricultural applications were added for R&D funded by the company and R&D funded by others.

Differences between the 2010 and 2011 BRDIS Questionnaires

For the 2011 data collection, the innovation questions and instructions in section 1 were changed based on the results of the 2010 experiment. Cycling continued for data items that were not needed every year. The survey was expanded in several ways to address data gaps: the list of countries in which companies could report foreign R&D performance was expanded, a question was added to collect intracompany R&D transactions, and questions were added about companies’ second-largest R&D location. In addition, questions pertaining to full-time equivalent (FTE) R&D scientists and engineers were revised to improve respondent understanding of survey concepts.

Differences between the 2009 and 2010 BRDIS Questionnaires

For the 2010 data collection, the most notable changes made to the questionnaire were the inclusion of a one-time section (section 7) on R&D time frame and R&D product life, the inclusion of an experiment testing the impact of different innovation questions and instructions, and the addition of a survey supplement to collect detailed information from companies reporting R&D paid for by others. In addition, questions and instructions about company ownership were expanded to clarify, especially for foreign-owned companies, the information that should be reported on the survey. Cycling began for data items that were not needed every year from every company. These items will be returned to the questionnaire cyclically, depending on the demand for and quality of the collected data. Finally, data items that were poorly reported during the first two cycles of BRDIS were deleted.

The section titled “R&D Time Frame and R&D Product Life” was added for the 2010 cycle to aid in estimating the depreciation of R&D when it is treated as an investment in the U.S. System of National Accounts.

An experiment testing the impact of different innovation questions and instructions used two versions of the BRDIS short form. The innovation questions on the 2010 Form BRDI-1A were identical to questions used on the 2009 Form BRDI-1A, and the 2010 Form BRDI-1B altered the questions and instructions to replicate innovation questions on the European Union’s Community Innovation Survey. The experiment did not produce statistically significant differences in measured rates of innovation.

Differences between the 2008 and 2009 BRDIS Questionnaires

Several changes were made to the 2009 BRDIS questionnaire—in part, to address reporting errors observed during the 2008 survey cycle. These changes included the following:

  • A screening question at the beginning of the form asking companies whether they had R&D activity during the reporting period was removed.
  • Exclusion instructions in the main R&D expense question were replaced with a series of targeted questions. This approach was based on the premise that the economic concepts requested by BRDIS do not always conform to the R&D measures tracked by companies. Rather than directly asking for concepts that may diverge from respondent preconceptions about R&D, the approach in 2009 guided respondents to derive amounts that conformed to the BRDIS definition of R&D.
  • Instructions in the question about R&D paid for by others were replaced with a series of targeted questions.
  • Data for R&D performed by others were derived rather than collected directly. For the 2009 cycle of BRDIS, R&D performed by others was the sum of two R&D costs known to be tracked by companies: payments to business partners for collaborative R&D and purchased R&D services.
  • The order of the “Management and Strategy of R&D” and the “Financial Schedule B” (R&D paid for by others) sections was switched.

Definitions

Capital expenditure. Capital expenditures are payments by a business for assets that usually have a useful life of more than 1 year, like buildings, equipment, or software. The value of assets acquired or improved through capital expenditures is recorded on a company’s balance sheet. Expenditures for long-lived assets used in a company’s R&D operations are not included in its R&D expense, but any depreciation recorded for those assets would be included in its R&D expense. Data are collected in the BERD Survey for capital expenditures for R&D operations for land acquisition, structures, equipment, capitalized software, and other items.

Core-based statistical area (CBSA). A CBSA is a U.S. geographic area that consists of one or more counties (or equivalents) anchored by an urban center of at least 10,000 people plus adjacent counties that are socioeconomically tied to the urban center by commuting. CBSAs are defined by OMB in the Executive Office of the President.

Employment, total and R&D. Involves the number of people employed by R&D-performing or R&D-funding companies in all locations, both foreign and domestic, during the pay period that included 12 March of the survey year. (The date 12 March is what most employers use when paying first-quarter employment taxes to IRS.) R&D employees are those who provide direct support to R&D, such as researchers, R&D managers, technicians, clerical staff, and others assigned to R&D groups. Those not included are employees who provide indirect support to R&D, such as corporate personnel, security guards, and cafeteria workers. In addition to providing head counts of total and R&D employees, the BERD Survey also produces estimates of FTE domestic R&D employment. This is the number of persons employed who were assigned full time to R&D, plus a prorated number of employees who worked on R&D only part of the time.

Employment, leased and temporary. The number of people who work for R&D-performing or R&D-funding companies, but who are not considered employees of the reporting company. These workers perform tasks similar to the reporting companies’ own employees but are technically employed by another company (such as a temp or staffing agency or a consulting firm) or are independent on-site consultants.

Expense and R&D expense. Involves money spent or costs incurred in an organization’s efforts to generate revenue, representing the cost of doing business. Expenses may be in the form of actual cash payments (such as wages and salaries), a computed expired portion of an asset (depreciation), or an amount taken out of earnings (such as bad debts). Expenses are summarized and charged in the income statement as deductions from the income before assessing income tax. Whereas all expenses are costs, not all costs are expenses (e.g., costs incurred in acquisition of income-generating assets—see the definition of capital expenditure above). R&D expense is the cost of R&D funded by the company itself and performed within the respondent company’s facilities, both foreign and domestic, or performed by others outside of the company under contract, subcontract, grant, or other funding arrangement.

R&D and business R&D. R&D is planned, creative work aimed at discovering new knowledge or devising new applications of available knowledge. This includes (1) activities aimed at acquiring new knowledge or understanding without specific immediate commercial applications or uses (basic research), (2) activities aimed at solving a specific problem or meeting a specific commercial objective (applied research), and (3) systematic use of research and practical experience and resulting in additional knowledge, which is directed to producing new or improved goods, services, or processes (development). R&D includes both direct costs, such as salaries of researchers, and administrative and overhead costs clearly associated with the company’s R&D. However, R&D does not include expenditures for routine product testing, quality control, and technical services unless they are an integral part of an R&D project. R&D also does not include market research; efficiency surveys or management studies; literary, artistic, or historical projects, such as films, music, or books and other publications; and prospecting or exploration for natural resources.

R&D, artificial intelligence (AI). AI is a branch of computer science and engineering devoted to making machines intelligent. Intelligence is that quality that enables an entity to perceive, analyze, determine response, and act appropriately in its environment.

Systems with AI perform functions including, but not limited to, speech recognition, machine vision, or machine learning:

  • Speech recognition transforms human speech into a format useful for computer applications (e.g., a digital assistant).
  • Machine vision uses sensors and software that allow images to be used as an input for computer applications (e.g., systems that sort or inspect objects or support navigation in mobile equipment).
  • Machine learning uses statistical software and data to “learn” and make better predictions without reprogramming (e.g., recommender systems for websites, or sales and demand forecasting).

AI technologies also include virtual agents, deep learning platforms, decision management systems, biometrics, text analytics, and natural language generation and processing.

R&D, biotechnology. Biotechnology is the application of science and technology (S&T) to living organisms, as well as parts, products, and models thereof, to alter living or nonliving materials for the production of knowledge, goods, and services. The following list provides examples of areas of biotechnology in which R&D may be performed.

  • DNA or RNA: genomics; pharmacogenomics; gene probes; genetic engineering; DNA or RNA sequencing, synthesis, or amplification; gene expression profiling; and use of antisense technology, large-scale DNA synthesis, genome editing and gene editing, and gene drive
  • Proteins and other molecules: sequencing, synthesis, or engineering of proteins and peptides (including large-molecule hormones); improved delivery methods for large molecule drugs; proteomics; protein isolation and purification; signaling; and identification of cell receptors
  • Cell and tissue culture and engineering: cell or tissue culture, tissue engineering (including tissue scaffolds and biomedical engineering), cellular fusion, vaccine or immune stimulants, embryo manipulation, marker-assisted breeding technologies, and metabolic engineering
  • Process biotechnology techniques: fermentation using bioreactors, biorefining, bioprocessing, bioleaching, biopulping, biobleaching, biodesulfurization, bioremediation, biosensing, biofiltration and phytoremediation, and molecular aquaculture
  • Gene and RNA vectors: gene therapy and viral vectors
  • Bioinformatics: construction of databases on genomes, protein sequences, and modeling complex biological processes, including systems biology
  • Nanobiotechnology: application of the tools and processes of nano- or microfabrication to build devices for studying biosystems and applications in, for example, drug delivery or diagnostics

R&D, domestic. R&D performed in the 50 U.S. states and the District of Columbia. Adjusted domestic R&D is calculated in tables designed to enable comparisons between U.S. BERD Survey statistics and those of other nations. There are three types of tables corresponding to the three principal measures of domestic R&D in the BERD Survey—that is, R&D paid for by the respondent company and others outside of the company and performed by the respondent company, R&D paid for and performed by the respondent company, and R&D paid for by others and performed by the company. In each table, an internationally comparable adjusted domestic R&D estimate is calculated by subtracting depreciation relating to R&D operations, which is included in U.S. BERD Survey domestic R&D, and adding capital expenditures relating to R&D operations, which is excluded from U.S. BERD Survey domestic R&D. Also shown in each table are the differences between the U.S. BERD Survey estimates in this report and the adjusted internationally comparable estimates for domestic R&D.

R&D intensity, or R&D-to-sales ratio. R&D intensity is a ratio, expressed as a percentage, calculated by dividing the cost of R&D by sales. The ratio serves as an indicator of the relative incidence of R&D among groups of companies, primarily among industries and company size classifications. In this report, R&D intensity ratios are calculated in various ways, depending on the measure of R&D (R&D paid for by the respondent company, paid for by others outside of the company, or both) and whether the respondent company performs and funds or only performs R&D.

R&D, nanotechnology. The understanding of processes and phenomena and the application of S&T to organisms and to organic and inorganic materials—as well as parts, products, and models thereof—at the nanometer scale (but not exclusively below 100 nanometers) in one or more dimensions, where the onset of size-dependent phenomena usually enables novel applications. These applications utilize the properties of nanoscale material that differ from the properties of individual atoms, molecules, and bulk matter for the production of knowledge, goods, and services, like improved materials, devices, and systems that exploit these new properties. The following list provides examples of areas of nanotechnology in which R&D may be performed.

  • Nanomaterial: material with any external dimension in the nanoscale or having internal structure or surface structure in the nanoscale
  • Nanoelectronics: field of S&T concerned with the development and production of functional electronic devices with nanoscale components
  • Nanophotonics: branch of photonics concerned with interaction of photons with nanomaterials aiming to design optical or optoelectronic components
  • Nanomedicine: medical application of nanotechnology (e.g., medical application of nanomaterials and biological devices, nanoelectronics biosensors, and possible future applications of molecular nanotechnology, such as biological machines)
  • Nanomagnetics: the study of the magnetic response of nanomaterials to an applied magnetic field and their applications
  • Nanomechanics: a branch of nanoscience studying fundamental mechanical (elastic, thermal, and kinetic) properties of physical systems at the nanometer scale
  • Nanofiltration: a membrane filtration process used for the softening of water and the removal of organic matter; includes nanomembranes
  • Nanotools: multicomponent tools and devices used for manipulation, nanolithography, and nanofabrication
  • Nanoinstruments or nanodevices: multicomponent instruments or devices used for observation, analysis, or control of matter at the nanometer scale
  • Nanomanufacturing: intentional synthesis generation of control of nanomaterials, or fabrication steps in the nanoscale, for commercial purposes

R&D paid for by others, worldwide and domestic. The cost of R&D funded by others outside of the company, including the U.S. federal government, and performed within the respondent company’s facilities, both foreign and domestic.

R&D paid for by the company and others, worldwide and domestic. The cost of R&D funded by the company or by others outside of the company and performed within the respondent company’s facilities, both foreign and domestic, or performed by others outside of the company under contract, subcontract, grant, or other funding arrangement.

R&D performed by the company, worldwide and domestic. The cost of R&D performed within the respondent company’s facilities, both foreign and domestic, funded by the company itself or by others outside of the company.

R&D performed by the company and others, worldwide and domestic. The cost of R&D performed within the respondent company’s facilities, both foreign and domestic, or performed by others outside of the company under contract, subcontract, grant, or other funding arrangement.

R&D performed by others, worldwide and domestic. The cost of R&D funded by the company or by others outside of the company and performed by others outside of the company under contract, subcontract, grant, or other funding arrangement.

R&D, software and Internet. R&D activity in software and Internet applications refers only to activities that have an element of uncertainty and that are intended to close knowledge gaps and meet scientific and technological needs. This item is reported in this survey regardless of the eventual user (internal or external). R&D activity in software includes software development or improvement activities that expand scientific or technological knowledge and construction of new theories and algorithms in the field of computer science. R&D activity in software excludes software development that does not depend on a scientific or technological advance, such as supporting or adapting existing systems, adding functionality to existing application programs, routine debugging of existing systems and software, creating new software based on known methods and applications, converting or translating existing software and software languages, and adapting a product to a specific client, unless knowledge that significantly improved the base program was added in that process.

Sales, worldwide and domestic. Dollar values for goods sold or services rendered by R&D-performing or R&D-funding companies located in the 50 U.S. states and the District of Columbia to customers outside the company, including the U.S. federal government, foreign customers, and the company’s foreign subsidiaries. Included are revenues from a company’s foreign operations and subsidiaries and from discontinued operations. If a respondent company is owned by a foreign parent company, sales to the parent company and to affiliates not owned by the respondent companies are included. Excluded are intracompany transfers, returns, allowances, and freight charges, as well as excise, sales, and other revenue-based taxes.

Data Availability

Publications

The data from the BERD Survey can be found online at https://ncses.nsf.gov/surveys/business-enterprise-research-development/. Detailed historical statistics from the predecessor survey, SIRD, can be obtained from NCSES’s Industrial Research and Development Information System at https://www.nsf.gov/statistics/iris/. Information from the BERD Survey is also included in Science and Engineering Indicators and in National Patterns of R&D Resources.

Electronic Access

The BERD Survey contains confidential data that are protected under Title 13 and Title 26 of the U.S. Code. Restricted microdata can be accessed at the secure Federal Statistical Research Data Centers (FSRDCs) administered by the Census Bureau. FSRDCs are partnerships between federal statistical agencies and leading research institutions. FSRDCs provide secure environments supporting qualified researchers using restricted-access data while protecting respondent confidentiality. Researchers interested in using the microdata can submit a proposal to the Census Bureau, which evaluates proposals based on their benefit to Census, scientific merit, feasibility, and risk of disclosure. To learn more about the FSRDCs and how to apply, please visit https://www.census.gov/about/adrm/fsrdc.html.

Disclosure Avoidance

Disclosure is the release of data that reveals information or permits deduction of information about a particular survey unit through the release of either tables or microdata. Disclosure avoidance is the process used to protect each survey unit’s identity and data from disclosure. Using disclosure avoidance procedures, the Census Bureau modifies or removes the characteristics that put information at risk of disclosure. Although it may appear that a table shows information about a specific survey unit, the Census Bureau has taken steps to disguise or suppress a unit’s data that may be “at risk” of disclosure while making sure the results are still useful.

The BERD Survey uses cell suppression and rounding to avoid disclosing information about a particular company. Cell suppression is a disclosure avoidance technique that protects the confidentiality of individual survey units by withholding cell values from release and replacing the cell value with a data code, usually a “D” or an estimate range. If the suppressed cell value were known, it would allow one to estimate an individual survey unit’s value too closely.

The cells that must be protected are called primary suppressions. To make sure the cell values of the primary suppressions cannot be closely estimated by using other published cell values, additional cells may also be suppressed. These additional suppressed cells are called complementary suppressions. The process of suppression does not usually change the higher-level totals. Values for cells that are not suppressed remain unchanged. Before the Census Bureau releases data, computer programs and analysts ensure that primary and complementary suppressions have been correctly applied.

The Census Bureau has reviewed this data product to ensure appropriate access, use, and disclosure avoidance protection of the confidential source data (Project No. P-7504682, Disclosure Review Board approval number: CBDRB-FY23-0161).

Data

Worldwide R&D paid for by others and performed by the company and others

Domestic R&D paid for by the company and others and performed by the company

Worldwide R&D paid for by the company and performed by the company and others

Domestic R&D paid for by the company and others and performed by the company

The Business Enterprise Research and Development (BERD) Survey is the primary source of information on domestic and global research and development (R&D) expenditures and the R&D workforce for companies operating in the 50 U.S. states and the District of Columbia. The survey is conducted annually by the Census Bureau in accordance with an interagency agreement with the National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation. NCSES has cosponsored an annual business R&D survey since 1953. The Survey of Industrial Research and Development (SIRD) collected data for 1953–2007, and its successor, the Business R&D and Innovation Survey (BRDIS), collected data for 2008–16. Beginning with 2017, the collection of innovation data was moved to the Annual Business Survey, another survey cosponsored with the Census Bureau, and the 2017 and 2018 business R&D data collections reported in this report series were renamed the Business Research and Development Survey (BRDS). Beginning with the data collection for 2019, the survey was renamed for international comparability. BERD Survey questionnaires, reports, and data can be found at https://ncses.nsf.gov/surveys/business-enterprise-research-development/.

The results of the survey are used to assess trends in the performance and funding of R&D. Government agencies, corporations, and research organizations use the data to investigate productivity, formulate tax policy, and compare individual company performance with industry averages. Individual researchers in industry and academia use the data to investigate a variety of topics and to prepare professional papers, dissertations, and books. Total R&D expenditure statistics are used by the Bureau of Economic Analysis for inclusion in its System of National Accounts and Foreign Direct Investment programs.

Further, statistics from the BERD Survey make it possible to evaluate more fully the status of R&D in the United States and to compare the R&D activities of the United States with those of other nations. The usefulness of the information collected in this survey is enhanced by linking it to the Census Bureau’s Longitudinal Establishment and Enterprise Microdata file, which contains information on the outputs and inputs of companies’ manufacturing plants. Response to this survey is mandatory and confidential under Title 13 of the U.S. Code.

For the BERD Survey, data are collected from a probability sample of for-profit companies, which are classified in select manufacturing and nonmanufacturing industries. The BERD Survey is administered both to companies known to have performed R&D and to companies with no known history of R&D activity. The survey is sent to a single coordinator within each company, but it is organized into sections that help the coordinator collect specific types of information from different experts in the company (human resources, accounting, R&D managers, etc.). Foreign-owned companies are instructed to report only for company operations owned by the U.S. subsidiary and, for purposes of the survey, to treat the foreign owners as if they were unrelated third parties.

The target population for the BERD Survey consists of all for-profit companies that have 10 or more paid employees in the United States, that have at least one establishment located in the United States in business during the survey year, and that are classified in certain industries, with a particular focus on those companies that perform R&D in the United States. A company is defined as one or more establishments under common domestic ownership or control.

The Census Bureau’s Business Register contains information on more than 7.3 million establishments with paid employees. It serves as the primary input to the sampling frame from which the sample is selected. For companies with more than one establishment, data are summed to the company level to assign an industry classification code and a measure of size, which are used in designing the sample. Companies are excluded from the frame if they are classified in an industry that is outside the scope of the BERD Survey or if they have fewer than 10 employees, based on their prior year aggregated annual payroll and employment data.

In addition to single-year tables for 2021 (table 1 through table 57, table 66 through table 68), this report includes tables containing statistics for multiple years (table 58 through table 65). In these tables, most prior year statistics have been revised since they were originally published. Revised statistics include adjustments based on information obtained after the original statistics were prepared. For any given year, after the tables are published, the statistical file used to produce the tables stays open for about 1 year before it is permanently closed. During that year, respondents may provide updated data; also, information from other Census Bureau surveys or publicly available sources may become available. The new data and information are evaluated and tested; if appropriate, revisions are made to the statistical file. The multiyear tables in this report were prepared using the statistical files for the 2021 BERD Survey, the revised 2020 BERD Survey, and the revised BRDIS and BRDS statistical files for 2008–19.

Beginning with the tables for the 2017 survey year, disclosure avoidance is now handled differently than it had been in prior years’ tables. Cell suppression is a disclosure avoidance technique that protects the confidentiality of individual survey units by withholding cell values from release and replacing the cell value with a data code. This technique was used in SIRD, BRDIS, and BRDS tables; data in affected cells were replaced with a “D.” For most of the tables in this report, disclosure avoidance is accomplished by replacing cells with an estimate range. Confidentiality is maintained while the usefulness of the statistics is enhanced because most of the completely suppressed cells in past years’ tables have been eliminated. For cases where confidentiality cannot be maintained even with the new estimate range technique, a “D” is substituted.

Terms used in business accounting and incorporated throughout the tables are defined in the section “Technical Notes.” The information and statistics in this report supersede information and statistics in previously published reports.

Acknowledgments

Audrey E. Kindlon of the National Center for Science and Engineering Statistics (NCSES) developed and coordinated this report under the guidance of Amber Levanon Seligson, NCSES Program Director and under the leadership of Emilda B. Rivers, NCSES Director; Christina Freyman, NCSES Deputy Director; and John Finamore, NCSES Chief Statistician.

The Census Bureau, under National Science Foundation interagency agreement number NCSE-2103021, collected and tabulated the data and produced the statistics for this report. This work was performed by Ebenezer Amoako, Lucia Chavez, Melvin Dangan, Robert Ford, David Garrow, Deena Grover, Kristy Harley, Neil Hillis, Bibi Khan, Michael Osman, Dominique Patton, and Jessica White and was supervised by Jeffrey Kellner, Yvette Moore, Susan Shrieves, and Steven Wilkinson under the direction of Michael Flaherty. Under the same interagency agreement, mathematical statistician support was provided by Lucas Streng, Toyeeb Oyolola, and Abigail Legge under the direction of James Hunt and Roberta Kurec; business accounting and subject-matter support was provided by Ron Lee and Brandon Shackelford. RTI International provided data editing services.

Suggested Citation

National Center for Science and Engineering Statistics (NCSES). 2023. Business Enterprise Research and Development: 2021. NSF 23-351. Alexandria, VA: National Science Foundation. Available at https://ncses.nsf.gov/surveys/business-enterprise-research-development/2021.

Analysis

Survey Contact

For additional information about this survey or the methodology, contact

Arthur Hughes
Survey Manager
Phone
(703) 292-2639
E-mail
arhughes@nsf.gov
Address
National Center for Science and Engineering Statistics, 2415 Eisenhower Avenue, Suite W14200, Alexandria, VA 22314