Technical notes and list of tables
Note: In order to expedite dissemination of the data, these technical notes are
being presented before final copy editing. The text of the printed report may
vary slightly from that shown here and in the accompanying WordPerfect file.
Contents
Table Page
General Notes .......................................... vii
Section A. Detailed Statistical Tables ................... 1
B. Technical Notes.................................... 85
Introduction....................................... 85
Survey Methodology................................. 87
Overview......................................... 87
Frame Creation................................... 87
Probability Proportionate to Size................ 91
Sample Allocation and Relative Standard Error
Constraints................................................ 93
Sample Selection................................. 93
The Annual Panel................................. 97
Follow-Up for Survey Nonresponse................. 97
Imputation for Item Nonresponse.................. 97
Response Rates and Mandatory Versus Voluntary
Reporting ................................................ 109
Comparability of Statistics....................... 113
Two-year Comparability.......................... 113
Historical Revisions............................ 113
Survey Definitions................................ 117
C. Survey Documents.................................. 119
The Survey of Research and Development in Industry
provides national estimates of the total expenditures on
research and development (R&D) performed within the United
States by industrial firms, whether U.S. or foreign owned.
It is a sample survey in which all R&D-performing companies,
including privately held firms, are intended to be included
or represented. All companies that are identified in the
sample year as spending more than $1 million annually on R&D
in the United States receive a survey form every year.
Information from individual companies in the sample is used
to develop national estimates on an industry-by-industry
basis. The industry R&D survey does not cover trade
associations. Although their primary mission is to serve
industry, these associations are established as nonprofit
organizations, and it is estimated that their R&D
expenditures are less than 1 percent of the industry R&D
total.
The survey is enterprise based rather than establishment
based. Therefore, all R&D data for each enterprise are
reported in the major Standard Industrial Classification
(SIC) code of the firm. Data are aggregated in this manner
for all tables in this report.
For surveys up to and including the 1991 survey, data are
based on probability samples selected about every 5 years.
In intervening years a subset of the last sample, called a
"panel," is used. The most recent probability sample prior
to the 1992 survey, was selected and first used for survey
year 1987.
The Bureau of the Census (Census), Department of Commerce,
has conducted the annual Survey of Industrial Research and
Development for the National Science Foundation (NSF) since
1957. Census staff conduct the survey under title 13 of the
U.S. Code, which prohibits publication or release of data
that may reveal information about individual companies.
Therefore, on some tables of this report, the symbol "(D)"
is used as a footnote reference to indicate that data are
being withheld to avoid possible disclosure of information
about operations of individual companies.
For detailed discussions of the history of the survey,
survey methodology, comparability of data, and survey
definitions, see section B. For more information regarding
the survey, please contact Raymond Wolfe or John Gawalt:
National Science Foundation
Division of Science Resources Studies
4201 Wilson Boulevard, Suite 965
Arlington, VA 22230
Wolfe: (703) 306-1772, ext. 6936 (voice
phone) rwolfe@nsf.gov (Internet)
Gawalt: (703) 306-1772, ext. 6940 (voice
phone) jgawalt@nsf.gov (Internet)
NOTE
To obtain accurate historical data, use only the latest
detailed statistical tables and not data published earlier.
Data in trend tables are assembled from the most recently
completed survey cycle. Data for prior years are reviewed
for consistency with current-year responses and are revised
when necessary. For that reason, references to prior-year
data should be restricted to those published in this
document.
Classification of Reporting Units
The basic reporting unit is the company or corporate
family that includes all establishments under common
ownership or control. All research and development (R&D)
expenditures and all scientists and engineers of each
company are classified into a single standard industrial
classification (SIC) code and size category. An exception
is made for most of the companies classified in SIC code 13
(oil and gas extraction) and SIC code 50 (wholesale trade,
durable goods). Because the R&D activity of these companies
is in manufacturing product areas rather than in service
areas, their R&D expenditures are included with the SIC
classifications of the manufacturing codes that encompassed
the highest dollar value of these companies' payroll. The
R&D expenditures of all such firms in SIC 13 are included
with those of the firms in SIC 29 (petroleum). The R&D
expenditures of these types of firms in SIC 50 are allocated
to several manufacturing SIC codes.
Company Size-Class
Companies are categorized by their total number of
domestic employees. The following are the six company
size-classes used in this report:
1. Fewer than 500 employees;
2. 500 to 999 employees;
3. 1,000 to 4,999 employees;
4. 5,000 to 9,999 employees;
5. 10,000 to 24,999 employees; and
6. 25,000 or more employees.
Current/Constant Dollars
Statistics in all tables are reported in terms of current
dollars. Constant dollars are also presented in two summary
trend table, tables A-1 and A-25.
Geographic Statistics
The statistics cover only those operations in the 50
States and the District of Columbia. Company-sponsored R&D
performed outside the United States by foreign subsidiaries
of U.S. domestic companies is reported in table A-11 but is
excluded from all other tables.
Industry Classification: Census assigned an enterprise-
level SIC code to each company. A single SIC code was
assigned to multiestablishment companies based on the code
that represented the most dominant aggregated activity for
that firm in terms of total payroll. Data for the following
industry groupings are published in this report (SIC code(s)
are shown in parentheses):1
Transportation equipment
(37)
Textiles and apparel (22, 23) Motor vehicles and
motor vehicles equip. (371)
Lumber, wood products, and furniture (24, 25)
Other transportation equipment (373-75, 379)
Paper and allied products (26) Aircraft and
missiles (372, 376)3
Chemicals and allied products (28) Professional
and scientific instruments (38)
Industrial chemicals (281-82, 286)
Scientific & mechanical measuring instru. (381-82)
Drugs and medicines (283) Optical, surgical,
photo., & other instru. (383-87)
Other chemicals (284-85, 287-89)
Petroleum refining and extraction (13, 29)
Rubber products (30)
Stone, clay, and glass products (32)
Primary metals (33)
Ferrous metals and products (331-32, 3398-99)
Nonferrous metals and products (333-36)
Fabricated metal products (34)
Other manufacturing industries--printing and publishing
(27), leather products (31), and miscellaneous
manufacturing industries (39)2
Nonmanufacturing industries--mining (10-11, 14);
construction (15-17); transportation, communica-
tions, electric, gas, and sanitary services (40-42, 44-
49); wholesale and retail trade (50-51, 53-54, 56);
finance, insurance, and real estate (60, 62-63);
personal and business services (72-73); motion
pictures (78); health services (806-07); and engineer-
ing, accounting, research, management, and related
services (87)
Percentages:
Percentages were calculated on the basis of thousands
of dollars and may differ from those based on the rounded
figures shown.
Rounding:
Because of rounding, details may not add to totals
shown.
Suppression of Statistics:
Census conducts the survey under title 13 of the U.S.
Code, which prohibits publication or release of data that
may reveal information about individual companies. Also,
missing data are imputed for some data items. Therefore,
data in some of the table cells may have been deleted and
replaced with one of the following notations:
"(D)," which indicates that data are being withheld to
avoid possible disclosure of information about operations
of individual companies. This suppression occurs when a
small number of companies, usually one or two, account
for a large percentage of the R&D funds or of scientists
and engineers in a particular data cell. The tables most
often affected by this rule are those that contain data
on Federal support to companies for R&D performance.
"(S)," which indicates that the imputation rate-the
percentage of the statistic not reported by respondents
and consequently estimated by Census-exceeds 50 percent
for that item. See table B-3 in Section B for imputation
rates for specific items.
Although publication of data may be withheld from certain
cells, these data are always included in totals. In some
instances data withheld because of high imputation rates
(footnote reference "(S)") can be derived by subtraction
from higher level totals. In such cases the user should be
aware that the derived numbers are statistically unreliable.
In no instance can data be derived that would disclose
operations of individual companies (footnote "(D)").
List of Tables
Table Page
Total Funds for Research and Development
A-1. Trends in industrial R&D performance, by
source of funds: 1953-91 and
estimates for 1992-93... 9
A-2. Selected data for R&D-performing companies,
by industry: 1990-91... 10
A-3. Total (company, Federal, and other) funds
for industrial R&D performance,
by industry and size of company: 1981-91... 12
A-4. Total (company, Federal, and other) funds
for industrial R&D performance,
by industry and size of company: 1991... 14
A-5. Total (company, Federal, and other) funds
for industrial R&D performance,
by industry, size of company, and size of
R&D program: 1991... 15
A-6. Number of R&D-performing companies, by
industry, size of company, and
size of R&D program: 1991... 16
Company and Other (Except Federal) Funds for
Research and Development
A-7. Company and other (except Federal) funds for
industrial R&D performance,
by industry and size of company: 1981-91... 17
A-8. Company and other (except Federal) funds for
industrial R&D performance,
by industry and size of company: 1991... 19
A-9. Company-financed R&D contracted to outside
organizations, by industry and
size of company: 1991... 20
A-10.Number of companies contracting R&D
performance to outside organizations
and total number of R&D-performing
companies, by industry and size of
company: 1991... 21
A-11.Company-financed R&D performed outside the
United States by U.S. domestic
companies and their foreign subsidiaries, by
selected industry: 1981-91... 22
Federal Funds for Research and Development
A-12. Federal funds for industrial R&D
performance, by industry and size of company:
1981-91 .... 23
A-13. Federal funds for industrial R&D
performance, by industry and size of company:
1991... 25
A-14. Federal funds for industrial R&D
performance, by agency and selected industry in
selected years: 1974-91 ... 26
A-15. Number of R&D-performing companies reporting
Federal R&D funds and total
number of R&D -performing companies, by
industry and size of company: 1991... 27
A-16. Industry-administered federally funded R&D
centers (FFRDCs)-- funds by
character of work, full-time-equivalent
(FTE) R&D scientists and engineers,
and total number of employees: 1989-91... 28
R&D Funds by Size of Company and
as a Percent of Net Sales
A-17. Number of R&D-performing companies in
manufacturing and nonmanufacturing
industries, by size of company: 1991... 29
A-18. Concentration of total, Federal, and company
and other R&D funds, and net
sales of R&D-performing companies by size of
R&D program: 1981-91... 30
A-19. Domestic net sales of R&D-performing
companies, by industry and size of
company 1990-91... 31
A-20. Total (company, Federal, and other) R&D
funds as a percent of net sales in
R&D-performing manufacturing companies, by
industry and size of company:
1981-91... 33
A-21. Company and other (except Federal) R&D funds
as a percent of net sales in
R&D-performing manufacturing companies, by
industry and size of company:
1981-91... 35
A-22. Total (company, Federal, and other) R&D
funds as a percent of net sales in
R&D-performing manufacturing companies, by
industry and ranked by size of
R&D program: 1991... 36
A-23. Company and other (except Federal) R&D funds
as a percent of net sales in
R&D-performing manufacturing companies, by
industry and ranked by size of
company-financed program: 1991... 37
A-24. Federal R&D funds as a percent of net sales
in R&D-performing manufacturing
companies, by industry and ranked by size of
Federal R&D program: 1991... 38
Funds for Basic Research, Applied Research,
and Development
A-25. Total (company, Federal, and other) funds
for performance of basic research,
applied research, and development, in
current and in constant dollars: 1953-91... 39
A-26. Funds for performance of basic research,
applied research, and development, by
industry and source of funds: 1989... 40
A-27. Funds for performance of basic research,
applied research, and development,
by industry and source of funds: 1990... 42
A-28. Funds for performance of basic research,
applied research, and development,
by industry and source of funds: 1991... 44
A-29. Number of R&D-performing companies
conducting basic research, by
industry and size of company: 1991... 46
A-30. Funds for industrial applied R&D
performance, by product field and source
of funds: 1991... 47
A-31. Total (company, Federal, and other) funds
for industrial applied R&D
performance, by industry and product field:
1991... 48
A-32. Total (company, Federal, and other) funds
for industrial applied R&D
performance, by industry and secondary
product field: 1991... 50
Energy and Pollution Abatement Research
and Development
A-33. Total (company, Federal, and other) funds
for industrial energy R&D
performance, by industry: 1981-91 and
projected 1992... 55
A-34. Company and other (except Federal) funds for
industrial energy R&D
performance, by selected industry: 1981-91
and projected 1992... 56
A-35. Federal funds for industrial energy R&D
performance, by selected industry:
1981-91 and projected 1992... 57
A-36. Total (company, Federal, and other) funds
for industrial energy R&D
performance, by primary energy source:
1981-91 and projected 1992 ... 58
A-37. Company and other (except Federal) funds for
industrial energy R&D
performance, by primary energy source:
1981-91 and projected 1992 ... 59
A-38. Federal funds for industrial energy R&D
performance, by primary energy
source: 1981-91 and projected 1992... 60
A-39. Number of companies performing energy
research and development, by industry
and size of company: 1991... 61
A-40. Total (company, Federal, and other) funds
for industrial pollution-abatement
R&D performance and
total number of R&D performing companies, by industry:
1981-91 and projected 1992 ... 62
A-41. Company and other (except Federal) funds for
industrial pollution-abatement
R&D performance, by
selected industry: 1981-91 and projected 1992 ... 63
A-42. Federal funds for industrial
pollution-abatement R&D performance, by selected
industry: 1981-91 and projected 1992... 64
A-43. Total (company, Federal, and other) funds
for industrial pollution-abatement R&D
performance, by type of pollution: 1981-91
and projected 1992 ... 65
A-44. Total (company, Federal, and other) funds
for industrial pollution-abatement R&D
performance, by type of pollution and
source of funds: 1990-91 and projected
1992... 66
Geographic Distribution and Distribution by Type
of Cost
A-45. Geographic distribution of total (company,
Federal, and other) funds for industrial
R&D performance in selected years: 1974-91... 67
A-46. Geographic distribution of total (company,
Federal, and other) funds for industrial
R&D performance, by source of funds: 1991... 69
A-47. Distribution of total (company, Federal, and
other) funds for industrial R&D
performance, by industry and type of cost:
1991... 70
Employment
A-48. Number of full-time-equivalent (FTE) R&D
scientists and engineers in R&D-
performing companies, by industry and size
of company: 1982-92... 71
A-49. Number of full-time-equivalent (FTE) R&D
scientists and engineers in R&D-
performing companies, by industry and source
of R&D funds: January 1992... 73
A-50. Cost per R&D scientist or engineer in R&D-
performing companies, by industry
and size of company: 1981-91... 74
A-51. Cost per R&D scientist or engineer, by
industry and size of company: 1991... 76
A-52. Cost per R&D scientist or engineer in
companies ranked by size of R&D
program: 1981-91... 77
A-53. Domestic employment of R&D-performing
companies, by industry and size of
company: 1990-91... 78
A-54. R&D scientists and engineers per 1,000
employees in manufacturing companies,
by industry and size of company: 1981-91... 80
A-55. R&D funds per employee, by size of company:
1981-91... 82
A-56. Concentration of total employment in
R&D-performing companies ranked by
size of R&D program: 1985-91... 83
B-1. Number of companies in the universe, sample, and 1991 panel...
B-2. Standard error of estimate (percentage) of total (company, Federal,
and other) fundsfor industrial R&D performance for all company size groups
and for companies with fewer than 1,000 employees, by industry: 1991...
B-3. Imputation rates for selected items, by industry: 1991...
B-4. Funds for performance of basic research, applied research, and
development, by industry and source of funds: 1989...
B-5. Funds for performance of basic research, applied research, and
development, by industry and source of funds: 1990 ...
B-6. Funds for performance of basic research, applied research, and
development by industry and source of funds: 1991...
B-7. Unit response rates: Number of companies responding to survey,
by industry: 1991...
B-8. Item response rates: 1991...
Section B. Technical Notes
Introduction
The Survey of Research and Development in Industry
provides national estimates of the total expenditures on
research and development (R&D) performed within the United
States by industrial firms, whether U.S. or foreign owned.
It is a sample survey in which all R&D-performing companies,
including privately held firms, are intended to be included
or represented. All companies that are identified in the
sample year as spending more than $1 million annually on R&D
in the United States receive a survey form every year.
Information from individual companies in the sample is used
to develop national estimates on an industry-by-industry
basis.
The National Science Foundation (NSF) has been sponsoring
a survey of industrial R&D since 1953. The 1991 Survey of
Research and Development in Industry is the 35th in the
annual series sponsored by NSF and conducted since 1957 by
the Bureau of the Census. NSF's Division of Science
Resources Studies monitors the survey. NSF also sponsored
two industrial R&D surveys covering the 1953-56 period that
were conducted by the U.S. Department of Labor, Bureau of
Labor Statistics (BLS).1 Data obtained in the earlier BLS
surveys are not directly comparable with Census figures for
1957-91, because of methodological and other differences.
Respondents receive detailed definitions to help them
determine which expenses to include or exclude from the R&D
data they provide. Nevertheless, the statistics presented
in this report are subject to response and concept errors
caused by different respondent interpretations of the
definitions of R&D activities and by variations in company
accounting procedures. Consequently, the data are better
indicators of changes in, rather than absolute levels of,
R&D spending and personnel. Data quality has improved
substantially since the early surveys were conducted, mainly
because respondents have adopted more accurate and
sophisticated accounting procedures over the years.
The survey's primary focus is on U.S. industry as a
performer of, rather than as a source of funds for, R&D.
Thus, data on Federal support of R&D activities performed by
industry are collected and appear in several tables, but
data on industrial funding of R&D undertaken at universities
and colleges and other nonprofit organizations are not
included in the major tables.2
The survey data provide (1) national estimates of total
R&D performed by industry in the United States; (2) the
portion of the effort that is financed by U.S. Government
funds; and (3) the amount financed by the companies
themselves or by other non-Federal sources such as State and
local governments or other industrial firms. Also included
are data on both the number of employees and the number of
R&D-performing scientists and engineers at the firm, as well
as on the domestic net sales for the company, and on the
total funds for R&D financed by the domestic firm but
performed outside the United States. The tables developed
from the survey data are in section A, and copies of the
1991 survey questionnaire, instructions, and other survey
documents are reproduced in section C.
The scope of the survey has been expanded and refined over
the years in response to an increasing policy need for more
detailed information on the nation's R&D effort. For
example, questions on energy R&D were added in the early
seventies, following the first oil-shortage crisis. On the
other hand, the frequency of collection of certain data
items has been reduced in an attempt to alleviate some of
the respondent burden that has been placed on industry from
all sources in recent years. Since the 1978 survey, a
detailed questionnaire, Form RD-1, has been used only to
collect data for odd-numbered years and an abbreviated
version, containing only the most crucial data elements, has
been used to collect data for the intervening, even-numbered
years. Questions appearing only on the long form, request
detail on: R&D by product field; company expenditures for
R&D projects that were contracted to outside organizations,
rather than performed in-house; Federal R&D support to the
firm, by contracting agency; R&D expenditures by geographic
area; and some detailed data on energy and pollution-
abatement R&D activities. This report provides data
collected from the long form.
Survey Methodology3
Overview
Data in this survey are based on a sample of industrial
firms, selected approximately every 5 years (e.g., 1976,
1981, 1987). In intervening years only a subset of the
sample, or panel, receives annual survey forms and Census
makes estimates for the changes in R&D for the firms not
canvassed annually. The sampling unit for this survey is
the enterprise, or company, defined as a business
organization consisting of one or more establishments under
common ownership or control. The 1991 data in this
publication are based on responses from a panel derived from
the sample selected and first used for survey year 1987
(table B-1).
The Standard Statistical Establishment List (SSEL), a
Census compilation that contains information on 3.5 to 4.0
million establishments, was the universe frame used to
select the 1987 sampling frame. Where necessary for
multiestablishment companies, Census summed establishment-
level data to the company level and then assigned a single
Standard Industrial Classification (SIC) code to that
firm-the SIC code of the establishment(s) having the highest
dollar-value of payroll.4 The frame from which the sample
was drawn includes companies in all manufacturing industries
and, on the basis of earlier samples, a select number of
nonmanufacturing industries known to conduct R&D.
The weight given to an individual company is, in general,
the inverse of the probability that the company would have
been selected for inclusion in the sample. Certainty
companies have a weight of 1.00. The company weight is
retained both in the sampling year and in the years between
samples. To minimize respondent burden, only the panel
subset of the sample is canvassed between sample years.
Most small firms are not re-contacted, but, in each
succeeding year, Census estimates the data for each
noncanvassed firm on the basis of the changes in the initial
value of the R&D reported by the firm and the average growth
rate for the firm's industry.
Frame Creation
In selecting a frame from which to draw a sample of R&D-
performing industries, NSF staff, given a finite budget for
sample selection, make certain assumptions about industries
to maximize sampling efficiency. That is, they apply a
priori knowledge of industrial R&D activity to eliminate
some industries from the possibility of selection by the
initial SSEL sort and to designate other companies for
sampling with 100-percent certainty. In addition, NSF staff
made several innovations for the sample drawn in 1987 to
improve its quality.
As in previous sample years, all companies that had been
on the previous panel received a survey form. In addition,
Census staff reviewed lists of R&D contractors published by
the Department of Defense (DoD) and the National Aeronautics
and Space Administration (NASA) to ensure that all their
large industrial R&D-performing contractors were included in
the sample with certainty.
From the outset, in the latest sample selection the major
goal was to eliminate from the sampling frame to the
greatest extent possible companies unlikely to have R&D
programs. These companies were eliminated to minimize the
number of sample companies chosen thathad no R&D activity.
To accomplish this objective, two steps were taken:
1. NSF staff narrowed the list of nonmanufacturing
industries considered in-scope by eliminating those likely
to engage in little or no R&D activity. Thus, companies in
the eliminated nonmanufacturing industries had no chance of
being selected. This method gave companies in the
manufacturing industries and the remaining nonmanufacturing
industries a greater probability of selection than they had
in the past.
2. Companies with more than 500 employees in the in-scope
industries were sampled with certainty. Companies with less
than 500 employees were subjected to or eliminated from
sampling on the basis of varying employment size level
cutoffs. An assumption was made that in some industries
companies with only a small number of employees are unlikely
to engage in R&D activity, and an employment cutoff was set
for each industry group. Generally, the cutoff was 250
employees with some exceptions (e.g., the cutoff for
hospitals was 1,000 employees). Those companies falling
below the cutoff were eliminated from the universe frame.
To improve coverage of R&D-performing companies, NSF staff
in 1987 provided to Census the names of firms, identified
through media reports or other sources as R&D performers,
that were to be included in the sample with certainty. Most
of these companies, because they met other established
criteria, would have received questionnaires anyway, but a
few were added to the sample by this effort. All certainty
companies-those on lists provided by NSF staff, on lists of
DoD and NASA contractors, companies with more than 500
employees, and previous panel members-are self representing,
i.e., they have sampling weights of unity.
Based on (1) SIC code, (2) total employment cutoffs, (3)
inclusion on an NSF, DoD, or NASA list, or (4) previous
panel membership, approximately 154,000 companies were
identified as in scope of the survey and, therefore were
included in the sampling frame. The new efforts to improve
targeted coverage resulted in a sharp reduction in the size
of the total in-scope universe from about 450,000 companies
in 1981.
It is likely that a small number of companies actually
engaged in R&D activity were omitted from the sampling frame
as a result of these sample selection operations. It was
agreed, however, that the benefit from the new
operations-greater sampling efficiency, resulting in
improved national estimates of industrial R&D expenditures
and employment-far outweighed the cost of the loss of a few
companies that may have been eliminated from the in-scope
sampling frame.
Probability Proportionate to Size
As with most types of economic surveys, the sample
selection process for the industrial R&D survey used
probabilities proportionate to size. That is, large
companies have a proportionately higher probability of
selection than do small companies, where large or small is
measured relative to the statistic being estimated.
For the R&D survey, it would have been ideal if size could
be determined by the amount of a company's R&D expenditures.
Unfortunately, except for the companies that were in the
current panel, it was impossible for Census to know the R&D
expenditure values for firms in the universe frame. One
logical solution was to estimate each company's R&D
expenditures and base the probability of selection on the
estimated values. This strategy had been employed in the
1981 sampling operation.
Clearly, this strategy has weaknesses. Even with the
reduced number of in-scope industries in the sampling frame,
many companies chosen for the frame may not have engaged in
any R&D activity. Nevertheless, the procedure used to
estimate the size of companies was forced to treat all
companies as if they did in fact perform R&D.
Census estimated the size of each company's R&D
expenditures by using a relationship linking the size of
each company's employment to its amount of R&D expenditures.
[Since company employment was known for the universe, it was
possible to use this relationship to estimate R&D
expenditures values for all companies in the frame. Census
derived this relationship for each SIC classification
category, using data collected in the then most recent
(1985) R&D survey]. Rather than treating all companies
equally, the larger the number of employees in a company,
the higher the probability of selection for inclusion in the
sample. It was deemed reasonable to assume that large
companies were more likely to have R&D programs than were
small companies.
One further adjustment was introduced to the sample
selection process. This was based on the assumption that
multiestablishment companies, on average, would be expected
to perform more R&D than single-establishment companies of
the same size and in the same industry. The 1985 panel data
were used to develop this adjustment factor. It should be
noted that, for companies that were in the previous panel,
the actual reported R&D activity was used and the data were
not adjusted.
Sample Allocation and Relative Standard Error Constraints
The sampling program utilized for this operation allowed
parameters to be assigned permitting the sample to be
allocated across various levels, or strata, that correspond
to industry groupings. This procedure permitted a desired
sample size, or a desired sampling error, to be achieved for
each stratum. Estimated errors of total R&D estimates for
these strata were not to exceed certain levels. Since the
amount of funds provided by NSF determined the size of the
sample to be drawn, the only constraint in achieving these
results was that the total sample size across all the strata
could not exceed 12,000-13,000 companies. NSF staff
provided relative rankings for each industry group-high,
medium, or low-to determine the precision of the estimate.
An actual translation to what high, medium, or low meant,
specifically, could not be determined until Census staff
arbitrarily investigated several sampling error levels,
computed the sample size that these levels implied, and
applied the constraint of the total sample size of 13,000.
The result of this investigation led to the following
criteria for the target sampling error of estimate of funds
for R&D performance:
a. High precision: sampling error not to
exceed 2 percent
b. Medium precision: sampling error not to
exceed 5 percent
c. Low precision: sampling error not to
exceed 10 percent
Based on the desired precision, these criteria suggested a
total sample size of approximately 13,500. This number was
not excessively beyond the stated limit of 13,000, so this
sample size parameter was chosen for the selection process.
One limitation should be noted. Sampling errors were
controlled by using a universe total that, in large part,
was improvised; that is, and as previously noted, Census
assigned an R&D value to every company in the frame, even
though many of these companies may not actually have had R&D
expenditures. The value assigned was imputed for the great
majority of companies in the frame, and, as a consequence,
the estimated universe and the distribution of individual
company values did not necessarily reflect reality.
Estimates of sampling variability were nevertheless based on
this distribution. The presumption was-and this had been
confirmed in the previous sample selection--that actual
variation in the sample design would be less than that
estimated, because many of the sampled companies have true
R&D values of zero, not the widely varying values that were
imputed using total employment as a predictor of R&D. Thus,
the 2-percent, 5-percent, and 10-percent error levels
described earlier are conservative. (See table B-2 for a
complete list of the actual standard errors in the 1991
survey.)
The particular sample selected is one of a large number of
the same type and size that, by chance, might have been
selected. Estimates from each of the different samples would
differ somewhat from each other and from the results of a
complete canvass conducted under essentially the same
conditions as the survey. In addition to sampling error the
estimates are subject to nonsampling error that would also
occur if a complete canvass were to be conducted under the
same conditions.
Sample Selection
The sample selection program was run with a specified
expected sample size of 13,500 and with other parameters set
to ensure compliance with the relative standard error
constraints. An actual sample of 13,917 was selected. The
actual sample size differs from the specified for two
reasons. First, the selection program used independent
sampling. Each company had an independent chance of
selection, based on its assigned probability; the selection
of a company was completely independent of the selection of
any other company. In independent sampling, sample size
itself is a random variable. Theoretically, a sample of
size zero or a sample the size of the entire universe is
possible, but the probabilities of these extremes are so
small that these are nearly impossible situations. The
actual sample size is usually quite close to the specified
size. If there is too much deviation, the program is simply
executed again.
Second, a minimum probability rule was imposed. As noted
earlier, the sampling program assigns probabilities
proportionate to size (where size in this case is the
imputed R&D value assigned each company). Selected
companies that have R&D programs vastly larger than their
assigned values can have adverse effects on the estimates
once the data are collected. To lessen these effects, the
maximum weight a company can assume was arbitrarily
controlled by specifying that the probability of selection
cannot be less than a certain value. If the probability,
based on its size, is less than this minimum value, then it
is set equal to this value. The consequence of raising
these original probabilities to the minimum probability is
to raise the expected sample size. It is likely that most
of the difference between the specified sample size and the
actual sample size is due to this rule.
The Annual Panel
The panel is a group of companies that receive a survey
questionnaire, form RD-1, annually. The following is a
description of how the present panel was formed from the new
sample in 1987.
The basic tool for the survey is form RD-1, which is used
to collect detailed R&D information. The 1,095 companies
that were in the old panel and had received a 1986 RD-1 form
received an RD-1 form for 1987. The remaining certainty
companies (6,903) and other companies (5,919) in the new
sample received an RD-1A survey form for 1987. Form RD-1A
is an abbreviated version of RD-1 and generally is mailed to
companies only in the year in which a new sample is drawn.
The purpose is to canvass smaller R&D performers but to
impose a minimum of reporting burden on them.
Of the 13,917 companies that received a form in 1987,
3,648 respondents reported that their companies were R&D
performers or were identified as R&D performers based on
information from other sources. The 3,648 companies were
ranked by total R&D funds (both Federal and company and
other) within each SIC code. All companies with over $1
million in total R&D expenditures were placed on the RD-1
panel. In some industries Census added to the panel
companies with less than $1 million in R&D expenditures in
an attempt to ensure 95-percent coverage of the R&D total
for each industry.
The result of this process was an increase of panel size
from 1,095 companies based on the previous sample to 1,742
companies based on the sample drawn in 1987. All companies
selected for the panel will receive the RD-1 questionnaire
annually until the next sample is drawn. The other
companies will not receive another questionnaire; their R&D
data will be estimated, in subsequent years, by Census
staff, using their 1987 reports as a base.
Followup for Survey Nonresponse
The 1991 survey questionnaires were mailed in March 1992,
and recipients were asked to respond within 60 days. Thirty
days later, letters were mailed to all survey recipients,
reminding them that their completed questionnaire was due
within the next 30 days. At 60 days, an initial set of
followup letters was sent to all nonresponding firms. Two
additional followup mailings were made to persistent
nonrespondents, at 90 and 120 days. The 90-day followup
mailing included a replacement questionnaire.
In addition to the followup mailings, telephone followup
was used to encourage response among those nonresponding
firms ranked among the 200 largest R&D performers, based on
1989 expenditures. Telephone followup was also used for
these firms during the initial data edit phase of survey
operations.
Imputation for Item Nonresponse
Even when forms are returned, they are often incomplete.
Many firms choose to return the survey forms with one or
more items blank, for varying reasons-for instance, the
internal accounting procedures of the firm may not allow it
to quantify R&D by pollution-abatement expenditures. In
addition, some firms, as a matter of policy, refuse to
answer any voluntary questions. Four data items on the
questionnaires (total R&D expenditures, Federal R&D funds,
net sales, and total employment), however, are included in
Census' mandatory statistical program. The response rate
for these mandatory items was significantly higher than for
the other items on the survey.
When respondents do not provide the requested information,
Census estimates the missing data for each firm, using an
imputation algorithm. In general, the imputation algorithms
employed by Census compute values for missing items by
applying the average percentage change for the target item
in the nonresponding firm's industry to the item's prior
year value for that firm, reported or imputed. This
approach, with minor variation, is used for most items on
the survey form. Note that all items for nonpanel
firms-that is, firms that are surveyed in the sample year,
but that, because of size, are not surveyed again until a
new sample is selected-are estimated in the nonsample years,
using these imputation processes.
Table B-3 contains imputation rates for the principal
survey items. For a detailed discussion of the problems
stemming from item nonresponse, see the NSF technical
report, Estimating Basic and Applied Research and
Development in Industry: A Preliminary Review of Survey
Procedures, NSF 90-322.
There are two survey items that do not use this general
imputation methodology: (1) geographic distribution of R&D
expenditures and (2) R&D expenditures distributed by basic
research, applied research, and development (character of
work). The imputation method for computing estimates of the
geographic distribution of R&D expenditures was revised in
1989 to include the use of secondary source information as
well as focused telephone followup. The secondary source
used is the Directory of American Research and Technology
1991, R.R. Bowker, New York, which lists the locations of
R&D laboratories for over 6,700 firms in the U.S. R&D
expenditures for nonresponding firms listed in this
directory as having laboratories in only a single State were
assigned to that State. The remaining nonresponding firms
were contacted by telephone and asked to distribute their
reported R&D expenditures. Most were able to provide this
information. This modified approach is used only for
nonresponding firms ranked among the 200 largest R&D
performers. The imputation rates listed in table B-3
reflect gains made using this method.
The imputation procedure used for the basic research,
applied research, and development expenditure estimates was
revised in 1987 for use with 1986 and later data and also
differs from the general imputation approach. Response to
character-of-work items has been declining in recent years
and, as a result, imputation rates have been increasing.
The general imputation procedure became increasingly
dependent upon information that had been imputed in prior
years, thereby distancing current-year distribution
estimates from the time they were reported. In addition, to
reduce the reporting burden of this survey, smaller,
nonpanel firms in the sample have never been asked to
disaggregate their R&D expenditures into character-of-work
categories; consequently, imputation is required for all
such firms.
Because of the increasing dependence on imputed data, NSF
chose not to publish estimates of industrial basic research,
applied research, and development in this report series,
beginning in 1986. In 1987, Census revised its approach for
imputing character-of-work estimates. The revised method
calculates the character-of-work distribution for a
nonresponding firm only if that firm reported a distribution
within a 5-year time period, extending from 2 years before
to 2 years after the year requiring imputation. Imputation
for a given year is initially performed in the year the data
are collected and is based on a character-of-work
distribution reported in either of the 2 previous years, if
any. It is again performed using new data collected in the
next 2 years. Thus, character-of-work estimates are revised
as newly reported information becomes available and are not
final for 2 years following their initial publication. This
revised imputation procedure was first used for the 1986
estimates.
If reported data are not available for a firm, Census does
not impute a character-of-work distribution for that firm.
As a consequence of this restriction, only approximately 70
percent of the total estimated R&D expenditures were
distributed across these categories in the 1986 Census
estimates. Those expenditures not meeting the requirements
of the new imputation methodology were placed in the newly
created "not distributed" category. Tables B-4 through B-6
show the current character-of-work estimates provided by
Census along with the "not distributed" component for 1989,
1990, and 1991.
However, NSF's objective in this survey is to provide
estimates for the entire population of firms performing R&D
in the United States. Because the revised Census imputation
procedure no longer produces such estimates because of the
"not distributed" component, this objective was not being
met for character-of-work estimates. A separate NSF
technical report, Estimating Basic and Applied Research and
Development in Industry: A Preliminary Review of Survey
Procedures, NSF 90-322, presents character-of-work estimates
for 1986, 1987, and 1988 and examines several alternative
approaches to apportioning the "not distributed" data to the
basic research, applied research, and development
categories. The tables in the report illustrate how
estimated figures for basic research, applied research, and
development can vary, depending on assumptions made about
the R&D expenditures that are not distributed. The 1989,
1990, and 1991 estimates of basic research, applied
research, and development provided in section A of this
report were calculated using the baseline estimation method
presented in NSF 90-322. In the baseline estimation method,
the "not distributed" expenditure data are allocated, by
industry group, to basic research, applied research, and
development categories, using the percentage splits in the
distributed category for that industry. The allocation is
done at the lowest level of published industry detail only;
higher levels are derived by aggregation.
Response Rates and Mandatory Versus Voluntary Reporting
Detailed unit and item response rates for the 1991 survey
are shown in tables B-7 and B-8, respectively.
Current survey reporting requirements divide survey items
into two groups: mandatory and voluntary. Response to four
data items on the questionnaires (total R&D expenditures,
Federal R&D funds, net sales, and
total employment) is mandatory, whereas response to the
remaining items is voluntary. During the 1990 survey cycle,
NSF conducted a test of the effect of reporting on a
completely voluntary basis to determine if combining both
mandatory and voluntary items on one questionnaire
influences response rates. For this test the 1990 sample
was divided into two panels of approximately equal size.
One panel, the mandatory panel, was asked to report as usual
(four mandatory items and the remainder voluntary), and the
other panel, the voluntary panel, was asked to report all
items on a completely voluntary basis. The result of the
test was a decrease in the overall survey response rate to
80 percent from levels of 88 percent in 1989 and 89 percent
in 1988. The response rates for the mandatory and voluntary
panels were 89 percent and 69 percent, respectively.
Detailed results of the test were published in Research and
Development in Industry: 1990 (NSF 94-304).
Comparability of Statistics
Several procedures have been undertaken to revise data and
maintain the reliability of the time series. Revisions
affecting the comparability of current-year with most recent
prior year statistics and historical statistics are
explained below.
Two-Year Comparability
Before the survey forms are mailed each year, Census
imprints the data reported by the respondent the previous
year (or 2 years earlier for items asked only in odd-
numbered years). Respondents are asked to adjust data for
the previous year(s), if necessary, to make them comparable
to data provided for the current year. Such adjustments are
necessitated by any of several developments, e.g., changes
in company structure such as mergers, acquisitions, or
divestitures; changes in reporting concepts; and even
correction of any errors that may have been made previously.
Thus, there is comparability in the reporting of survey data
over any 2-year period.
These adjustments can be examined by comparing data for
the same year reported in two succeeding periods, e.g., 1990
data appearing in Research and Development in Industry: 1990
may differ from 1990 data in this volume. Totals for broad
classifications are likely to be very close in the two
editions; differences may be more noticeable in the finer
detail. These differences underscore the point that the
measures are approximate and indicative rather than precise.
Historical Revisions
The published industry R&D survey data are revised
periodically, usually because of changes in the SIC code
classifications of companies. In addition, two other major
data revisions were made in the data for the 1981-87
periods.
Companies may shift from one SIC code into another
because of (1) the growth or decline of product lines; (2)
the merger of two or more companies; (3) the acquisition of
one company by another; (4) divestiture; or (5) the
formation of conglomerates. When Census is aware of the
year in which changes (2), (3), (4), or (5) occurred
(respondents are asked on the questionnaire about changes in
ownership), the firm's data are reclassified into the new
industry for the year the change actually occurred. If a
change is not discovered until a new sample is selected or
if it cannot be determined when a change actually occurred
(i.e., (1)), other methodologies are used to adjust, or
backcast, the data to reflect the move of a company out of
one industry and into another. Since 1967, three revisions
were made in the data, covering the periods 1967-76, 1976-
81, and 1981-87.
1. The 1967-76 Period
The SIC codes assigned to companies in the panel for the
years 1967 through 1975 were based on data reported in the
1967 economic census. The 1974 Standard Statistical
Establishment List file was used to assign SIC codes to
companies in the next panel chosen for the 1976 survey.
These codes were used to revise 1975 data received in the
1976 survey. The SIC codes of companies in the 1967 and
1976 panels were examined to determine which companies had
changed classifications. Since it was not known in which
year changes actually occurred, data for companies that had
changed SIC codes were revised for the years 1968 through
1974, to smooth the changes over the period 1967-76. To
illustrate, if a company was originally in SIC A in 1967 but
was discovered to be in SIC B in 1974, its data for 1967-74
were allocated between the two industries as follows: for
1967, all of the company's data were retained in industry A;
for 1968, 14.3 percent of the company's data were allocated
to industry B, and the remainder retained in industry A; for
1969, 28.6 percent were allocated to industry B, and the
remainder retained in industry A; and so on until 1974, when
all of the company's data were allocated to industry B.
2. The 1976-81 Period
Backcasting revisions in the industry R&D data for 1976-80
were also made to reflect the classification changes
revealed in the new sample drawn in 1981, however, a
different methodology was used for the revisions.
When the new panel was selected in 1981, companies were
assigned SIC codes from Census' 1981 SSEL and the 1981
Enterprise Files. Prior-period (1980) data were collected
in the 1981 survey. These data were presumed to be more
accurate than those collected in the 1980 survey, because
they reflected not only updated SIC code classifications but
were obtained from a larger panel, providing better coverage
of U.S. industry. Data obtained for 1976-80, using the
panel selected in 1976, were revised subject to the
following constraints:
a. Data for 1976, revised from the 1977 survey,
would remain unchanged, since this was the first
year of the 1976 panel and since that panel was an
accurate reflection of company SIC codes in 1976.
b. Data for 1980 collected in the 1981 survey
would be used instead of the 1980 data collected
in the 1980 survey.
An algorithm was used to link data from 1976 with those
collected in 1981, preserving to the greatest extent
possible year-to-year trends in data for each industry by
revising data for the years 1977, 1978, and 1979.
The following data elements were adjusted, using the
methodologies just described: Funds spent on R&D (total,
Federal, company and other, and R&D performed outside the
United States); number of full-time equivalent (FTE) R&D
scientists and engineers; total and company and other R&D
funds as a percent of net sales; and cost per R&D scientist
or engineer. No adjustments were made in other data
elements.
3. The 1981-86 Period
Similar backcasting revisions in the industry R&D data
were made in 1987 for the industry data previously reported
for the years 1981-86. Several other adjustments were made
in 1987 in addition to the original methodology described
for the revisions made in 1981.
When the 1987 sample was selected, companies were assigned
SIC codes from the 1987 SSEL file. Companies in the sample
were asked to submit their prior-year (1986) data along with
the information collected for the 1987 survey. These new
1986 data were used in lieu of the data originally collected
in the 1986 survey because they reflected updated SIC code
classifications of companies and also were collected from a
much larger sample that was designed to provide better
coverage of U.S. industry, particularly the nonmanufacturing
portion. Data obtained for 1981-86 using the panel selected
in 1981 were revised in 1987, subject to the following
constraints:
a. Data for 1981, revised from the 1982 survey,
would remain unchanged, since this was the first
year the 1981 panel was used, and since that panel
was an accurate reflection of company SIC codes in
that year.
b. Data for 1986 collected in the 1987 survey
would be used instead of the 1986 data collected
in the 1986 survey.
There was an increase of $9.3 million between the 1986
data based on the old panel and the 1986 data reported by
the new panel identified in 1987. Most of that aggregate
revision was caused by the addition of new R&D performing
firms identified through the revised sampling procedures in
1987. An algorithm was used to link data from 1981 with
those collected in 1987, to the greatest extent possible
preserving year-to-year trends that had previously been
reported in data for each industry by revising data for the
years 1982, 1983, 1984, and 1985. Interested persons should
contact Census to obtain further information about the
construction and content of the algorithm.
4. Federal Data for 1986
Data for a merger between two companies were incorrectly
processed in the 1988 tabulations, resulting in an overcount
of Federal R&D support for industry in the amount of $356
million in 1986. The corrected 1986 industry R&D total
(company, Federal, and other funds) is $87,823 million, and
the Federal total is $27,891 million. These corrections are
reflected in the detailed statistical tables in this report.
No revisions to 1988 individual industry data that were
published were necessary, because data for the affected
industry were suppressed to avoid possible disclosure of
information about operations of individual companies (i.e.,
these data cells are marked with the symbol "D").
5. Data Reevaluations for 1987
Many of the small firms identified in the 1987 sample were
receiving this kind of survey questionnaire for the first
time, and some were unaccustomed to reporting financial
information rounded to the nearest thousand dollars.
Several of these firms, therefore, mistakenly reported their
expenditures for R&D in dollars, not thousands of dollars as
requested. Consequently, several firms were selected for
the 1987 panel based on an incorrectly reported size of
their R&D program. Many of these firms realized the error
when they received the 1988 form with the preprinted 1987
data. The respondents corrected their 1987 data when they
submitted the 1988 forms.
Furthermore, for 1988 several of these firms reported that
they had no R&D program, even though they had previously
reported expenditures for R&D activities in the 1987 survey.
After verifying by phone that these firms had no R&D
activity, Census staff declared these firms to be out-of-
scope and blanked the erroneous information from the 1987
estimates during processing of the 1988 survey results.
Similar reporting errors were encountered during the data
collection and processing for the 1989 survey. In addition,
review of the 1989 estimates revealed an error in the
revision procedure used for firms discovered to be out-of-
scope in 1988. This error resulted in only a partial
correction for these firms during the 1988 processing.
These errors caused Census to take the unusual step of
revising the 1987 R&D expenditure estimates during 1989
survey processing. Revisions to other than only previous
year data are rarely done, except during sample years when
historical adjustments are made to account for shifts in
industrial classification.
Survey Definitions5
Company (and other) funds for R&D - cost of R&D actually
performed within the company and funded by the company
itself or by other non-Federal sources via contract not
including the cost of R&D supported by companies but
contracted to outside organizations, such as research
institutions, universities and colleges, nonprofit
organizations, or (to avoid double-counting) other
companies. (Since this is a survey of R&D performers,
industrial firms that undertake R&D supported by outside
organizations do report the funds received in payment for
the R&D work they perform. These moneys are classified
under the industries of the performing companies.)
Cost per R&D scientist or engineer - the arithmetic mean of
the numbers of FTE R&D scientists and engineers reported for
January in 2 consecutive years divided into the total R&D
expenditures of the earlier year, with the ratio attributed
to the earlier year. (For example, the mean of the numbers
of FTE R&D scientists and engineers in January 1991 and
January 1992 is divided into total 1991 R&D expenditures for
a total cost per R&D scientist or engineer in 1991.)
Employment, FTE R&D scientists and engineers - persons
engaged in scientific or engineering work at a level that
requires knowledge of physical, life, engineering, or
mathematical science, equivalent, at least, to that acquired
through completion of a 4-year college program with a major
in one of those fields employed by the company during the
January following the survey year. (The statistics in this
report show the full-time equivalent (FTE) employment. The
FTE is the number of scientists and engineers in the company
who are assigned full-time to R&D projects together with the
fraction of total work time spent on R&D projects by non-
full-time R&D scientists and engineers.)
Employment, total - number of persons domestically employed
by R&D-performing companies in all activities during the pay
period that includes the 12th of March.
Federally funded research and development centers (FFRDCs):
R&D-performing organizations administered by industrial,
educational, or other institutions on a nonprofit basis,
exclusively or substantially financed by the Federal
Government. (FFRDCs are supported by the Federal Government
either to meet a particular R&D objective or, in some
instances, to provide major facilities at universities for
research and associated training purposes. R&D expenditures
of the FFRDCs -administered by industry are included with
the Federal R&D data of the industry classification of each
of the administering firms. The FFRDCs are listed below.)
Supported by the Department of Energy:
Bettis Atomic Power Laboratory Knolls Atomic Power
Laboratory
Westinghouse Electric Corp. General Electric Co
West Mifflin, PA Schenectady, NY
.
Hanford Engineering Development Laboratory Oak Ridge
National Laboratory
Westinghouse-Hanford Corp. Martin Marietta Energy
Systems, Inc.
Richland, WA Oak Ridge, TN
Idaho National Engineering Laboratory Sandia National
Laboratories
EG&G Idaho, Inc.; Western Electric
Company, Inc.-Sandia Corp.
Argonne National Laboratory, West; Albuquerque, NM
Rockwell International Corp.;
Westinghouse Electric Corp. Savannah River
Laboratory
Idaho Falls, IA Westinghouse Electric
Corp.
Aiken, SC
Supported by the National Institutes of Health,
Department of Health and Human Services:
NCI Frederick Cancer Research Facility
Program Resources, Inc.
Frederick, MD
Funds for R&D, Federal - receipts for R&D performed by the
company on Federal R&D contracts or subcontracts and R&D
portions of procurement contracts and subcontracts.
Funds for R&D, total - operating expenses incurred by a
company in the conduct of R&D in its own laboratories or
other company owned or operated facilities including wages
and salaries, materials and supplies consumed, property and
other taxes, maintenance and repairs, depreciation, and an
appropriate share of overhead and excluding capital
expenditures.
Net sales and receipts - dollar values for goods sold, or
services rendered, by R&D-performing companies to customers
(outside the company), including the Federal Government,
less such items as returns, allowances, freight, charges,
and excise taxes. (Domestic intracompany transfers and
sales by foreign subsidiaries are excluded, but transfers to
foreign subsidiaries and export sales to foreign companies
are included.)
Research and development - basic and applied research in the
sciences and engineering and the design and development of
prototypes and processes, excluding quality control, routine
product testing, market research, sales promotion, sales
service, other nontechnological activities or routine
technical services, and research in the social sciences or
psychology.
Basic research - original investigations for the
advancement of scientific knowledge not having specific
immediate commercial objectives, although such
investigations may be in fields of present or potential
interest to the reporting company.
Applied research - investigations for the discovery of
new scientific knowledge having specific commercial
objectives with respect to products or processes.
(Applied research differs from basic research chiefly
in terms of the objectives of the reporting company.)
Development - technical activities not routine in
nature concerned with translating research findings or
other scientific knowledge into products or processes.
(Not included are routine technical services to
customers or other activities excluded above.)
Section C. Survey Documents
Page
National Science Foundation Cover Letter 121
Bureau of the Census Cover Letter 123
Survey Instructions 125
Survey Questionnaire, Form RD-1 133