Appendix A: Technical Notes
Definitions for Classification and Measurement
Data Analysis
Human Resources
List of Supporting Data Sources on R&D Expenditures
Appendix B: Tables Corresponding to Technical Notes
Appendix C: Detailed Statistical Tables

Appendix A: Technical Notes

Definitions for Classification and Measurement

Classification of Sectors

The National Science Foundation (NSF) follows a four-sector division in reporting research and development (R&D) funds and personnel and maintaining time-series data on expenditures and employment. The sectors are (1) industry, (2) the Federal Government, (3) universities and colleges, and (4) other nonprofit organizations. These are described in more detail below. Data also are collected for federally funded research and development centers (FFRDCs), which are organizations exclusively or substantially financed by the Federal Government to meet a particular requirement or to provide major facilities for research and associated training purposes. Each center is administered either by an industrial firm, an individual university, a university consortium, or a nonprofit institution.

Federal Government. This sector consists of the agencies of the Federal Government of the United States.

Industry. This sector consists of both manufacturing and nonmanufacturing companies. Manufacturing companies are reported by major industry groupings. Nonmanufacturing companies include those in mining, construction, transportation, communications, and selected service industries, such as R&D laboratories and computer and data processing services. Performance of FFRDCs administered by industrial firms generally is included in industry totals, although FFRDC breakouts are available and reported separately from R&D totals. Industry's funding of industry R&D includes all funds received from non-Federal sources (e.g., from State and local governments).

Universities and Colleges. This sector consists of all institutions of higher education, both public and private. Performance of FFRDCs administered by universities and colleges are reported separately from totals for this sector. University funding of university R&D includes (1) State and local government funds separately budgeted for R&D and (2) restricted or general funds that the institutions themselves have been free to allocate for research. Funds from the Federal Government, industry, or other nonprofit institutions that are supplied in the form of grants or contracts for R&D at a university, are credited to the appropriate source. For example, research contracts from industry are treated as university performance funded by industry. Funds given to the institution by industry for general educational purposes and used by the school—at its discretion—for research are treated as university performance financed with the university's own funds.

Other Nonprofit Institutions. This sector consists of institutions that fall into two general groups: (1) organizations that are primarily granting in nature—i.e., private philanthropic foundations and voluntary health agencies; and (2) public and private organizations involved in performing R&D, including FFRDCs administered by nonprofit organizations.

Research and Development Categories

Research and Development. In this report R&D consists of basic and applied research in the sciences (including medical sciences) and in engineering, and activities in development, all defined below.

The Federal, university, and nonprofit sectors include data for the broad fields of physical sciences, environmental sciences, mathematical sciences, computer sciences, life sciences, psychology, social sciences, an all-inclusive "other sciences" category, and engineering. Industry coverage is limited to (1) the physical sciences, including related engineering and (2) the biological sciences, including medicine but excluding psychology. Industry R&D specifically excludes research in the social sciences.

Basic Research. Within the Federal, university, and nonprofit sectors, basic research is defined as research directed toward increases in knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific application toward processes or products in mind. For the industry sector, basic research projects are defined as "original investigations for the advancement of scientific knowledge . . . which do not have specific commercial objectives, although they may be in fields of present or potential interest to the reporting company."

Applied Research. Within the Federal, university, and nonprofit sectors, applied research is defined as research directed toward gaining ". . . knowledge or understanding necessary for determining the means by which a recognized and specific need may be met." The applied research definition for the industry sector is modified to include ". . . research projects which represent investigations directed to discovery of new scientific knowledge and which have specific commercial objectives with respect to either products or processes."

Development. The NSF survey definition of development is ". . . the systematic use of the knowledge or understanding gained from research directed toward the production of useful materials, devices, systems or methods, including design and development of prototypes and processes." It excludes quality control, routine product testing, and production.

Defense-Space-Civilian Classification

This report contains data on (1) the preliminary percentage distribution of total U.S. R&D performance by national objective (table C-14) and (2) the reported distribution of Federal R&D authority by budget function (table C-23). The performer-based U.S. shares differ from the Federal budget authority shares for several reasons. The U.S. shares are based on expenditures reported by performers that often expend Federal R&D funds in a year other than the one in which the Federal Government provided authorization, obligations, or outlays. In addition, the two series are based on slightly different concepts. For example, whereas in the U.S. series all of the National Aeronautics and Space Administration's (NASA's) R&D funds are considered to be expenditures for space R&D, the budget authority data are distributed according to the functional categories that constitute the Federal budget. Thus, NASA's R&D budget authorizations are distributed between the space research and technology function and the transportation function.

"Defense R&D" consists of R&D spending by the Department of Defense (DOD) and defense-related atomic energy programs of the Department of Energy. All DOD activities are classified as defense, although some activities have secondary objectives (for example, space). "Space R&D" consists of R&D spending by NASA. All industry-funded R&D is classified as civilian R&D, including expenditures by aerospace and electronic industries.

Current Operating Costs

Funds used for R&D refer to current operating costs. These costs consist of both direct and indirect costs. They include not only salaries, but also fringe benefits, materials, supplies, and overhead. The R&D costs include depreciation, insofar as this information is available to respondents. Capital expenditures are excluded by definition in the surveys of the industry and academic sectors. Under the accounting practices of some Federal agencies, obligations for capital items may be included.

For universities and colleges, R&D data are for separately budgeted expenditures only. Consequently, these data exclude that portion of salaries for research time or other research expenses financed by funds not specifically earmarked for R&D from State and local governments and other non-Federal sources, including endowments.

Intramural Federal Performance of R&D

Intramural R&D performance by Federal agencies refers to work carried on directly by agency personnel. Federal obligations reported under this category are for activities performed or to be performed by the reporting agency itself or represent funds that the agency transfers to another Federal agency for performance of work, as long as the ultimate performer is that agency or any other Federal agency. If the ultimate performer is not a Federal agency, the funds so transferred are instructed to be reported by the transferring agency under the appropriate extramural performer category (universities and colleges, other nonprofit institutions, or industrial firms).

Intramural activities cover not only the actual intramural R&D performance, but also the costs associated with the planning and administration of both intramural and extramural programs by Federal personnel. Intramural activities also include the costs of supplies and equipment, essentially of an " off-the-shelf" nature, that are procured for use in intramural R&D. For example, the purchase from an extramural source of an operational launch vehicle (i.e., one that has gone beyond the development or prototype stage) that is used for intramural performance of R&D is reported as a part of the cost of intramural R&D.

Controlling for Inflation and Foreign Currency

Throughout this report, the term "current dollars" refers to dollar amounts as they are measured and exchanged in the actual year(s) in question. In contrast, "constant dollars" refers to dollar amounts normalized for inflation. For example, if the same dollar amount is reported in current dollars for two different years, this means that, because of inflation, fewer actual goods and services could be purchased with that amount in the more recent year than in the earlier year. If the same amount is reported in constant dollars for two different years, this means that, because the amount has been normalized for inflation, the same purchasing power would exist in each of the two years. Terms that are equal in meaning to current and constant dollars are, respectively, "nominal" and "real" dollars. These terms are also used to describe changes in dollar amounts over time. For instance, suppose a particular type of expenditure grew at a rate of 5 percent per year over a 10-year period. Such growth may be described as a 5 percent growth in real terms, or equivalently, a real growth of 5 percent, meaning the constant-dollar amounts grew at a 5 percent rate, while the current dollar amounts grew at a greater rate due to inflation.

In keeping with U.S. Government and international standards, R&D trend data usually are deflated to constant 1987 dollars using the gross domestic product (GDP) implicit price deflator. (See table C-1.) Since GDP deflators are calculated on an economy-wide rather than R&D-specific basis, their use more accurately reflects an "opportunity cost" criterion, rather than a measure of cost changes in doing research. That is, the GDP deflator, when applied to R&D expenditure or funding data, reflects the value of R&D in terms of the amount of other goods and services that could have been purchased with the same amount of money. The constant-dollar figures reported here thus should be interpreted as real resources foregone in engaging in R&D rather than in other activities such as consumption or physical investment.

Broad-based deflators—such as the GDP deflator—could also be useful in approximating changes in the costs of conducting R&D activities.27 However, these deflators are less appropriate for calculating real R&D costs at a disaggregated level, e.g., in estimating the costs over time of conducting the level of R&D within a particular science or engineering subfield. In addition, even when an opportunity cost criterion is used, the usefulness of the deflator is constrained by the length of the time span examined—the longer the time span, the less meaningful the deflator. That is, over long spans of time, such as 20 years, dramatic changes in the makeup of goods and services create ambiguities in the interpretation and measurement of quality change, which in turn adversely affect the reliability of price deflators.

Comparisons in this report of U.S. and international R&D expenditure data are based on reported R&D investments converted to U.S. dollars using purchasing power parity (PPP) exchange rates. PPP exchange rates are designed to reflect differences in the purchasing power of currencies, based on the quantity of currency needed in order to purchase equivalent quantities of actual goods and services in the countries in question. That is, PPP exchange rates reflect real purchasing power, in the same sense that real dollars, described above, control for inflation. The PPP exchange rates used are generally not equivalent to market exchange rates, i.e., how much one currency would cost if it were to be bought (with another currency) from a financial institution. Market exchange rates are often influenced by factors other than real purchasing power, namely the relative supply of, and demand for, different currencies in international financial markets. A PPP exchange rate would not be equivalent to an ideal R&D exchange rate, which does not exist at present, but would, in theory, account for international differences in R&D costs alone. Nevertheless, the PPP exchange rate is generally better at reflecting differences in R&D costs between countries than a market exchange rate.

Performer Reporting

There is no single survey of R&D activity in the United States. Rather, NSF sponsors a series of surveys to collect data on the financial and human resources devoted to R&D in the various sectors of the U.S. economy (defined above). Although these surveys are not designed specifically for this purpose, they provide the primary source material for estimating the national R&D totals. Respondents indicate the amounts they spend on R&D in their own sector and, generally, the sources of these funds. To the greatest extent possible, national totals are based on data as reported by performers because they are in the best position to (1) indicate how much they spend in the actual conduct of R&D in a given year; (2) classify their work as basic research, applied research, or development; and (3) identify the sector of the economy in which their financing originated. For these reasons, and because the consistent use of performer reporting reduces the possibility of double-counting and conforms to international standards (as outlined by the Organisation for Economic Co-operation and Development), R&D data are presented on a performer basis whenever possible.

Separate R&D performance totals are reported for (1) the Federal Government, (2) industry, (3) universities and colleges, (4) university-administered FFRDCs, and (5) other nonprofit organizations. R&D performed by State and local government agencies is not included in the national R&D totals. When State and local governments are listed by a survey respondent as the source of non-Federal R&D funds, those amounts are included in the source totals of the sector reporting the R&D performance.

The following material outlines the approaches and assumptions used for deriving the national totals. The procedures for estimating R&D expenditures by performer and source for the years 1992 through 1996 are further summarized in table B-7. The structure of the table is the same as that of tables C-2, C-5, C-8, and C-11, in which the expenditure totals are detailed.

Federal Government

Federal Performance Expenditures.Federal agency R&D obligations for intramural performance are treated as the equivalent of R&D expenditures in the National Patterns series. As detailed in the Federal Funds for Research and Development series (Federal Funds), such intramural activities cover costs associated with the planning and administration by Federal personnel of intramural and extramural R&D programs as well as actual intramural R&D performance. In general, the universe of Federal agencies with R&D programs has been surveyed annually since 1953 for their R&D performance and since 1963 for the distribution by character of work. The most recent survey included R&D funding as reported by more than 300 reporting sites aggregated into 102 individual respondents from 32 Federal agencies or their subdivisions.

Federal Agencies as a Source of R&D Funding. NSF collects data on federally financed R&D from both Federal funding agencies and performers of the work (Federal labs, industry, universities, and other nonprofit organizations). As reported by Federal agencies, National Patterns uses data on Federal R&D budget authority and outlays, in addition to Federal obligations. The use of each series is clearly noted in the text.

Budget authority is the primary source of legal authorization to enter into financial obligations that will result in outlays. Budget authority most commonly is granted in the form of appropriations laws enacted by Congress with the approval of the President.28
Obligations represent the amounts for orders placed, contracts awarded, services received, and similar transactions during a given period, regardless of when the funds were appropriated or when future payment of money is required.
Outlays represent the amounts for checks issued and cash payments made during a given period, regardless of when the funds were appropriated or obligated.

For the reasons cited above, national R&D expenditure totals are constructed primarily based on data reported by performers and include estimates of Federal R&D funding to these sectors. But until performer-reported survey data on Federal R&D expenditures are available from industry and academia, data collected from the Federal agency funders of R&D are used to project R&D performance. When survey data from the performers subsequently are tabulated, these statistics replace the projections based on funder expectations. Historically, the two survey systems tracked fairly closely. For example, in 1980 performers reported using $29.5 billion in Federal R&D funding, and Federal agencies' reported total R&D funding between $29.2 billion in outlays and $29.8 billion in obligations (table B-1). In recent years, however, the two series have diverged considerably: For 1994, performers reported $60.2 billion in Federal R&D support, compared with the $66.2 billion to $68.3 billion reported by Federal agencies.

The difference in the Federal R&D data totals appears to be concentrated in funding of industry (primarily aircraft and missile firms) by the Department of Defense. Overall, industrial firms have reported significant declines in Federal R&D support since 1990, while Federal agencies reported level or slightly increased funding of industrial R&D. For 1994, Federal agencies reported $31.7 billion in total R&D obligations provided to industrial performers, compared with an estimated $22.5 billion in Federal R&D funding reported by industrial performers (table B-2). NSF is examining the causal factors for these divergent trends.


Sample Design Prior to 1992. In general, the industry sector has been surveyed annually since 1953 for its total R&D performance and since 1956 for the distribution by character of work. The U.S. Bureau of the Census conducts the survey for NSF. The survey's target population is companies, whether U.S. or foreign owned, that perform R&D in the United States. Prior to the 1992 survey, a new sample was drawn and canvassed only every 5 or 6 years (for example, in 1976, 1981, and 1987). In the intervening years, a subset of the last sample—called a panel and including all companies reporting more than $1 million in R&D—was surveyed. As a result, for the 1987 survey, approximately 14,000 firms were selected for the sample. For the 1988-91 studies, approximately 1,600 of these firms were annually resurveyed; the other firms did not receive another questionnaire, and their R&D data were estimated though not observed. Accordingly, data for the years in which a sample was not drawn did not include companies that were new entrants in the R&D field, and such data were generally biased in a downward direction. The Census Bureau, however, did estimate the annual changes in R&D data for companies that reported R&D in the sample year but were not included in the panel. As new samples were drawn, revisions to previous years' estimates were issued—a process called "wedging."

For example, a new sample was drawn for 1987, from which R&D data were collected for 1986 as well as 1987. The Census Bureau used the data from the new sample to revise 1986 R&D performance estimates. It also used the new 1986 data in combination with data from the last sample year, 1981, to revise estimates for the intervening years. NSF subsequently provided a second round of revisions to the 1982-85 R&D series. In both cases the revisions were done on an industry basis: an effort was made to apply the overall 1981-86 growth rates while preserving the relative year-to-year movements in each industry's R&D. This approach resulted in major revisions of the 1982-87 industry R&D time series previously published in National Patterns, especially of the nonfederally funded component of industry's R&D performance. The revised totals, as well as the industry data reported in the 1990 National Patterns (NSF 90-316), are presented in table B-3. (Some of these data have since been revised, but the revisions were unrelated to the drawing of a new sample.) 29 Additional details on survey methods, coverage, concepts, definitions, and reliability of the estimates associated with the R&D expenditure data are contained in the 1990 edition (NSF 94-304) of the Research and Development in Industry series (Industry R&D).

Sample Design Revision in 1992. More recent surveys of industrial R&D performers have included revised R&D data based on relatively large industry samples. In contrast to data being based on probability samples selected around every 5 years, in 1992 NSF began to draw new samples annually, with the size of each sample increasing to approximately 24,000 firms. Industry classifications also were updated. The new sampling method now better reflects the widening population of R&D performers among firms in nonmanufacturing industries and small firms in all industries. As a result of these survey improvements, the revised 1991 industry R&D performance total ($117 billion) was 14 percent higher than was previously reported ($102.2 billion), and the national R&D total was 10 percent higher. These revisions were first reported in the 1994 National Patterns (NSF 95-304).

Of the $14.7 billion revision, $13.7 billion resulted from the new sample and $1.0 billion from normal data revisions for firms sampled in both surveys. Furthermore, $11.4 billion of the $13.7 billion increase stemming from the enlarged sample design was reported for nonmanufacturing industries, including $2.0 billion of R&D in industries not previously included in the sample frame. Complete technical details on industry's new survey methodology are contained in Research and Development in Industry: 1992 (NSF 95-328).

As in previous sampling cycles, National Patterns includes revisions to the industry data for years intervening the two sample years (i.e., for 1988, 1989, and 1990). The industry and U.S. time series reported here include the wedged data reported for 1988-90 and the revised data for 1991-92. Table B-4 provides summary statistics for wedged data that appeared in the 1994 National Patterns, along with other data that were previously published.

For almost all of the aggregate statistics (for example, industry R&D by Federal and non-Federal sources of funding), NSF believes that time-series comparisons (for example, between 1981 and 1994 data) are still reasonable: Surveys undertaken in both years provided the best estimates of the Nation's industrial R&D performance total by sampling those industries then believed to be conducting R&D. However, changes in the survey series between some data elements for consecutive years may be problematic. Not only do the 1987 and 1992 surveys' sample size and frame differ considerably (see above), but $9.2 billion (in constant 1992 dollars) of R&D performed by firms reporting in both surveys was shifted from one industry in 1987 to another in 1992—primarily from manufacturing industries in 1987 to nonmanufacturing industries in 1992. Such classification shifts can be attributed to (1) product mix changes of individual firms that occurred some time between 1987 and 1992, (2) changes in the 1987 Standard Industrial Classification (SIC) that were effected in the 1992 survey, and (3) a change in the methodology used by NSF/Bureau of Census for classifying companies to specific two- and three-digit SIC industries. Given that NSF has, since 1992, been committed to drawing new samples annually, the issue of wedging, and the reporting biases it creates, is unlikely to recur.

Use of "Nonmanufacturing" as a Single Industrial Category. The enormous growth in, and increasing economic importance of, nonmanufacturing industries is common knowledge. Thus, listing a single nonmanufacturing sector which includes all services alongside such specific manufacturing sectors as lumber products is a somewhat archaic method of categorizing U.S. industries. Indeed, circumstances in the future may support the opposite taxonomy: a single sector called manufacturing, within a categorization scheme that delineates different types of services (e.g., health services, communication services, financial services). Use of the current categorization scheme for R&D statistics, however, is justified on the historical grounds that the vast majority of R&D performance takes place within the manufacturing sector.

The observation that R&D is carried out primarily in manufacturing, however, is a product of historical precedence in the interpretation of "where R&D is located" rather than reflecting any real distinction between manufactures and services in terms of the utilization of new technologies. Services are just as reliant upon technological change as manufactures. In particular, many new forms of equipment and materials that result in technological innovation in services derive from R&D in manufacturing where such equipment and materials are first made. Health services is a case in point. Continual innovation in medical services generally results from R&D in the manufacture of pharmaceuticals and new medical equipment. Because such R&D was carried out for the specific purpose of improving services, the attribution of such R&D to manufacturing rather than services is a matter of interpretation and precedence, not an absolute difference between the sectors in terms of their dependence on, or promotion of, scientific and engineering advances.30

Another issue is that services and manufactures often differ in the nature of the R&D that they conduct. As a result, the relative quantity of R&D measured for services, in comparison to manufactures, is dependent on how R&D is defined. For example, software development for particular computer entertainment packages, which would fall under services, would involve idea development that integrates computer science techniques with artistic creation. Whether such an activity would be classified as "R&D" would be a matter of interpretation and degree. In contrast, research on new hardware equipment would be much less subject to interpretation, and would tend to be automatically classified as R&D.

Cognizant of these changes in relationships between technology and R&D and between manufactures and services, NSF has expanded its coverage of the nonmanufacturing sector in its industry R&D surveys. As a consequence, nonmanufacturing firms as a group comprised approximately 25 percent of the total industrial R&D performance in 1994, compared with an estimated 11 percent share in 1988. In terms of dollars spent in 1994, among the largest nonmanufacturing performers were computer-related service firms ($6 billion) and research, development, and testing firms ($2 billion). In future publications, NSF hopes to report considerably more detail on the composition and location of nonmanufacturing R&D.

Character-of-Work Revisions. As first noted in the 1990 National Patterns, the procedures used by the Bureau of Census for imputing character-of-work splits for industry's R&D performance were changed for 1986 and later years; hence, these data are not directly comparable with data for 1985 and earlier years. A full description of the various imputation methodologies—and alternatives—is presented in the 1988 Industry R&D report (NSF 90-319). Briefly, for 1985 and earlier years, for companies that did not report character-of-work splits, the Bureau of Census imputed the splits based on either (1) the company's percentage distribution reported in its most recent year of available data or (2) in the absence of any prior year breakdown for the company, the average character-of-work split for the industry to which the company was assigned. For years after 1985, the Bureau of Census does not impute a company's character-of-work distribution unless the company has reported a breakout within 2 years of the year being imputed. When distributions are not imputed, the Bureau of Census assigns the company's R&D to an "undistributed residual" category. (For example, and as detailed in table B-3, industry reported $96.7 billion in 1992 R&D performance with non-Federal funds; of that, the Bureau of Census did not distribute $16.4 billion.)

To provide character-of-work estimates for the entire population of firms performing R&D in the United States, each industry's (as contrasted with each individual company's) undistributed residual was allocated to basic research, applied research, and development categories using the average character-of-work splits reported for that industry. This approach resulted in relatively higher performance shares for basic and applied research than had been previously estimated and relatively lower estimates for development's share of industry's total R&D performance.

For 1994, $11.1 billion of the $97.1 billion in company-performed non-Federal R&D was not allocated to specific character-of-work categories by the reporting firms, nor was $1.3 billion of the company-reported Federal R&D total. Table B-5 provides the industry R&D category distributions used in this National Patterns as well as a historical summary for the years 1985-93.31

This National Patterns contains a major revision in industry's estimated basic research performance for the years 1991-94 from that published in the 1994 National Patterns. Previously, basic research was estimated to have almost doubled in 1991 (to $9.4 billion) from the company-reported $5 billion in 1990. The revised basic research amount for 1991 is $7.8 billion and approximately $7 billion for each year thereafter. The Census Bureau has since determined that several firms in the computer software service sector reported their R&D as basic research when development was the more appropriate reporting category. These changes have been made to the historical series.

Universities and Colleges

The academic sector, including all university-administered FFRDCs, has been surveyed annually since 1972 for R&D performance. It was surveyed less frequently before 1972. For 1994, data were collected from a sample of 500 institutions drawn from a population of 681 institutions of higher education in the United States and outlying areas that (1) granted a graduate degree in science or engineering and/or (2) performed activities for which at least $50,000 had been funded from separately budgeted R&D expenditures. The institutions sampled comprised all doctorate-granting institutions, all historically black colleges and universities with any R&D expenditures, and a random sample of all other institutions. For 1989-92, R&D performance was estimated from a census of institutions that granted doctorates in science and engineering plus a sample of all other universities and colleges. For 1993, data were collected from the full population of 681 institutions that met the criteria listed above.

Character-of-Work Revisions. With the exception of 1978, data on the basic research performance of universities and colleges and of university-administered FFRDCs have been collected annually since 1972. Since 1979, however, only the combined total for applied research and development performance has been collected. Furthermore, data on the character of work from individual non-Federal sources of funds (i.e., industry, institutional funds, State and local governments, and other sources) are not surveyed. For the years 1978 to the present, the distribution of applied research and development from Federal sources is based largely on data from Federal Funds; the development split from each of the non-Federal funding sources is calculated as approximately 7 percent of the surveyed non-Federal R&D funding totals; and the applied research share from each of the non-Federal funding sources is an estimated residual (total R&D funds, minus basic research funds, minus development funds).

Revised estimates for Federal funding of applied research and development to universities and colleges and to university-administered FFRDCs were first included in the 1992 National Patterns. University performers report the amount of R&D and basic research that they undertake with Federal funds. The residual is their combined applied research and development performance. The distribution between applied research and development is approximated based on the percentage shares of Federal obligation data to the academic sector as reported by Federal agencies in Federal Funds. Although the estimating procedures used previously had been loosely based on the data provided by the Federal funding agencies, the approach adopted here formally links performer- and source-reported survey data. Applied research and development expenditures for universities and colleges were revised for the period 1978 to the present; for university-administered FFRDCs, revisions were made back to 1975. The general result is that the applied research share is slightly lower and the development share higher than previously reported. For example, of the $9.0 billion in Federal R&D support to universities and colleges in 1989, $2.9 billion was for applied research and development and $6.1 billion was for basic research activities. In the 1990 National Patterns, $2.5 billion (28 percent of all Federal funds to the sector) was tabulated as applied research and $0.3 billion (3 percent) as development. In the 1992 National Patterns and in this report, 1989 Federal applied research funds are tabulated at $2.1 billion (24 percent) and development at $0.7 billion (8 percent). The revised totals, as well as the data reported in the 1990 National Patterns, are presented in table B-6.

Subcontracting. Only for the academic sector does R&D performance include research funds subcontracted to outside organizations. (For performance reported by respondents in the other surveyed sectors, R&D subcontracted to other organizations is excluded.) Details on survey methods, coverage, concepts, definitions, and reliability of the estimates associated with R&D expenditure data are reported in the fiscal year (FY) 1994 report (NSF 96-308) of the Academic Science and Engineering: R&D Expenditures (Academic R&D) series. There is preliminary evidence from NSF surveys that approximately 3 percent of total academic R&D funds are passed through the university to other recipients.

Other Nonprofit Institutions

It has not been possible to maintain the same survey frequency for other nonprofit institutions; the last complete survey was conducted in 1973. Since then, small and informal surveys of this sector have been undertaken periodically, most recently for 1983. For the years 1984 to the present, estimates for federally funded total R&D and character-of-work performance by nonprofit institutions—including associated FFRDCs—are derived from Federal obligation data reported in Federal Funds. Industry as a source of R&D funds to this sector is approximated based on the average of the annual percentage change in (1) industry's funding of industry-performed R&D (from Industry R&D) and (2) industry funding of university-performed R&D (from Academic R&D). Nonprofit funding as a source of R&D funds to this sector is approximated based on the annual percentage change in nonprofit funding of university-performed R&D (from Academic R&D). The character-of-work splits from the non-Federal funding sources that were surveyed in 1983 are carried forward to the present.

Data Analysis

Preliminary Data and Projection Procedures for 1995 and 1996

To the greatest extent possible, this report incorporates data for 1995 and 1996 R&D programs included in the administration's 1997 budget proposal. For example, the 1997 budget contains data on total R&D outlays and budget authority by agency and by character of work. However, the budget does not contain reliable estimates on the amount of Federal R&D funds received by each of the R&D-performing economic sectors, but only the federally funded totals and funds received by universities and colleges. The detailed sector-specific information is obtained from an NSF survey of Federal agencies' R&D obligations. This information is collected after the President's proposed budget has been published. For this reason, some of the 1995 and 1996 Federal R&D data reported here are based on the administration's 1995 through 1997 budget proposals.

Preliminary R&D performance totals in National Patterns are calculated for each sector, by character of work, and by source of funds from surveys and time-series extrapolation techniques, as follows.

Federal Government. Projections for 1995 and 1996 are based on changes in intramural R&D obligations reported in Federal Funds. The amounts reported for 1995 are preliminary and reflect congressional appropriations, apportionments, and reprogramming decisions as of the third quarter of FY 1995. Data for 1996 are projections that reflect the reduction in intramural R&D represented in the administration's 1996 budget proposal.

Industry. Preliminary data for company-funded 1995 performance are based on (1) industry responses to the 1995 Industry R&D Survey, as of June 1996, which accounted for 44 percent of the R&D performed by industrial firms in 1994; (2) a mail survey conducted in August 1994 by the Industrial Research Institute (IRI) of its membership; and (3) data provided in the Standard and Poors Compustat Services database as of June 1996. 32 By the Compustat account, nominal corporate R&D spending of the top 100 firms is reported to increase 15 percent in 1995, compared with the 0.6 percent growth for 1995 projected by IRI members in the fall of 1994, and the 8 percent growth for 1995 reported in this National Patterns. Projections for 1996 are based on a mail survey completed by 151 IRI members in August and September 1995. Information on the 1996 IRI survey may be found in Research Technology Management, Vol. 39, No. 1, January-February 1996. Previously, NSF had conducted its own survey of companies' planned R&D expenditures.

Projections for 1995 and 1996 federally funded R&D performance are based on (1) time-series modeling for the Federal sector as a source of funds, supplemented by (2) Federal obligations data reported in Federal Funds (NSF, 97-302 and unpublished updates).

Universities and Colleges. Preliminary data for 1995 are based on university responses to the FY 1995 Academic R&D Survey, as of June 1996. These respondents accounted for 89 percent of the R&D performed by universities and colleges in FY 1994. Projections for 1996 are based on (1) Federal obligations to the academic sector reported in Federal Funds (NSF, 97-302 and unpublished updates) and (2) time-series modeled extrapolations of recent trends for each of the non-Federal sources.

Other Nonprofit Institutions. Preliminary tabulations for 1995 and 1996 are based on (1) Federal obligations reported in Federal Funds (NSF, 97-302) and (2) time-series modeled extrapolations of recent trends in R&D performance and funding within the industry and academic sectors, as outlined in the preceding section, "Performer Reporting Basis."

Sector Summary. Table B-7 provides a concise listing of sectors for which performer-reported data, source-reported data, or a combination of the two are used; the general approaches adopted (for example, time-series techniques); and assumptions used in preliminary national expenditures on R&D, basic research, applied research, and development, all of which are outlined above. The table includes such information for the current projection years (1995-96), for finalized data of recent years (1992-94), and for a historical reference year (1973).

Use of Time-Series Data

Data presented in trend tables are assembled from the most recently completed survey cycles. Data for prior years are reviewed for consistency with current year responses and—when necessary—revised in consultation with survey respondents. In addition, changes in sample design or imputation methodologies can result in revisions to previously published data. For trend comparisons, the historical data contained in this report should be used rather than the data published in previous National Patterns volumes.

Geographic Distribution

This report contains information on the State distribution of R&D performance for 1993 (table C-16). These data cover R&D performance by Federal agencies, industry, academia, and the federally funded R&D activities of nonprofit institutions. The zero figures reported for some States in table C-16 represent R&D performance of less than $0.5 million. These State-distributed data are meant to be indicative of general distribution patterns; they may not be precise enough to warrant strong comparisons among similiar numbers.

The geographic data on 1987 R&D expenditures in the 1990 National Patterns included State agencies' intramural R&D performance that was obtained from a special NSF survey, reported on in Research and Development Expenditures of State Government Agencies: Fiscal Years 1987 and 1988 (NSF 90-309). The estimates for 1993 provided here do not include this State agency component.

State-distributed data for the industry sector are collected for odd-numbered years. The latest available detailed data are for 1993 and are from Industry R&D (NSF 96-304). The data include R&D performance by industry-administered FFRDCs.

State-distributed data for the academic sector are collected only for doctorate-granting institutions and university-administered FFRDCs (Academic R&D). R&D performance by an FFRDC is assigned to the State in which the FFRDC is located, which is not necessarily the State in which the administering institution is located.

State-distributed data for Federal laboratories are intramural R&D obligations in FY 1993. These data are available from the 10 major R&D-supporting agencies (Federal Funds).

State-distributed data for other nonprofit institutions are Federal R&D obligations to this sector in FY 1993 as reported by the 10 major R&D-supporting Government agencies (Federal Funds). These agencies provided approximately 98 percent of total Federal R&D obligations in 1993. Data on R&D performance by this sector using non-Federal sources of funds are not collected.

Human Resources

The 1994 National Patterns was the first to include revised data on scientists and engineers (S&Es) engaged in R&D activities. This national series consists of separate survey estimates of R&D S&Es employed in industry and in the Federal Government and doctorate-holding R&D S&Es employed in educational institutions and in nonprofit organizations. The industry series are for S&Es employed on a full-time-equivalent (FTE) basis; totals for the other sectors reflect the primary work activity of S&Es.

A variety of surveys and estimation techniques are used to gather information on the numbers and characteristics of persons engaged in science and engineering activities in all sectors of the economy. In general, two types of surveys are used in reporting worker inputs for R&D: surveys directed at individuals and surveys directed at employers.

Surveys of Individuals

These surveys (in this report, of scientists and engineers holding doctorate degrees) result in data on the primary work activities and demographic and economic characteristics of the respondents. In the survey of doctoral scientists and engineers, respondents are asked to report their primary work activity—i.e., the activity on which they spend the largest proportion of their time, but which is not necessarily a full-time activity. This survey is conducted only in odd-numbered years. The latest tabulated data are available for 1993 and are summarized in table C-19. Details on survey methods, coverage, concepts, definitions, and reliability of the estimates associated with these S&E data are in Characteristics of Doctoral Scientists and Engineers in the United States: 1993 (NSF 96-302).

Surveys of Employers

These surveys generally are focused on the amount of time—in terms of person-years—devoted to the performance and management of R&D. In this report, data on the number of S&Es—not just those holding doctoral degrees—employed by industry on an FTE basis in R&D are summarized in table C-35. For example, if each of two scientists/engineers spends 50 percent of the workday on R&D, the equivalent is one FTE R&D job.

Previously, National Patterns provided national estimates of FTE R&D scientists and engineers. At one point, NSF had survey data for FTE estimates in all sectors of the economy. Currently, NSF collects such data only for the industrial sector. The last FTE R&D personnel survey of the academic sector was for 1985, and the last manpower survey of the nonprofit sector was for 1973, although a small telephone survey was conducted for 1983.

The loss of such survey data necessitated increased reliance on analytically derived figures (including the use of regression equations) which were based largely on estimating assumptions that could not be empirically tested for their continued validity. Consequently, those preliminary series are replaced here with survey counts of the number of doctorate-holding S&Es who self-report their primary work activity as R&D or R&D management. How well these head counts might approximate an FTE estimate is unknown. On the one hand, they may provide an overestimate of FTE activity since many of the surveyed S&Es are not engaged in R&D full-time even though it is their primary work activity. On the other hand, this approach may underestimate FTE R&D personnel since it does not account for S&Es engaged in R&D who do not hold a doctorate degree. Sources for the revised estimates and comparison with the 1985 and 1989 figures published in the 1992 National Patterns, are described and summarized in table B-8. For the total United States, the revised figures for 1989 (924,200) are 3 percent lower than previously reported (949,300).

Federal Government. For the Federal sector, survey data on civilian scientists and engineers are collected annually (Federal Scientists and Engineers: 1989-93, NSF 95-336). The estimates are compiled from the U.S. Office of Personnel Management's (OPM's) Central Personnel Data File on all white collar civilian jobs and are reported in terms of primary work activities. Scientists or engineers are included in the National Patterns totals if their primary work activity is research or development. These head counts exclude (1) military personnel (but include civilian S&Es employed in defense agencies) and (2) Federal employees classified in a management occupational code, even if they manage an R&D program. The earliest year for which these OPM statistics have been compiled is 1985. Data for 1985-89 published in the previous National Patterns included estimates for R&D managers; these estimates are no longer included in the Federal totals. For years prior to 1985, the figures reported were based on NSF surveys since discontinued.

Industry. Industry is the only sector for which FTE R&D S&E survey estimates are available. Firms report (Industry R&D) FTE employment levels for January of each year, and a simple 2-year moving average is used for the national R&D S&E series. For example, the total reported for 1989 (733,000) is the average of the level reported by firms for January 1989 and January 1990. Except for minor data revisions resulting from the inclusion of wedged statistics, the industry totals reported here do not differ from those reported previously. As detailed above for the industry expenditure data, improvements in the sample design for 1992 and later years resulted in data that now better reflect R&D performance among firms in the nonmanufacturing industries and small firms in all industries.

Universities and Colleges. For the academic sector, two series are reported: doctoral scientists and engineers and graduate students performing research. The head counts for research students are from the Survey of Graduate Students and Postdoctorates in Science and Engineering and are for full-time science and engineering graduate students in all institutions whose major financial support is research assistantships. In this revised series, FTE estimates are derived assuming a 50 percent workload (or working half-time on R&D); previously, a 47 percent workload assumption was used.

Academic institutions were previously surveyed for estimates of FTE R&D S&Es; however, 1985 is the most recent year for which this survey was conducted. Since then, the academic estimates published in National Patterns were derived, usually, from a regression analysis of the 1975-85 academic FTE survey data based on the explanatory variables: (1) academic R&D expenditures and (2) the number of academic doctoral S&Es who reported R&D as their primary work activity. The revised series directly uses reported employment levels from the Survey of Doctorate Recipients (SDR). The academic R&D employment totals are of doctoral scientists and engineers employed in all educational institutions who self-report their primary work activity as "research," "development or design," or the "management or administration of R&D." No adjustments are made to derive FTEs. For 1989, the revised primary work activity total (83,500) is approximately 11% percent less than the FTE figure (93,700) last published in National Patterns. Because the doctoral data are collected only biennially, the revised national FTE series also are reported biennially.

NSF introduced a number of improvements into the 1991 SDR (for example, changes in the age-based cohorts collected and in the definition of doctoral scientists and engineers) that may affect comparability with SDR data published for prior survey years. The academic S&E total for those reporting R&D as their primary work activity for 1989 is 83,500 and the total for 1991 is 74,600. Whether changes in the survey design or in actual employment patterns caused the academic R&D S&E decline is unknown. The report Characteristics of Doctoral Scientists and Engineers in the United States: 1991 (NSF 94-307) provides additional information on these methodological changes.

Other Nonprofit Institutions. The last survey of the nonprofit sector was for 1973. Since then, the nonprofit estimates published in National Patterns generally were based on survey data from the early 1970s and trends in the ratio of national R&D expenditures to FTE R&D S&Es. In the revised series, nonprofit R&D employment levels are taken from the Survey of Doctorate Recipients. The figures are for doctoral scientists and engineers employed in nonprofit organizations who self-report their primary work activity as "research," "development or design," or the "management or administration of R&D." These figures were not adjusted for part-time R&D activity or for R&D activity by nondoctoral scientists and engineers. For 1989, the revised primary work activity total (9,200) is approximately 75 percent less than the FTE figure (34,500) last published in National Patterns. The effect on the Nation's total FTE estimate is approximately a 2.7 percent downward revision.

List of Supporting Data Sources on R&D Expenditures

Characteristics of Doctoral Scientists and Engineers in the United States: 1993 (NSF 96-302). Presents data on demographic and employment characteristics of the Nation's doctoral scientists and engineers. These characteristics include citizenship, place of birth, field of degree, occupation, sector of employment, median salary, and rates of employment.

Federal Scientists and Engineers: 1989-93, (NSF 95-336). Contains information on the status of Federal scientists and engineers. The report provides a descriptive analysis of various characteristics of Federal scientists and engineers, including the agency of employment, primary work activity, educational attainment, age, salary, and geographic location.

"Industrial Research Institute's Annual R&D Trends Forecast for 1996," Research Technology Management, Vol. 39, No. 1, January-February 1996, pp. 15-17. Provides a summary of expected 1996 industrial R&D activity and includes information on changes related to companies' R&D budgets (such as participation in consortia and hiring patterns). Report is based on a mail survey completed by the Industrial Research Institute's membership in August and September 1995.

National Science Board, Science & Engineering Indicators: 1996, NSB 96-21 (Washington, DC: U.S. Government Printing Office, 1996). Source of information on R&D and other science and engineering data. Published source of national R&D expenditures estimates for years when the biennial National Patterns report is not published.

National Science Foundation, Academic Science and Engineering: R&D Expenditures, Fiscal Year 1994, NSF 96-308 (Arlington, VA, 1996). Detailed statistical tables cover academic R&D performance as reported in a survey of U.S. universities and university-administered federally funded research and development centers. Data include distribution by source of funds, performing institution, character of work, field of science, and geographic location.

National Science Foundation, Federal Funds for Research and Development: Fiscal Years 1994, 1995, and 1996, (NSF 97-302) (Arlington, VA, 1997). Detailed statistical tables cover R&D (and R&D plant) funding levels through November 1995 as reported by all Federal agencies with R&D programs. Includes data by agency, performer, character of work, geographic distribution, and field of science and engineering.

National Science Foundation, Federal R&D Funding by Budget Function: Fiscal Years 1995-97, (NSF 97-301) (Arlington, VA, 1997). Provides information on Federal R&D budget authority by Federal budget function as proposed in the administration's 1997 budget.

National Science Foundation, Research and Development in Industry: 1994, (forthcoming) (Arlington, VA,1996). Detailed statistical tables cover industrial R&D performance as reported in a sample survey of companies. Data include distribution by source of funds, industry classification, character of work, product field, geographic location, company size, and other tabulations.

Office of Management and Budget, The Budget of the United States Government, Fiscal Year 1997 (Washington, DC: U.S. Government Printing Office, 1996). Provides quantitative and qualitative information on R&D funding as proposed in the administration's 1997 budget.

Appendix B: Tables Corresponding to Technical Notes

List of Tables

B-1 Difference in agency-reported and performer-reported Federal R&D, all performers: 1980-96
B-2 Difference in agency-reported and performer-reported Federal R&D: industrial performers by agency source, 1980-94
B-3 Revisions in industry R&D performance totals: 1982-87
B-4 Revisions in industry R&D performance, and their impact on other variables: 1987-91
B-5 Distribution of industry R&D performance, by character of work: 1985-94
B-6 Revisions in university & college performance by Federal source of funds: 1974-90
B-7 Sources of data in compiling estimates on the level of R&D expenditures in the United States: 1973 and 1992-96
B-8 Revisions in number of R&D scientists and engineers: 1985 and 1989

Appendix C: Detailed Statistical Tables

List of Tables

National Totals
C-1 Gross domestic product and GDP implicit price deflators: 1953-99
C-2 National expenditures for R&D, by performing sectors and sources of funds: 1955-96
C-3 Sources of funds for research and development, by sector: 1953-96
C-4 Research and development performance by sector: 1953-96
C-5 National basic research expenditures: 1953-96
C-6 Sources of funds for basic research, by sector: 1953-96
C-7 Basic research performance, by sector: 1953-96
C-8 National applied research expenditures: 1953-96
C-9 Sources of funds for applied research, by sector: 1953-96
C-10 Applied research performance, by sector: 1953-96
C-11 National development expenditures: 1953-96
C-12 Sources of funds for development, by sector: 1953-96
C-13 Development performance, by sector: 1953-96
C-14 Trends in Federal and non-Federal R&D expenditures: 1953-96
C-15 National expenditures for R&D as a percentage of gross domestic product, by source of funds: 1953-96
C-16 Geographic distribution of U.S. R&D expenditures, by performer and source of funds: 1993
C-17 Total R&D and GSP by state, 1993
C-18 Scientists and engineers primarily engaged in research and development activities, by sector of employment: 1985-93
C-19 Number of employed doctoral scientists and engineers, by field of doctorate and primary work activity: 1993
C-20 Scientists and engineers engaged in R&D, and per 10,000 labor force population, by country: 1981-93
C-21 International total R&D expenditures and R&D as a percentage of GDP: 1981-95
C-22 International nondefense R&D expenditures and nondefense R&D as a percentage of GDP: 1981-95
Federal Government
C-23 Federal R&D budget authority, by budget function, fiscal years 1984-96
C-24 Federal outlays for R&D and other activities, by DOD and non-DOD sources: 1970-96
C-25 Federal obligations for research and development, by agency: fiscal years 1985-96
C-26 Federal obligations for basic research, by agency: fiscal years 1985-96
C-27 Federal obligations for applied research, by agency: fiscal years 1985-96
C-28 Federal obligations for development, by agency: fiscal years 1985-96
C-29 Total (company, Federal, and other) funds for industrial R&D performance, by industry and size of company: 1984-94
C-30 Company and other (except Federal) funds for industrial R&D performance, by industry and size of company: 1984-94
C-31 Federal funds for industrial R&D performance, by industry and size of company: 1984-94
C-32 Total R&D funds as a percent of net sales in R&D-performing manufacturing companies: 1984-94
C-33 Company and other (except Federal) R&D funds as a percent of net sales in R&D-performing manufacturing companies: 1984-94
C-34 Funds for performance of basic research, applied research, and development, by industry and source of funds: 1994
C-35 Number of full-time-equivalent (FTE) R&D scientists and engineers in R&D-performing companies: 1984-95
C-36 Cost per R&D scientist or engineer in R&D-performing companies, by industry and size of company: 1984-94
Universities and Colleges
C-37 R&D expenditures at universities and colleges, by source of funds and science and engineering field: fiscal years 1987-94
C-38 Federally financed R&D expenditures at universities and colleges, by science and engineering field: fiscal years 1987-94
C-39 R&D expenditures at university-administered FFRDCs, by science and engineering field: fiscal years 1987-94
C-40 Revised gross domestic product and GDP implicit price deflators: 1960-99


27See J.E. Jankowski, "Do We Need a Price Index for Industrial R&D?," Research Policy 22, 1993: 195-205.

28See NSF, Federal R&D Funding by Budget Function: Fiscal Years 1994-96 (Budget Function), NSF 95-342 (Arlington, VA, 1995).

29Note that although the Bureau of the Census reestimated 1982-86 R&D totals by funding source, it did not provide a character-of-work distribution for the revised data. After investigating several possible alternatives, NSF chose to allocate the revisions (table B-3) on the basis of average character-of-work distribution published in earlier annual Research and Development in Industry reports. Allocations for the federally funded and nonfederally funded R&D revisions were applied separately.

30For more detailed discussion on the interrelationship between R&D in manufacturing and advances in services, see, for example, B. Guile and J. Quinn, eds. Technology in Services: Policies for Growth, Trade, and Employment (Washington, DC: National Academy Press, 1988).

31See NSF, Research and Development in Industry: 1992, NSF 95-324, (Arlington, VA, 1995) for further discussion of this issue.

32IRI is an independent, nonprofit association of over 260 R&D-performing companies. These companies represent the aerospace, automotive, chemical, computer, and electronics industries, among others, and carry out approximately 80 percent of the industrial research in the United States. Detailed results of the Compustat tabulations for the year 1995 may be found in Inside R&D, Vol. 25, No. 23, June 5, 1996.

Top of
Table of