Section B. Technical Notes

Scope and Method Definitions Changes in Reporting Changes in the Scope of the Survey Limitations of the Data Data Collection

Scope and Method  

During the period February 2003 through February 2004, a total of 30 Federal agencies and their subdivisions—74 individual respondents—submitted data in response to the National Science Foundation's (NSF's) annual Survey of Federal Funds for Research and Development (Federal Funds survey), which was distributed in February 2003. The agencies reported their data as obligations and outlays incurred or expected to be incurred, regardless of when the funds were appropriated or whether they were identified in the respondents' budgets specifically for R&D activities.

Only those agencies that had obligations in the variables represented by a particular table appear in that table. Appendix A provides a complete list of the Federal agencies that have been included in the Federal Funds survey. Additional notes associated with these agencies are in appendix B.

Definitions  

Definitions are essentially unchanged from those used in past Federal Funds surveys.

1. An agency is an organization of the Federal Government whose principal executive officer reports to the President. The Library of Congress is also included in the survey, even though its chief officer reports to Congress. Subdivision refers to any organizational unit of a reporting agency, such as a bureau, division, office, or service.
   
   
2. Obligations and outlays reported are consistent with figures shown for FY 2002, 2003, and 2004 appearing in The Budget of the United States Government, FY 2004. The R&D data in both agency submissions were based on the same definitions and are reconcilable.
   

   Obligations represent the amounts for orders placed, contracts awarded, services received, and similar transactions during a given period, regardless of when the funds were appropriated and when future payment of money is required.

   Outlays represent the amounts for checks issued and cash payments made during a given period, regardless of when the funds were appropriated. Obligations and outlays cover all transactions that occurred in FY 2002 and those estimated for FY 2003 and 2004.

   The data include all Federal funds available to an agency that the agency received or expects to receive from direct appropriations, trust funds, special account receipts, corporate income, or other sources, including funds appropriated to the President.

   The amounts shown for each year reflect obligations or outlays for that year regardless of when the funds were originally authorized or received and regard- less of whether they were appropriated, received, or identified in the agency’s budget specifically for research, development, or R&D plant.

   In reporting its obligations or outlays, each agency includes the amounts transferred to other agencies for support of research and development. The receiving agencies do not report funds transferred to them. Similarly, a subdivision of an agency that transfers funds to another subdivision within that agency reports such obligations or outlays as its own.

   Obligations and outlays for R&D performed for an agency in foreign countries include all funds available to the agency for this purpose, including funds separately appropriated for special foreign currency programs.

   Funds reported for research and development reflect full-cost coverage, which is the costs of specific R&D and the applicable overhead costs. The amounts reported include the costs of planning and administering R&D programs, laboratory overhead, pay of military personnel, and departmental administration.

3. The fiscal year in the Federal Government accounting period begins October 1 of a given year and ends September 30 of the following year; thus, FY 2002 began on October 1, 2001 and ended September 30, 2002.
   
   
4. Research, development, and R&D plant include all direct, incidental, or related costs resulting from, or necessary to, performance of R&D and costs of R&D plant, defined as follows below, regardless of whether the R&D is performed by a Federal agency (intramurally) or by private individuals and organizations under grant or contract (extramurally). R&D excludes routine product testing, quality control, mapping and surveys, collection of general-purpose statistics, experimental production, and the training of scientific personnel.
   
   
1. Research is systematic study directed toward fuller scientific knowledge or understanding of the subject studied. Research is classified as either basic or applied according to the objectives of the sponsoring agency.

   Basic research is defined as systematic study directed toward fuller knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications towards processes or products in mind.

   Applied research is defined as systematic study to gain knowledge or understanding necessary to determine the means by which a recognized and specific need may be met.

2. Development is defined as systematic application of knowledge or understanding, directed toward the production of useful materials, devices, and systems or methods, including design, development, and improvement of prototypes and new processes to meet specific requirements.

   To better differentiate between the part of the Federal R&D budget that supports "science and key enabling technologies" (including for military and nondefense applications) and the part that primarily concerns "testing and evaluation" (of mostly defense-related systems), NSF now collects from the Department of Defense (DoD) development dollars in two categories, advanced technology development and major systems development.

   DoD uses service codes 6.1 through 6.7 to classify data into the survey categories. Within DoD's research categories, basic research is classified as 6.1, and applied research is classified as 6.2. Within DoD's development categories, advanced technology development is classified as 6.3A. Major systems development is classified as 6.3B through 6.7 and includes demonstration and validation, engineering and manufacturing development, management and support, and operational system development.

3. Demonstration activities that are part of R&D (i.e., that are intended to prove or to test whether a technology or method does in fact work) are included. Demonstrations intended primarily to make information available about new technologies or methods are excluded.
   
   
4. R&D plant (R&D facilities and fixed equipment, such as reactors, wind tunnels, and particle accelerators) includes acquisition of, construction of, major repairs to, or alterations in structures, works, equipment, facilities, or land for use in R&D activities at Federal or non-Federal installations. Excluded from the R&D plant category are expendable or movable equipment (e.g., spectrometers, microscopes) and office furniture and equipment. Also excluded are the costs of predesign studies (e.g., those undertaken before commitment to a specific facility). These excluded costs are reported under "total conduct of research and development." Obligations for foreign R&D plant are limited to Federal funds for facilities that are located abroad and used in support of foreign research and development.
   
   
5. Fields of science and engineering in this survey consist of eight broad field categories, each consisting of a number of detailed fields. The broad fields are life sciences; psychology; physical sciences; environmen- tal sciences; mathematics and computer sciences; engineering; social sciences; and other sciences, not elsewhere classified. The term "not elsewhere classified" (nec) is used for multidisciplinary projects within a broad field and for single-discipline projects for which a separate field has not been assigned. The following list presents the detailed fields grouped under each of the broad fields, together with illustrative disciplines of detailed fields.
   

   The illustrative disciplines are intended to be guidelines, not sharp definitions; they represent examples of disciplines generally classified under each detailed field. A discipline under one detailed field may be classified under another detailed field when the major emphasis is elsewhere. Research in biochemistry, for example, might be reported as biological, agricultural, or medical, depending on the focus of the project. Human biochemistry would be classified under biological, but animal biochemistry or plant biochemistry would fall under agricultural. In no case is the research reported under more than one field. No double counting is intended or allowed.

1. Life sciences are concerned with the scientific study of living organisms and their systems. They consist of five detailed fields: biological sciences (ex- cluding environmental biology), environmental biology, agricultural sciences, medical sciences, and life sciences, nec. Examples of the disciplines under each of these fields are as follows:

   Biological sciences (excluding environmental biology): anatomy, biochemistry, biology, biometry and biostatistics, biophysics, botany, cell biology, entomology and parasitology, genetics, microbiology, neuroscience (biological), nutrition, physiology, zoology, other biological sciences, nec.

   Environmental biology: ecosystem sciences, evolutionary biology, limnology, physiological ecology, population and biotic community ecology, population biology, systematics, other environmental biology, nec.

   Agricultural sciences: agronomy, animal sciences, food science and technology, fish and wildlife, forestry, horticulture, phytopathology, phytoproduction, plant sciences, soils and soil science, general agriculture, other agriculture, nec.

   Medical sciences: dentistry, internal medicine, neurology, obstetrics and gynecology, ophthal- mology, otolaryngology, pathology, pediatrics, pharmacology, pharmacy, preventive medicine, psychiatry, radiology, surgery, veterinary medi- cine, other medical sciences, nec.

   Life sciences, nec.

2. Psychology deals with behavior, mental processes, and individual and group characteristics and abilities. Psychology in this survey is divided into three categories: biological aspects, social aspects, and psychological sciences, nec. Examples of the disciplines under each of these fields are as follows:

Biological aspects: animal behavior, clinical psychology, comparative psychology, ethology, and experimental psychology

Social aspects: development and personality; educational, personnel, and vocational psychology and testing; industrial and engineering psychology; social psychology

Psychological sciences, nec.

3. Physical sciences are concerned with understanding of the material universe and its phenomena. They comprise the fields of astronomy, chemistry, physics, and physical sciences, nec. Examples of disciplines under each of these fields are as follows:

Astronomy: laboratory astrophysics; optical astronomy; radio astronomy; theoretical astrophysics; x-ray, gamma-ray, and neutrino astronomy

Chemistry: inorganic, organic, organometallic, and physical chemistry

Physics: acoustics, atomic and molecular physics, condensed-matter physics, elementary particle physics, nuclear structure, optics, plasma physics

Physical sciences, nec.

4. Environmental sciences (terrestrial and extraterrestrial) are, with the exception of oceanography, concerned with the gross nonbiological properties of the areas of the solar system that directly or in- directly affect human survival and welfare. Obligations for studies pertaining to life in the sea or other bodies of water are reported as support of oceanography and not biology. Environmental sciences comprise the fields of atmospheric sciences, geological sciences, oceanography, and environmental sciences, nec. Examples of disciplines under each of these fields are as follows:

Atmospheric sciences: aeronomy, extraterrestrial atmospheres, meteorology, solar science, weather modification

Geological sciences: engineering geophysics, general geology, geodesy and gravity, geomagnetism, hydrology, inorganic geochemistry, isotopic geochemistry, laboratory geophysics, organic geochemistry, paleomagnetism, paleontology, physical geography and cartography, seismology

Oceanography: biological oceanography, chemical oceanography, marine geophysics, physical oceanography

Environmental sciences, nec.

5. Mathematics and computer sciences employlogical reasoning with the aid of symbols and are concerned with the development of methods of operation using such symbols and, in the case of computer sciences, with the application of such methods to automated information systems. Examples of disciplines under these fields are as follows:

Mathematics: algebra, analysis, applied mathematics, foundations and logic, geometry, numerical analysis, statistics, topology

Computer sciences: computer and information sciences (general); design, development, and application of computer capabilities to data storage and manipulation; information sciences and systems; programming languages; systems analysis

Mathematics and computer sciences, nec.

6. Engineering is concerned with studies directed toward developing engineering principles or toward making specific principles usable in engineering practice. Engineering in this survey is divided into eight fields: aeronautical, astronautical, chemical, civil, electrical, mechanical, metallurgy and materials engineering, and engineering, nec. Examples of disciplines under each of these fields are as follows:

Aeronautical engineering: aerodynamics

Astronautical engineering: aerospace, space technology

Chemical engineering: petroleum, petroleum refining process

Civil engineering: architectural, environmental, hydraulic, hydrologic, marine, sanitary, and structural engineering; transportation

Electrical engineering: communication, electronic engineering, power

Mechanical engineering: engineering mechanics

Metallurgy and materials engineering: ceramic engineering, mining, textile engineering, welding

Engineering, nec.: agricultural engineering, bioengineering, biomedical engineering, industrial and management engineering, nuclear engineering, ocean engineering, systems engineering

7. Social sciences are directed toward an understanding of the behavior of social institutions and groups and of individuals as members of a group. Social sciences include anthropology, economics, political science, sociology, and social sciences, nec. Examples of disciplines under the fields of social science are as follows:

Anthropology: applied anthropology, archaeology, cultural anthropology and personality, social anthropology and ethnology

Economics: econometrics and economic statistics; economic systems and development; economic theory; history of economic thought; industrial, labor, and agricultural economics; international economics; macroeconomics; microeconomics; public finance and fiscal policy

Political science: area or regional studies, comparative government, history of political ideas, international relations and law, national political and legal systems, political theory, public administration

Sociology: comparative and historical sociology, complex organizations, culture and social structure, demography, group interactions, social problems and social welfare, sociological theory

Social sciences, nec.: linguistics, research in education, research in history, research in law (e.g., attempts to assess impact on society of legal systems and practices), socioeconomic geography

8. Other sciences, nec.: This category is used for multidisciplinary or interdisciplinary projects that cannot be classified within one of the broad fields of science already listed.
   
   
6. A performer is either an intramural group or organization carrying out an operational function or an extramural organization or person receiving support or providing services under a contract or grant.
   
   
1. Intramural performers are the agencies of the Federal Government. Their work is carried on directly by agency personnel. Obligations reported under this category are for activities performed or to be performed by the reporting agency itself or are for funds that the agency transfers to another Federal agency for performance of work, as long as the ultimate performer is that agency or any Federal agency. If the ultimate performer is not a Federal agency, the funds transferred are reported by the transferring agency under the appropriate extramural performer category (universities and colleges, other nonprofit institutions, or industrial firms).

   NOTE: Intramural activities cover not only the actual intramural R&D performance but also the costs associated with the planning and administration of both intramural and extramural programs by Federal personnel. Intramural activities also include the costs of supplies and off-the-shelf equipment, which has gone beyond the development or prototype stage, procured for use in intramural R&D. For example, the National Aeronautics and Space Administration's (NASA's) purchase from an extramural source of an operational launch vehicle used for intramural performance of R&D is reported as a part of the cost of intramural R&D.

2. Extramural performers are organizations outside the Federal sector that perform R&D with Federal funds under contract, grant, or cooperative agree- ment. Only those costs associated with actual R&D performance are reported, but these costs would include costs of materials and supplies to carry out R&D activities. Note, however, that the costs of off-the-shelf supplies and equipment required to support intramural R&D and procured from extramural suppliers are considered part of the costs of intramural performance and not part of the costs of extramural performance. Extramural performers are identified as follows:
   
   
1. Industrial firms: Organizations that may legally distribute net earnings to individuals or to other organizations.
   
   
2. Universities and colleges: Institutions of higher education in the United States that offer at least one year of college-level study leading toward a degree. Included are colleges of liberal arts; schools of arts and sciences; professional schools, such as schools of engineering and medicine, including affiliated hospitals and associated research institutes; and agricultural experiment stations.
   
   
3. Other nonprofit institutions: Private organizations other than educational institutions whose net earnings in no part inure to the benefit of a private stockholder or individual and other private organizations organized for the exclusive purpose of turning over their entire net earnings to such nonprofit organizations.
   
   
4. Federally funded research and development centers (FFRDCs): R&D-performing organizations that are exclusively or substantially financed by the Federal Government and are supported by the Federal Government either to meet a particular R&D objective or, in some instances, to provide major facilities at universities for research and associated training purposes. Each center is administered either by an industrial firm, a university, or another nonprofit institution.

   In general, all of the following criteria are met by an organization that is included in the FFRDC category:

1. Its primary activities include one or more of the following: basic research, applied research, development, or management of research and development (specifically excluded are organizations engaged primarily in routine quality control and testing, routine service activities, production, mapping and surveys, and information dissemination).
   
   
2. It is a separate operational unit within the parent organization or is organized as a separately incorporated organization.
   
   
3. It performs actual research and development or R&D management either upon direct request by the Federal Government or under a broad charter from the Federal Government but in either case under the direct monitorship of the Federal Government.
   
   
4. It receives its major financial support (70 percent or more) from the Federal Government, usually from one agency.
   
   
5. It has, or is expected to have, a long-term relationship with its sponsoring agency (about 5 years or more), as evidenced by specific obligations assumed by it and the agency.
   
   
6. Most or all of its facilities are owned by or are funded under contract with the Federal Government.
   
   
7. It has an average annual budget (operating and capital equipment) of at least $500,000.
   
   
5. State and local governments: State and local government agencies, excluding state or local universities and colleges, agricultural experiment stations, medical schools, and affiliated hospitals. (Federal R&D funds obligated directly to such state and local institutions excluded in this category are included under the "universities and colleges" category in this report.) R&D activities under the state and local category are performed either by the state or local agencies themselves or by other organizations under grants or contracts from such agencies. Regardless of the ultimate performer, Federal R&D funds directed to state and local governments are reported only under this sector.
   
   
6. Foreign performers: Foreign citizens, foreign organizations, or foreign governments, as well as international organizations (such as the North Atlantic Treaty Organization; United Nations Educational, Scientific, and Cultural Organization (UNESCO); and World Health Organization), performing R&D work abroad financed by the Federal Government. Excluded from the survey are U.S. agencies, organizations, or citizens performing R&D abroad for the Federal Government; the survey does not seek information on "offshore" payments. An exception in the past was made in the case of U.S. citizens performing R&D abroad under special foreign currency funds; these activities were included under "foreign performers" but have not been collected since the mid-1990s. Foreign scientists performing in the United States, however, are excluded.
   
   
7. Private individuals: For cases wherein an R&D grant or contract is awarded directly to a private individual, obligations incurred are placed un- der "industrial firms."
   
   
7. Federal obligations for research performed at universities and colleges, by detailed field of science: Only six agencies are required to respond to the portion of the survey covering the funding of research at universities and colleges by detailed field of science. These six agencies represent approximately 97 percent of the Federal research obligations to universities and colleges. The six agencies are the Departments of Agriculture, Defense, Energy, and Health and Human Services; NASA; and NSF.
   
   
8. Geographic distribution of FY 2002 R&D obligations.
   
   
1. Only the 10 largest R&D funding agencies are required to respond to the portion of the survey covering the geographic distribution of obligations for research and development and R&D plant. These 10 agencies accounted for approximately 98 percent of total Federal R&D and R&D plant obligations in FY 2002. The respondents are the Departments of Agriculture, Commerce, Defense, Energy, Health and Human Services, the Interior, and Transportation; the Environmental Protection Agency; NASA; and NSF.
   
   
2. Actual FY 2002 data were requested in terms of the principal location (state or outlying area) where the work was performed by the primary contractor, grantee, or intramural organization. When this information was not available in their records, the respondents were asked to assign the obligations to the state, outlying area, or U.S. offices abroad where the headquarters of the U.S. primary contractor, grantee, or intramural organization was located.
   
   
3. Obligations for basic research, applied research, and development were reported as a combined R&D amount for non-Department of Defense (DoD) agencies. However, DoD agencies break out development obligations by advanced technology development and major systems development. Therefore, obligations for basic research, applied research, advanced technology development, and major-system development were reported as a combined R&D amount for DoD agencies.

   Geographic distribution of Department of Defense development funding to industry reflects only the location of prime contractors, not the location of numerous subcontractors who perform most of the research and development.

4. Specifically omitted from the geographic portion of the survey were R&D obligations to foreign performers and support of foreign performers. Foreign performer data, by country, are reported in a separate section of the Federal Funds survey.
   
   

Changes in Reporting  

While completing the survey each year, agency respondents make revisions to their estimates for the latest two years of the previous report, in this case FY 2002 and 2003. Such revision is part of the budgetary cycle. From time to time, survey submissions also reflect reappraisals and revisions in classification of various aspects of agencies' R&D programs. When such revisions occur, NSF requests that the agencies provide revised prior-year data to maintain consistency and comparability with the most recent R&D concepts.

Changes in the Scope of the Survey  

The scope of the Federal Funds survey has changed over time, and the survey instrument has been revised accordingly. The most recent changes are described in the following paragraphs.

Since the volume 40 (FY 1990–92) survey cycle, DoD has reported research obligations and development obligations separately. Tables reporting obligations for research, by state and performer, and obligations for development, by state and performer, were specifically created for DoD. The additional detail provided by DoD highlights the following circumstances that are specific to DoD:

• DoD funds the preponderance of Federal development.
  
  
• DoD development funded at institutions of higher education is typically performed at university-affiliated nonacademic laboratories. These are separate from the universities' academic departments, where university research is typically performed.

During the volume 44 (FY 1994–96) survey cycle, the Director for Defense Research and Engineering (DDR&E) at DoD requested that NSF further clarify the true character of DoD's R&D program, particularly as it compares with other Federal agencies, by adding more detail to development obligations reported by DoD respondents. Specifically, DoD requested that NSF allow DoD agencies to report development obligations in two separate categories, advanced technology development and major systems development.

The reasoning behind DDR&E's request for the additional development categories is best explained by the following excerpt from a letter written by Robert V. Tuohy, Chief, Program Analysis and Integration at DDR&E, to John E. Jankowski, Program Director, Research and Development Statistics Program, SRS:

The DoD's R&D program is divided into two major pieces, Science and Technology (S&T) and Major Systems Development. The other Federal agencies' entire R&D programs are equivalent in nature to DoD's S&T program, with the exception of the Department of Energy and possibly NASA. Comparing those other agency programs to DoD's program, including the development of weapons systems such as F-22 Fighter and the New Attack Submarine, is misleading.

At several annual issues workshops held during FY 1992–96, NSF learned from survey respondents that there were certain survey items for which reliable data were difficult to obtain and report. As a result, NSF began to consider removing certain items from the Federal Funds survey instrument. The volume 42 detailed statistical tables publication was distributed with a flier notifying data users that NSF was considering eliminating several items from future volumes of the document. Data users were asked to review the list of affected tables shown on the flier, and to comment on the proposed eliminations to NSF.

Before publication of volume 43 (FY 1993–95) of the detailed statistical tables, NSF decided to remove from the document 54 tables that depicted data on two of the items slated for elimination: data for the special foreign currency program, and detailed field-of-S&E data for estimated outyears. NSF continued to collect data from Federal agencies for these items through volume 45 but eliminated the special foreign currency program and outyear detailed field-of-S&E lines on the survey instrument beginning with the volume 46 (FY 1996–98) survey cycle. A special flier was included in the volume 46 mailout packet that listed the data items that were no longer required.

NSF also decided to remove two tables depicting data on foreign performers by region, country, and agency before publication of volume 43 of the detailed statistical tables. These tables have been reinstated since volume 46.

Before the volume 48 survey cycle, the National Science Foundation's Division of Science Resources Statistics updated the list of foreign performers in the Federal Funds survey to match the list of countries and territories in the 1996 UNESCO Statistical Yearbook.

On November 25, 2002, President Bush signed the Homeland Security Act of 2002, establishing the Department of Homeland Security. The new department's preliminary funding for R&D for FY 2003 and 2004 are included in this report.

Limitations of the Data  

Funds for research and development were reported on a 3-year basis comparable with the 2002 budget, upon which the data were based. The amounts reported for each year, as already stated, are the obligations or outlays incurred in that year, regardless of when funds were authorized or received by an agency and regardless of whether the funds were identified in the agency's budget specifically for research, development, R&D plant, or some combination of the three.

The respondents reconciled the data reported to the Federal Funds survey with the amounts for R&D they reported under Max Schedule C to the Office of Management and Budget for the 2004 President's budget.

Some agencies are not able to report the full costs of research and development. For example, the headquarters costs of planning and administering R&D programs of the DoD (estimated at a fraction of 1 percent of the agency's R&D total) are excluded, because this agency has stated that identification of the amounts is impracticable.

R&D plant data are also underreported to some extent because of the difficulty encountered by some agencies, particularly DoD and NASA, in identifying and reporting these data. DoD's respondents report obligations for the R&D plant funded under the agency's construction appropriation, but they are able to identify only a small portion of the R&D plant support that is within R&D contracts funded from DoD's appropriation for research, development, testing, and evaluation. Similarly, NASA respondents cannot separately identify the portions of industrial R&D contracts that apply to R&D plant; R&D plant data are subsumed in the R&D data covering industrial performance. NASA R&D plant data for other performing sectors are reported separately.

Beginning in FY 2000, NASA reclassified space station as a physical asset and space station research as equipment, and it transferred funding for the program from R&D to R&D plant. Also, the National Institutes of Health (NIH) classified all of its development activities as research beginning in FY 2000. For more information on the classification changes at NASA and NIH, refer to the InfoBrief "Classification Revisions Reduced Reported Federal Development Obligations," NSF 02-309, February 2002, available on the Web at http://www.nsf.gov/statistics/.

Data Collection  

A Web-based data collection system (FedWeb) is used to collect the Federal Funds survey data. The FedWeb system is part of NSF's effort to enhance survey reporting and reduce data collection and processing costs by offering respondents direct online reporting and editing. Because the Federal Funds data are collected in electronic format, there is no paper instrument. Respondents provide data for their agency using similar data entry screens. The categories of funding are slightly different for Defense and non-Defense agencies. Mock survey instruments have been created to illustrate what information the respondent is asked to provide in FedWeb. See the Federal Fund's survey methodology report for further details on the FedWeb system.

Go to: Table of Contents    Suggested Citation, Acknowledgments    Section A. General Notes    Section B. Technical Notes    Section C. Detailed Statistical Tables    Appendix A. Agencies included in this Survey...    Appendix B. Notes Associated with Agencies Included...    Appendix C. Federally Funded Research and...

National Science Foundation, Division of Science Resources Statistics
Federal Funds for Research and Development: Fiscal Years 2002, 2003, and 2004
Arlington, VA (NSF 05-307) [April 2005]