U.S. Academic Scientific Publishing
1.0 Executive Summary
The publication of research results in peer-reviewed scientific journals is a key output of scientific research. In an unexpected development in the early 1990s, the absolute number of science and engineering (S&E) articles published by U. S. based scientists in the world's major peer reviewed journals plateaued while resource inputs—funds and personnel—kept increasing (figure 1).
The unprecedented plateau in the number of U.S. S&E articles should not be confused with a decades-long and familiar decline in the U.S. share of the world's S&E articles. As other states built up their S&E capabilities, the U.S. share of the world's articles in natural sciences and engineering dropped from 38% in 1973 to 28% in 2003. In an earlier report, "Changing U.S. Output of Scientific Articles: 1988–2003," NSF described the dimensions of this "plateauing" and compared trends in the United States with those in other countries and regions that are major producers of scientific articles.
Within the U.S. scientific community, the academic sector is critical to the overall health of the nation's research system. University-based scientists generate the most publications and, arguably, conduct much of the most important and innovative research (figure 2). In the course of this work, they train generations of new researchers, and they help to attract and retain talented scientists and engineers from around the world.
The implications of trends in article production and citation frequency are open to debate. Although publications and citations are indicators of productivity and influence, they are far from perfect indicators. The trends are worthy of attention because they indicate a marked shift from a historical pattern. Although the data might be taken to suggest that U.S. productivity and influence are declining relative to those of other major research-producing countries and regions, other interpretations are also reasonable.
However, the purpose of the analysis in this report is to examine the patterns and trends in the U.S. university sector in greater depth, not to make a judgment about its interpretation or significance. Specifically, this report addresses scientific publication trends in the top 200 U.S. academic R&D institutions, as measured by their 1988–2001 research expenditures, since these institutions produce most article output from the academic sector (figure 2). Such concentration of publications is not surprising because research is central to the overall mission of the top 200 R&D performing academic institutions. Many of these institutions achieve or aspire to worldwide recognition as research leaders.
1.2 Research Objectives
The present report examines quantifiable relationships among publications, resource inputs and institutional characteristics in the top 200 R&D performing academic institutions. It addresses four principal research questions.
In addressing these questions, our fundamental framework is that R&D funds, R&D personnel, and research infrastructure are instrumental in generating publications in S&E fields. In developing this framework, we are limited by the quantified measures that are available in our database. Additional unavailable measures, such as reliable institution-by-institution counts of S&E research faculty, would allow for substantially different analyses, especially regarding the association between article production (see Exhibit 1) and particular kinds of resource inputs or individual institutional characteristics.
At the end of this executive summary, we provide a list of limitations that should be borne in mind in interpreting the results of this study.
1.3 Resource Inputs, Institutional Characteristics, and Article Production
Variations among universities in research resource inputs (for which quantified data are available) largely account for differences in these universities' publication outputs. The primary measured resource inputs associated with publication counts across institutions are total academic R&D expenditures, the number of S&E postdoctorates, and (to a lesser extent) the number of S&E doctoral recipients (which may serve as a surrogate measure for the number of doctoral candidates available to assist in research). These three resource inputs account for over 90% of the variability in publication outputs, leaving little residual variability to be explained by other potential factors.
Analyses of the four different publication measures, defined by fractional or whole counting and a fixed or expanding journal set, yield very similar results (table 1). Likewise, analyses of the corresponding four different measures of citations yield results that corresponded closely to those of the publication measures. In short, essentially the same explanatory variables turned out to be important for each of these measures.
A difference between institutions of $1M a year in total academic R&D funding is associated with a difference of 4.8 whole count publications in the expanding journal set (table 1). A difference of one postdoctoral student is associated with 1.7 additional publications, and a difference of one S&E doctoral recipient is associated with 1.5 additional publications. (Since the number of S&E doctoral recipients is highly correlated with faculty counts, these publications may reflect the efforts of faculty who taught these doctoral recipients).
Within the category of total academic R&D expenditures, funds obtained from the federal government and other institutions such as foundations appear to be somewhat more influential than funds from state and local governments or private enterprise.
Examination of a number of other factors that might influence publication counts failed to markedly improve the results of the analysis:
These results suggest that once we take into account total academic expenditures, the number of S&E postdoctorates, and the number of S&E doctoral recipients, other factors not captured by these variables do not seem to matter much for article production. These intangibles include university prestige, unmeasured infrastructure, and unmeasured student or faculty quality. There are a number of possible explanations for this finding:
1.4 Variation Over Time and Among Kinds of Institutions
The primary inputs for producing publications, R&D expenditures and personnel, have generally increased faster than the number of publications. For example, the amount of inflation adjusted R&D expenditures per fractional count publication increased by 29% between 1990 and 2001 (figure 3).
This pattern of increasing amounts of inputs required to yield the same publication outputs occurred across the entire U.S. academic system. It did not appear to vary meaningfully across different kinds of institutions, and it cannot be explained by factors specific to certain regions, quality levels, or other segments of the system. Possible reasons for the increasing inputs per article include a rise in complexity of research required for publication; costs for faculty, S&E postdoctorates, S&E doctoral recipients, and research materials and equipment that are increasing faster than the GDP implicit price deflator; increased communication costs for integrative collaborations, etc.
A previous SRS report, "Perceptions of Academic Researchers and Administrators," considers these and other factors that might bear on changing article output. The report, based on qualitative data from interviews and focus groups, summarizes the views of experienced observers and practitioners in research universities about how the worlds of academic science and engineering research and publication have been changing over the past 15 years.
1.5 Relationship Between Changes in Institutional Characteristics and Changes in Article Production
Changes over time in resource levels available to a given institution appear to affect publication output by a smaller amount than would have been expected based upon the relationship between publication output and resource inputs across institutions (table 2).
These smaller than expected increases in publications output may reflect that the three key inputs, academic R&D expenditures, S&E postdoctorates and S&E doctoral recipients, are themselves in part surrogates for other, slower or unchanging university characteristics that in turn are related to publications volume. Alternatively, it may suggest that publications exhibit some degree of inertia to "short term" changes (which may span years).
While the relative level of publications of two or more institutions can be fairly reliably estimated based on the levels of these three inputs, there is considerable uncertainty concerning the change in fractional count publications that results from a change in resources for individual universities. This implies that a substantial portion of year-to-year changes in institutional outputs may be affected by unmeasured variations in inputs.
The analysis was nevertheless able to account for a substantial portion (about two-thirds) of the within-institution variability in publications over time, as measured by whole counts in the expanding journal set. Results suggest that changes in federally financed academic R&D expenditures have almost three times the impact on publications measured in this way as non-federally financed academic R&D expenditures.
In addition, the type of postdoctoral student funded has a differential impact on changes in such publication counts, with S&E postdoctorates without M.D.s increasing publication counts, S&E postdoctorates with M.D.s funded by federal traineeships (which often emphasize clinical training) decreasing publication counts (presumably by redirecting resources from research to non-research oriented activities), and S&E postdoctorates supported by federal research grants having little effect on changes in publication counts.
1.6 Variations in Relationships For Different Fields of Science
The three publication input variables that best explained institutional-level publications output—R&D expenditures, S&E postdoctorates, and Ph.D. recipients—also account for about 90% of the explanatory ability within each of five different fields of science (medical sciences; computer sciences; psychology and social sciences; biology, life and agricultural sciences; and engineering, math and physical sciences, including chemistry, geosciences and astronomy).
A few additional variables slightly improved this model's fit in the five individual fields of science. Those variables included Carnegie R-1 classification (which was associated with increased publication counts), basic research expenditures by institution (which may reflect the orientation of the institution and was associated with increased publication counts), and the number of S&E postdoctorates without M.D. degrees (which may reflect lesser number of publications per S&E postdoctorate with an M.D. degree, with the possible exception of postdoctorates with M.D. degrees who are supported by federal research grants). The precise effects of the explanatory variables differed by field; for example, S&E postdoctorates play a more prominent role in research in some fields than in others and tend to be clustered in some fields.
However, dividing article outputs by field of science, using field-specific input data, and then summing across fields did little to improve this model's fit. It also does not appear that institutional variations in output are substantially related to gross features of disciplinary concentration.
The trends in inflation-adjusted R&D expenditures per publication are not the same across the fields of science (figure 4). Increases in R&D expenditures per publication were greatest for medical sciences and the field consisting of biology, life and agricultural sciences (greater than 30%) and intermediate for the field consisting of psychology and social sciences (24%). R&D expenditures per publication in the field encompassing engineering, math and physical sciences increased 15% from 1990 to 1997 and then reversed itself, resulting in a net increase of 9% from 1990 to 2001. R&D expenditures used per publication in computer sciences followed an erratic pathway, resulting in an increase of 12% over this time period.
1.7 Study Limitations
Results from this study should be interpreted in light of the following limitations:
 During this same period, publication output in the rest of the world increased, but a comparison of U.S. and non-U.S. publication trends is outside the scope of this report.
 National estimates of S&E academic researchers are based on a sample survey of S&E doctorate holders that cannot reliably be aggregated to produce institutional counts, lacks information about those with non-U.S. degrees, and provides no information about researchers who do not have a Ph.D. A reliable count of total faculty at each institution was available and was used in our analyses, although that measure also had many limitations.
 S&E fields consist of the life sciences, natural and physical sciences, engineering and technology, mathematics, psychology and social sciences and related fields.
 All publications and citations data derive from the Science and Social Science Citation Indexes maintained by Thomson ISI, with special tabulations provided to NSF by ipIQ, Inc.
 Our model associates an additional $1M in academic R&D expenditures with an additional 2.5 S&E postdoctorates and 1.4 S&E Ph.D. recipients. These additional personnel are associated with 6.2 more whole count publications. Consequently, $1M in additional R&D expenditures results in 11.0 additional whole count publications if combining the direct (+4.8 publications) and indirect effect (+6.2 publications).
 Funds from state and local government and industry might support research with a more applied component and a lower expectation of publication.
 These three include the entire University of California system, which does not report patents for individual campuses, thus further weakening the evidence.
 But the link between NRC quality ratings and citations relative to publications suggests an additional dimension of quality or influence of publications outputs.
 A regression modeling the relationship between the number of fractional count publications (based on inflation adjusted academic R&D funding, number of S&E postdoctorates, and number of S&E doctoral recipients) assuming a constant relationship between resources and publication production will underestimate the observed number of publications produced in 1990 and will overestimate the observed number of publications in 2001. From the amount of under- or overestimation, it can be calculated that the same resources that produced 100 publications in 2001 would have produced 129 publications in 1990.
 The number of institutional authors per U.S. S&E article increased from 1.78 in 1988 to 2.44 in 2003. During this time period the number of author (person) names per U.S. S&E article increased from 2.98 to 4.42.
 The increase in resources used per publication does not appear to be related to a change in the proportion of articles appearing in higher quality journals. The NSF study classifies a journal as being among the top 5% in terms of citations in 1992 if the articles that appeared in that journal in 1988 through 1990 had more citations in 1992 than articles published in 95% of other journals that appear in the expanding journal set. For the top 5% cited journals in 1992 NSF has calculated that 61.0% of the articles written in those journals in 1988 through 1990 have U.S. authors. In contrast, the U.S. share of the world's articles in the top 5% cited journals in 2003 decreased to 50.5%. The proportions in 1992 and 2003 are almost identical using the fixed journal set. During these years the U.S. share of all S&E publications decreased from 37.8% to 31.1%. Thus, the U.S. share of both total articles and articles appearing in the most highly cited journals decreased by approximately the same percentage (i.e., 17%–18%).
 Based on the 1994 Carnegie Classification of Institutions, these universities have a full range of baccalaureate programs and a commitment to graduate education through the doctorate, and they award at least 50 doctoral degrees a year and annually receive $40 million or more in Federal research funding.
 For example, suppose that there is an unmeasured underlying factor that causes a university to be both effective in generating publications and obtaining academic R&D funding. In this case, the academic R&D funding itself does not cause the publications to increase, and an experiment that doubles R&D funding without changing this underlying factor might be relatively unsuccessful in increasing publications.