Future Directions for Research
The suggestions for future research needs that emerged can be grouped in six categories: standardizing definitions and survey instruments, conducting studies at the departmental and program level, improving data systems (particularly to capture financial information and the "in and out" students), mining of existing data bases, longitudinal studies to track student cohorts through graduate school and post-enrollment years, and qualitative studies.
Standardizing Definitions and Survey Instruments
As previously noted, institutions vary greatly in how they categorize who is a doctoral student. Thus the workshop participants were unable to arrive at a definition of this key term that would overcome the institutional and departmental differences in practice. Similarly, the issue of when to count "leavers" as an attrition statistic was not resolved.
There were, however, some suggestions on how to make attrition studies more comparable. One suggestion was to convene an expert group that would come up with a list of critical questions that everyone should ask, along with suggested ways of asking them. Another was to convene a group to devise a study model. Although the model might not be followed, it would provide a basis for comparing studies and for understanding the issues involved in attrition research.
Departmental and Program Level Studies
Perhaps the most frequently mentioned need was for more data and studies at the departmental and program level. The reasons given were as follows:
- The graduate school experience is mainly shaped at this level
- Such studies would provide meaningful data for comparative analysis
- The information is specific enough to use for changing policies and practices at the institution where the study is done
Some universities already have conducted completion or retention studies focused on departments and programs. These studies and those that extract departmental and field information from existing databases (see below, Mining Existing Databases) would facilitate cross-institutional comparisons by field.
Improving Data Systems
Several speakers noted that universities vary greatly in their capacity for collecting and reporting data on graduate programs. The resulting disparities create barriers to cross-cutting analyses. Some general obstacles to improving university data-gathering were mentioned, however. These include shortage of resources for information systems, the small size of most institutional research offices, and the weakly perceived need within the institution for such detailed information. In addition, initiating research at the department or program level is seen, in some cases, as risky politically.
The quality of graduate school databases may be improving, however. Some institutions were reported to be working, in some cases with outside vendors, to develop integrated institutional databases. To take advantage of this development, one speaker recommended establishing a list of common data elements needed to study doctoral persistence. One university consortium (including the Big Ten and the University of Chicago) has agreed upon eight common data elements they will collect at the program level.
Mining and Merging Existing Databases
Comparing information from one database to another has the advantage of eliminating the effort of gathering original data. One presenter described how the National Center for Education Statistics (NCES) data could be used in combination with NSF's Survey of Earned Doctorates. The NCES data on annual bachelor's degrees in science and engineering granted by U.S. institutions would be used as the denominator or universe against which awards of doctorates from NSF's SED are compared. The result would yield comparatively good estimates of the annual percentages of bachelor's degree recipients who complete science and engineering doctorates.
Another approach is to merge databases-for example, the ETS file (see page 4) with the SED. To merge the databases in this way would, however, require some adjustments to the data sets. An example would be the need to harmonize different definitions of who is enrolled and still a degree candidate.
Apart from providing a broad view, the same merged files could also be used to compare completion or attrition rates by gender or by different degree fields.
Studies which track a group of students over time provide advantages over the prevailing cohort or cross-sectional studies being done on graduate school attrition. What they supply that the latter type of studies do not are data that reveal the timing or sequence of events that shape key student decisions in graduate education: their plans about persisting, dropping out and returning, transferring, or abandoning pursuit of the intended degree. Longitudinal panel projects are, however, costly. Typically they require at least 10 years of coverage, with data gathered at several intervals. In order to reduce costs, research designs should target fewer fields and institutions in the sample. Rather than making sample selection a "statistical choice", the focus for policy purposes should be on institutions, departments, or programs with successful retention records.
In order to track individuals over long periods of time, particularly after they complete their course work, researchers require permanent identifiers. Much of the information needed, such as Social Security numbers (SSNs) is confidential and often not obtainable. If, however, a researcher can obtain a file containing SSNs, NSF can have the National Opinion Research Center merge the files and provide the merged data, under certain restrictions, through a licensing agreement.
Using the qualitative method, studies would capture individual experiences in specific fields, programs, and departments. This contextualized information can then be used to promote policies that encourage persistence and equity in doctoral programs, advocates of qualitative studies argue. These studies are also costly, however, requiring labor intensive interviewing, observation, and analysis. They also encounter confidentiality issues.
 For an example of such comparisons, see William G. Bowen and Neil L. Rudenstine, In Pursuit of the Ph.D, l992 (Princeton, NJ, Princeton University Press).