We now have an hour to talk about future direction. What I thought I would do is to sum up some of what I have heard to be the major themes during the day. Hopefully that will stimulate some further discussion.
Broadly, I think that we started looking at three aspects of why we were interested and from what perspectives we might be interested. One is from the perspective of the science community. What are the needs? What is the structure of that community, the relationships of graduate education to the existing workforce, the dynamism in different fields? We need to think about this from the perspective of the scientific community itself. Then we need to think of it in the aggregate from the institutional or departmental perspectives, which will require aggregate information. We also talked a lot about contextual, individualized information, about the correlates and the consequences of attrition and what happens with individuals as they are passing through graduate school.
Another area that we talked a lot about in many, many different ways was diversity and complexity. We talked about the diversity of institutions; the diversity of fields; the diversity of experiences in different institutions and in different fields; and certainly the diversity of students according to their background characteristics such as gender, race, citizenship, age, and the stage of their career that they are in. The diversity of students was really a fundamental theme all day long, I thought. The diversity of student experiences within their graduate life. The diversity of funding mechanisms, of programmatic objectives that different programs in different institutions have; the diversity of faculty and how they treat students and how they conceive of what the graduate experience ought to be and how they translate that into action. We heard about the diversity of definitions and that certainly is very important for us in the enterprise that we are involved in (e.g., the different ways of defining when a doctoral student becomes a doctoral student and the different ways of defining how an RA or a TA should be working). Diversity came up everywhere and that adds great complexity into what we are trying to think about, but it also makes it a lot more interesting. Issues of equity certainly were an underlying theme and certainly were a strong incentive for the study of this issue.
Another theme that I heard repeated over and over again in many different ways and on both sides of the issue was the importance of both qualitative and quantitative information. We certainly need more information from a contextual, experiential perspective than we are able to get from large-scale sample surveys. At the same time, the qualitative approach needs the quantitative information to give a national perspective to the issues and to be able to quantify what is going on overall. Neither perspective is sufficient in and of itself, and we clearly have a need to marry these different ways of looking at the issues. We heard discussion about the relative merits of longitudinal and cross-sectional data.
We heard a lot about different sources of support and how support is a fundamental issue interacting with the issues of graduate student attrition. Sources of support relate to so many different aspects of what happens in graduate school, to the experiences that graduate students have. We heard about the relationship of sources of support to race and ethnicity and how different students may be getting different kinds of support by virtue of the specific programs that are not available to them. Sources of support relate to resume-building; we heard a little bit of that, that students with different kinds of support may or may not be building their resumes in the same way through graduate school. We heard about that related to mentoring. We clearly need to think about sources of support.
We heard a lot about data problems in existing surveys and administrative records systems. We heard that a lot of the data problems that exist the shortcomings of existing databases affect not only the answers we get, but also how we can frame the questions. We cannot even ask certain questions because of issues with databases. The kinds of issue are definitional. They are related to the typologies of how you categorize things; they are related to the sample frames that you are able to build, they are related to data processing and how that would carry down.
I heard that we know a lot and I heard Charlotte [Kuh's] question, "Do we know what we need to know?" I throw out a follow-up question to that: "What else do we need to know?" I would like to think about that question from the perspective of how the answer varies according to the audience that we are speaking to. There are audiences who are thinking about national policy; others are thinking about state policy. Universities are thinking about it from their own perspectives as are other kinds of institutions. The professional societies and other organizations are thinking about it from their own perspectives. When you talk about graduate student attrition, you have to keep very clearly in mind whom it is you are addressing. There is a national-level need for information for broad public policy. There is a state-level need for information to serve policy-setting within states and within their public institutions.
There is clearly a need for information according to discipline, according to institution and at the same time, cross-cutting all of those ways of thinking about things, you need information for students and for institutions. Those two units of analysis cut across all of the levels of information that you need.
Gwendolyn Lewis, U.S. Department of Agriculture: I just want to bring up a couple of variables that I think are a little bit hidden in what has been said so far today, and that I think are becoming increasingly important nationally. One is the part-time status of students and the other is the degree to which they are working in a non-university department related job and the number of hours they are putting into that job. I had a little conversation at work, in which I felt we have an elitist perspective thinking that students in the science and engineering fields of course are going to school full-time. Some of the earlier, very interesting qualitative data talked about a lot of variables that struck me as being implemented well only when you have students at school most of the time when school is the center of their activity.
Marilyn Urion, Michigan Technological University: While I realize all of this is very important, we need to know why people do not finish. At the same time, I am a little bit nervous as the discussion turns to time to degree. We talk about economic pressures, and I do not mean to pick on the AAU Study, but talking about the flow of doctoral student talent makes it a quantity, a commodity, instead of people. At the same time, there is such emphasis on interdisciplinary work, which notoriously takes longer. We have a new mathematics Ph.D. program at Michigan Tech, and one of its requirements is an internship, which will lengthen the period of time.
Part of my concern is that we will come up with a quantifiable model of who drops out, or is not likely to finish. When the economic crunch comes and the legislatures press us to be more efficient, especially with affirmative action under attack, we will achieve better statistics by being more careful about who we let in. That is where the policy of this gets to be very, very important. I wish we had heard more about that, but that may be something that has to come later.
Charlotte Kuh, National Research Council: We have mostly been focusing on attrition today, which is, after all, what we need to focus on. But there is a bigger picture in which attrition happens. That picture has to do with both the size of doctoral programs (there is a lot of talk now about oversupplies of Ph.D.s) and how in some fields we are training too many of them. When we think about attrition, we really might want to think about "Suppose we would limit the size of graduate programs," or "Suppose some graduate programs were to close." What effect would that have on, among other things, attrition?
Now I am going to sound like an economist, but price, quantity, and quality all are dimensions of adjustment. Attrition is clearly a way in which quantity adjusts, with implications for price, both the price that the student pays and the price that people who support graduate education pay. I have not heard much discussion of that. A number of these questions about attrition may very well hinge on whether or not we feel we are producing too many doctoral students. We might say the question is not how many are we producing, but who is paying for it; again, that may also have an impact on attrition. Attrition will be affected not only by the contextual kinds of variables that Willie [Pearson] and Barbara [Lovitts] and Toni [Clewell] were talking about, but also by these larger decisions about how well we are going to fund the system and who pays for it.
Participant: I want to respond to the question that was brought up about time to degree. I think it has something to do with attrition. One of the strengths of our system is that it is somewhat forgiving. One of the tradeoffs to being forgiving is that it is somewhat inefficient. People are able to mess up and still get through or are able to leave and then come back or are able to transfer from one place to another. It makes it harder to collect data and it makes it harder to have very strict institutional rules, but there is a very good strength there in terms of salvaging human worth and value and realizing the potential that people have. It is very important when we start talking about cutting time to degree and tightening up attrition rules, that we remember that this kind of looseness has a very important value to the entire system and to the people who are part of that system.
Barbara Lovitts, University of Maryland: When you start bringing issues to the policy level, the individual often gets lost. The fundamental question is: What is policy if it is not to make individual lives better? When you start looking at the national policy level you lose sight of the individual and the costs and the personal tolls that the completion of attrition has on them.
The other issue I want to talk about comes back to the issue of completion. I used two different surveys for my completers and non-completers; they were basically parallel but I used a different survey for completers to avoid taking them through some complex skip patterns. In order to keep the surveys as parallel as possible I did ask the completers the question, "Did you ever seriously consider leaving without completing the Ph.D.?" If the answer was no, then they skipped a huge section, and if the answer was yes, I took them through the same series of reasons for leaving that the non-completers went through, but the questions asked how seriously these factors influenced their thoughts about leaving.
More significantly, I took that cut question, "Did you ever consider leaving without completing the Ph.D.?" and I began analyzing the completers. I divided them into two groups, which I called pure completers and at-risk completers. Forty-two percent of the sample of completers was at risk of non-completion. There were a lot of significant differences between these two groups. The at-risk completers had profiles that were somewhere in between those of completer and non-completer. They were given more resources for completion than were the non-completers, but they were given fewer resources than the pure completers. They were less satisfied with their advisors than were the pure completers. They had an S-shaped pattern to their time to degree. Some of them left very quickly and some of them took a long time to leave. They were all likely to end up in the non-academic sector. While you are concerned about attrition, there is also another group that is sitting there on the fence. It would be worthwhile to look at them at the same time.
Carla Howery, American Sociological Association: I wanted to follow-up on Barbara Lovitts' comments and this other question because I think we probably agree that there are multiple reasons for attrition. There are people who by their self-definition want to be there, are capable of being there, and are having difficulty and may end up being in this at-risk group. So rather than speaking only of national policies, it seems to me that there are things that can happen at the departmental level.
The reason I am raising this is that I work at a disciplinary association, and I think that the disciplinary association has to think about its role in connecting with faculty who are our members and are representing our discipline on the campus. I would like people to speak to what has worked effectively or where they think there are opportunities, but I will mention one thing that we have done that I think has potential.
For some years, we, like many other associations, have convened our chairs; we see chairs as important disciplinary leaders, not as the people who drew the short straws. That person is at the intersection of the discipline in the institution and needs to make that interface as good as possible. In the past two years, we have convened another group, which is the directors of graduate studies, and they not yet been mentioned here. There is more ambiguity in that role than in almost any other one. Sometimes a person is secretary, maybe literally a secretary or a faculty member functioning as a secretary. Maybe they only handle a most small slice, admissions, but then they are off the hook. Maybe they handle the entire graduate career through placement whatever. There is incredible variety in the role, as well as variety of individual engagement in the role. But there is almost no consensus about what the expectations are, the role's importance, and what needs to be known in order to do it effectively and so on. We have come to see that person as really the departmental representative.
The director of graduate studies has an institutional role that buttresses, hopefully in a positive direction, the better advisors within a department. We have just started working with that person and thinking about what that role really could mean as a disciplinary representative in addition to the chair. I would be interested in what other sort of concrete solutions people think that the disciplinary associations can do with the faculty in departments.
Harvey Waterman, Rutgers University: I will not respond to that except to say let us hear it for the graduate program directors. My issue is to repeat something that Orlando [Taylor] said before. If the difficulties of collecting data from the institutions are what they are; if good studies of the course of a graduate student's career in graduate school, whether it means finishing or not, are to be done institutionally; and if we do not have a code of common practice that suggests some standard definitions of beginning time and who to include and categories of leaving and all the various things that make for systematic study, then we will end up with more institutions doing something akin to the Tower of Babel.
I would hope that at least the people in this room who work fewer than 10 minutes from the Potomac River might be able to get together at some point or form a committee, and to assign somebody the task of coming up with some kind of study model that can be done locally. It might be the beginnings of a national study as well. It will not guarantee that everybody follows it, but the people who do not follow it will know what to compare what they are doing to, and the people who are trying to do comparisons will know what the differences are. There will be some hope of having a general understanding of the issue in addition to the local understandings that are useful in and of themselves.
Chris Golde, University of Wisconsin-Madison: In the research I did, one of things I found interesting was the departmental policies and practices. So often, the program is structured in a particular way because "that is the way we have always done it." No one has the faith, the goodwill, or the time or the energy to step back and think about what we are doing what the impact is on students. Do we have a common core of courses in the first year? What are the reasons we have that set of experiences as opposed to a different set? Does it introduce students to what it is really like to do research in this field and this department, or is this just really replicating what they did as undergraduates? Is this about weeding students out in some way that is not very useful? How do they get connected with an advisor? What are our processes for introducing students to a lot of advisors? Is it a room where every one sits and someone comes in and does a little parade and little slide show, or do you spend 10 weeks working in a particular person's lab to really understand that person's working style? That is done a lot in the biological sciences, and seems to reduce attrition and it helps to make a good match. Whatever this model study might be, I think is really marvelous idea, and I hope that we would look at some of those kinds of things too.
Paul Seder, National Institutes of Health: I am not convinced that early attrition is a bad thing. In fact, the important thing may be to convert later attrition to earlier attrition. If we saw greater earlier attrition (and it is not just a matter of resources that the university spends and of fellowships), then we could be talking about years of productive employment for some of the most talented people who were in the wrong place. They may spend 10 years in the wrong career when they could be in some other career and if we could show them that or work with them in the first one or two years, that is an enormous benefit.
In some sense, it is a local management problem at the department and the university level. I am not sure the Federal government can have much influence on that. Maybe the professional societies can do more in terms of getting a handle on some of the processes. Perhaps the National Institutes of Health should put more money into these small-scale/medium-scale studies done at the universities by people like Barbara [Lovitts] and Vincent [Tinto].
Getting a handle on national data sounds like somewhat of a patchwork of existing systems. ETS's work sounds like an exciting thing. We may in fact have enough to get some handle on this. I think we have some ideas, but it is important to try to encourage local universities' departments to manage better in order to to figure out what is going on. There is a general trend toward much more accessible computer software and databases that more universities' departments might use to routinely track these kinds of things. It may be helpful to give them some standards, so then we could pull them together at some later point. Providing these things on the Web may help. It is not clear to me what public policy, what Congress could do other than maybe requiring at the graduate level something like what they do on the time to degree at the undergraduate level. I am not sure what else Federal policy otherwise would do to handle the attrition problem.
Catherine Millett, University of Michigan: In May at the University of Michigan, I participated in a conversation with the director of institutional planning, and with a representative from the graduate school. They were preparing for a conversation that was supposed to take place at the Association of Institutional Researchers. They were beginning to think what would it be like to participate in a data collection project that would go across institutions. I am not sure if at that point they were limiting it to the AAU institutions or not. I am not quite sure what happened to that.
Gerard Crawley, Michigan State University: The graduate deans at the CIC institutions, the big 10 and the University of Chicago, for a couple of years have been talking pretty much along the lines that Harvey mentioned, trying to compile a database of graduate information. We have been meeting and the institutional data people have been involved in that to some extent as well. We have actually begun the process. We have selected, I think it is eight data elements that we have agreed to collect.
The important thing is that we have agreed to collect the data at the program level. In many of the institutions there are 200 or more different graduate programs, so we are collecting the data at that level of disaggregation. You can then later on aggregate the data if you choose to do so. We are collecting the data by gender, ethnicity, and visa type. The data are on a server at Michigan State. We have begun collecting enrollment data; this is sort of the first data element. I think it will be interesting after six months or so to see how far we have gotten with that and how much of the data we have in common. We now have data on enrollment at eight institutions. I think that is probably related to the kind of thing you are talking about.
Catherine Millett, University of Michigan: The other point that I wanted to make is that in a past life, I spent eight years working with students when they were thinking about leaving, at an institution focused on undergraduates. We had a form that they had to fill out. Having to fill out a piece of paper and take it to someone was a big step. We would ask people, "Why are you are leaving?" It was kind of a five-minute, quick and dirty, "Tell me why you are leaving, tell me when you think you are coming back, and we will flag you in our records." Then they would go off and have very pointed conversations with people saying why they were leaving.
Some of it was very easy. Their responses would be, "I have got a great opportunity to go work on a film," or "It is Olympics year and we have got to go. But then, some of them were much harder: "I cannot afford this" or "I do not fit in." It is hard to say that to people about why you do not think you can succeed. We need to take a little bit more account of that. It is hard to say that you are not measuring up. On the other hand, it is very hard for faculty or program administrators to tell somebody, "You are not measuring up." We have to figure out how to have that conversation without having some of what Barbara [Lovitts] seems to be saying are the devastating outcomes. In the program that I am affiliated with at the moment, I do not think they tell you that you are not cutting it until about your qualifying exam, which I think is a little late that all of the sudden you are being told this is not quite what we expect. It is a two-way conversation. It is very hard, but I think we have to figure out how to have it.
Boris Iglewicz, Temple University: I have three quick comments. Advising a fair number of students personally now, I really should invite them to the house to dinner and treat them nicer. Many of us get so busy seeing students in the office that we forget that sometimes taking them out, inviting them home, asking what their problems are, getting to know them, is really important.
The second comment: We have research assistantships and teaching assistantships, but we also have fellowships maybe one fellowship to a star student. Maybe NSF or the university should take away the fellowship entirely and give $2,000 more to all of the assistants so they get, instead of $10,000, $12,000 a year to make their lives a little bit more comfortable. As was mentioned, with inflation $10,000 is not what it used to be.
The third comment: We talked a lot about students being in a program for a long time and if they do not get the Ph.D., it is a total failure. That is field-related. In some fields like ours, where the job market is quite nice, we have a very low attrition rate, but we do have an attrition rate. Some people can get As in courses, can pass exams, can do a dissertation, and can do research. That is a fact of life. Now what happens to those students? They have phenomenal jobs already lined up and they will do very nicely, and the education they received is very valuable for them. They will be very productive people, but they may not get Ph.D.s. In some areas like statistics, the education may be wonderful, and the person may not have a Ph.D., but may be doing wonderfully.
Barbara Lovitts, University of Maryland: First, I want to address the issue of asking students about the reasons for leaving. I think it is important, but students do not leave for one reason. They leave for a constellation of reasons. I had students offering up to six and seven reasons for their attrition; even when I asked them to tell me what clinched their decision to leave, they still would not give me one single reason for leaving. The other thing about leaving is that although they can give you reasons for leaving, students might not really know what the reason is.
Vincent Tinto in his 1987 book made a point that stayed with me when I read it. When students tell you they left for financial reasons, it is often an ex post facto explanation when something else was happening. If you look at my data, not all, but a lot of students leave for lack of integration. Academic and social integration is not a concept that we carry around in our heads. We cannot say I left because I was unintegrated. I can say that but not very many people can. I mean this in a real positive way. I know people are going to laugh but, when I read Vincent's book back in 1988, it upset me. It was identifying for me for the first time why I did not make it through graduate school the first time. I mean, I had major disillusions, positive but negative at the same time because for the first time it was saying, this is what happened to you.
Students do not necessarily know all the factors that are affecting their attrition. I do think that it is important to ask them. Obviously, it does give you some insight, but there is also this social context surrounding their attrition that they may be unaware of.
Picking up on Chris [Golde's] comment relative to department context, I did interview the directors of graduate studies (DGS) about the integration structures in the departments. I also asked them a series of questions about how they help students come to understand different processes in their departments. One of the major findings was that departments really assume that graduate students understand a lot more about the structure and process of graduate education than they do. It is like two ships passing in the night between what they expect of students and what students expect of them. There should be some way for the directors of graduate studies to get together and share best practices on how to integrate students, how to help students develop cognitive maps to their graduate programs.
I will make one last point. When I was walking around the departments, a lot of the graduate departments had picture boards of their new graduate students and their current graduate students. My favorite one was a department that had black-and-white individual portraits of their students they were framed by class, going back to 1987. This department had an 86 percent attrition rate. What struck me in thinking about it many months later was that no department celebrates its successes. No department has pictures of students in their caps and gowns. No department has lists of names of graduate students and their advisors on public display in the department. The closest you will get might be copies of students' dissertations, but nobody's celebrating the successes. The closest thing I did see in one division of one department was a map of the United States with push pins for where their students had gotten jobs. That is important, and maybe departments should start thinking about doing it.
Carolyn Shettle, Division of Science Resources Studies, NSF: I know people who have left voluntarily before getting the doctorate degree, who did so for personal reasons but also because of what they wanted to achieve. I am talking about people who were even into the dissertation writing stage. They did not need it. That is a very different situation than what Barbara [Lovitts] is talking about. We cannot tell those different experiences just from looking at the aggregate numbers or just talking to a few people C. Neal Tate, University of North Texas: You talked about talking to students and interviewing them when they are leaving. How do you do that? I do not know how we are supposed to grasp these students when we do not know they have left until two or three years after they are gone. What kind of system is it that you are talking about? Literally, how does this happen? Why do they have to have permission to leave?
Participant: Until they get permission they cannot leave or enroll the next year.
C. Neal Tate, University of North Texas: What if they just do not pay their bills and do not register?
Participant: Some institutions have pre-registration and people have to submit cards back. The people who do not return the cards may receive a random stratified phone call. That is just one way but it is not perfect.
I have a question of you, Jeanne [Griffith]. You mentioned a "what works" conference. Can you share with us your thinking about that conference, how would you go about determining what is working? How do you plan to share this very important information with people in the field who need that information?
Jeanne Griffith: We are working with the University of Wisconsin to put together a conference next spring about innovative programs in universities and in departments. As far as sharing it, we are planning to produce proceedings of the meeting. We hope to have a fairly broad representation of people at the meeting. It is part of what we call the Graduate Education Initiative, in which we are trying to pull together information in a variety of ways.
I mentioned some of the data collection changes that we have been making and some of the other workshops that we have been holding. All of this is part of the Graduate Education Initiative. It was stimulated by reports from the several places, including the National Science Board, that are interested in trying to examine how we should be supporting students. They are looking at how we are supporting students, whether we are doing it the most effective way, and long-term information collection, and are thinking about the graduate experience and what needs to be done.
Participant: Will this be produced in a form that is published, that institutions can buy?
Jeanne Griffith: It will be produced in a form that is published but I do not know whether it will be for sale or whether we will just be distributing it.
Participant: At the undergraduate level, those conferences are always very well attended. We can wait for the research, but at some point people need to know what to do. That is a very important conference.
Jeanne Griffith: It clearly is an issue that many people are very concerned about.
Janice Madden, University of Pennsylvania: Michael Neuschatz started talking about what has been the successes of American graduate education, that it is personalized and forgiving. There is another hallmark that I think is very important and to my surprise has been absent from the discussion today. The success of the American research doctorate has been the ability to really provide it almost as a free good as part of our overall research mission as a nation. We produce our Ph.D.s in the process of producing the basic research itself. The faculty at the largest research doctorate programs are extremely competitive with respect to research and the production of research results. That is their life blood and that is what has been behind the success of the American science, and even very much the success of American graduate education.
It is also the case that faculty members work longer hours than any other profession in the country, including Wall Street lawyers. So if we are talking about having them do other things to change attrition, we have got a hard sale to convince them that in their already busy lives in some way this is where the interest of American sciences is headed.
A lot of this discussion has to be put in the context of American science and how we make American science better, either immediately in terms of attracting the very best undergraduates to Ph.D. education or in the long term in the future generations of scientists. That ultimately is going to be much more convincing at the disciplinary level than it ever will be from a graduate dean's office. I will have much more luck as I try to get my graduate chairs to really think about these issues if I can point to a body of evidence being collected in their disciplines that shows that if they have high attrition rates they are not getting the best students. Then ultimately it is going to be affecting their ability to do the research they want to do. We have to think in terms of the affect of high attrition on the graduate programs and on the research programs themselves.
Bertha Holliday, American Psychological Association: I want to share with you a few things that we are doing related to retention and attrition of ethnic minorities at the graduate level. I am not quite sure if there are any really good data on retention of minorities at the graduate level. Data at the undergraduate level suggest that the attrition rate is almost double for Blacks and Hispanics compared to that for Whites. If you think you have an attrition problem for Whites, then there is truly an attrition problem for persons of color.
The challenge for us was to figure out how to make that issue of attrition of minorities relevant to the discipline of psychology to make it a problem or an issue for the discipline. One of the ways we did this was by really taking a futuristic approach in terms of the future market for the services of psychologist, both research services and human delivery services. In taking that look at the future, it became obvious to us that it was imperative for the discipline itself to be more responsive to the needs for creating greater numbers of minority practitioners and scientists in psychology. To do that, to assume that responsibility as a discipline, we also had to be willing to recognize that there were some changes in the values and morals we had historically socialized our students that possibly needed to be changed and needed to be changed in a way that is more responsive and affirmative of the talents and the interests of students of color.
Based upon those kinds of premises, we developed a grand proposal to the National Institute of General Medical Sciences, and they bought it. We are trying to create five regional centers of excellence to demonstrate ways to effectively try to not change but shift, enrich, and strengthen departmental cultures in ways that would be supportive of minority graduate students and decrease their attrition. This would help all the students in the long run.
There are 15 institutions involved in this project. One of the things we are asking departments to do is to begin, since we are behavioral sciences, to engage in some new types of behavior to begin to collect new types of data that they have not collected before, to discuss the meaning of those, to begin to talk to some people at some different types of institutions that they have never talked to before, to begin to think about the concept of mentoring and how it can work in consideration of research schedules. I just wanted to share that with you and will be happy to share any information that we get from that project.
Jeanne Griffith: Thank you. I want to mention that there are going to be proceedings prepared from this meeting that will be placed on our Web site. There is a card out there with our Web site address on it and I believe the proceedings from the previous professional association meetings are also up on the Web. Thank you all very much for coming today. We are very grateful for your time and interest and we look forward to seeing you again.