SRS Professional Society Workshop

DATA NEEDS FOR ASSESSING THE RELATIONSHIP
BETWEEN GRADUATE EDUCATION AND EVOLVING TRENDS
IN THE S&E LABOR MARKET

SESSION II

SESSION II: CAROLYN SHETTLE, CHIEF STATISTICIAN, SCIENCE RESOURCES STUDIES, CHAIR

Professional society data relevant to the policy issues surrounding graduate education and its relationship to trends in the S&E labor market

Mike Neuschatz, American Institute of Physics
Mary Jordan, American Chemical Society
Cathy Gaddy, Commission on Professionals in Science and Technology

Association of American Universities/Association of Graduate Schools Project for Research on Doctoral Education

Rocco Russo, Educational Testing Service

SESSION II
PROFESSIONAL SOCIETY DATA
Shettle:
The speakers this morning were very helpful in setting a policy framework. We will turn now to representatives from the professional societies who are going to talk about the data we have to address the policy issues raised. There are four main questions that those of us who are data providers ask when we deal with policymakers interested in empirical information.

First of all, we like to ask, "Are there existing studies and analyses that answer the policy questions?" That's the least labor intensive approach.

Second, if there are not existing studies, are there at least existing databases that can be better mined to answer the questions that have been posed by the policymakers? If the answer is still no, then, we get into the very complicated area of asking what do we need to do to collect the desired information, and a corollary to that, especially in these tight budget times, are the identified data collection efforts feasible, feasible from a methodological perspective and from a cost-effectiveness perspective.


Sometimes we can't get information because we would have to intervene in programs too much. For example, we can't do some random experiments we would like to do, such as randomly assigning fellowships to certain students to see what happens 20 years later.

But also we need to consider feasibility in terms of cost effectiveness. If it's going to cost us a million dollars to study a program that costs $3 million, it's not going to work.

The data providers will be discussing these issues, as we respond to those people who have so ably set out some of the policy issues for us today.

I am going to turn now to Mike Neuschatz from the American Institute of Physics who will be our first presenter. I think those of you who have been coming to these sessions know that AIP is one of the premier data collectors among the professional associations.

Neuschatz:
We may have a lot of data. The question is how relevant are they to the issues.

Shettle:
A key question.

Neuschatz:
With that introduction, I guess the point to start out with is that we don't conduct surveys that specifically target policy questions. There are probably a lot of reasons for that -- and some of them are obviously political.

We have surveys that gather background data and surveys that nibble at the edges of the policy questions. We kind of sneak questions related to policy issues into the other surveys. We have found that that is the best and most direct way we can get answers to these questions.

One of the sad things is that often by the time we get around to asking the questions, the policy issue has changed and what we have is a retrospective look at what would have been the policy if things had stayed the same, which they rarely do.

I am going to start with a recap of some of the standard data that is background to the policy questions. That is very important and we talked about it when we went around the table. Next, I'll mention some of the more direct policy questions that we have tried to include in our surveys and some areas where there are pretty big gaps, that we hope that we or other groups can address over the coming years.

The first thing is the projection of Ph.D. numbers. This is based strictly on a projection of first year students. The thing that is significant here is that the ground underneath us is constantly shifting. Right now, we are on a high plateau in terms of Ph.D. production; however, we seem to be looking over a precipice. If we sit here five years from now and talk about the policy questions relative to graduate study and employment, some of the questions may be reversed, at least in physics.

Kruytbosch:
This is based on asking questions of first year graduate students in physics?

Neuschatz:
It is based on asking universities their first year enrollment.

Fechter:
Are you assuming a stable retention rate of first year students?

Neuschatz:
Actually, no. When I did my projections, I looked at previous periods of plateaus and declines, and I projected based on those rather than on the immediate preceding years, because I reasoned that the attrition rate would change during the period of rapid decline. However, it still involves estimates, projections, and guesswork. No matter how finely you refine the coefficients, it is still going to be a guess.

Kuh:
What is going to change when you go off the edge of the precipice?

Neuschatz:
I think issues of supply. Much depends on what is seen as driving the enrollment up to begin with. That is obviously a point of some contention. Future developments will hinge on whether the things that were driving enrollment up were deep seated, structural needs, and, if so, from where in the structure they stem -- did it involve primarily changing availability and characteristics of the students who were applying or changing needs of the "consumers" of graduate students, i.e., research professors, departments and, in the end, employers. On the other hand, if there has not been a clear cut shift in those motivating factors, you can switch from a period of surplus to a period of shortage relatively quickly.

I am not saying that is what is going to happen, because there are complicating factors, and here is one of them. This is our data asking graduating students, what are your plans after your degree. Clearly, there is a long term trend of rising postdocs. You will notice I am using U.S. citizens only here. The overall curve rises even more steeply, because foreign students increased their representation throughout that period and were more likely to take postdocs.

But even when you factored out foreign students, you had a rise from below 40 percent in the early eighties to around 60 percent recently. However, we have seen a downturn during the last two years. This is not necessarily the beginning of a trend, but at least it is a temporary reversal of a long term trend.

Now, why is that happening? One reason might be improving labor market conditions. I alluded to this when we were going around the table. In fact, in '93, there was a huge increase in the proportion of people who said that they were unemployed or did not have degrees, both at the time of degree and when we did our follow up six months later. Things looked a little better for those students from the class of '94. While there may be some improvement in the unemployment rate, we don't know what kind of jobs these are, and to say unemployment by itself is a reasonable and accurate measure of the situation for graduates, I think, is a very severe error. It is crucial to know what does the employment consist of.

The other part of the equation is the number of postdocs. Some indications we have gotten suggest that that number of postdocs may have risen to above 3,000 by now. That is an interesting thing that is happening. The number of people hitting the job market after coming out of a postdoc is probably on the order of double the number who come directly from their Ph.D.

If you want to think about it as the end of the pipeline, it is almost like the pipeline now has a new extension, and the spigot at the end of the pipeline is not any more at the point of Ph.D., it is several years beyond, and it varies according to how many postdocs people take, which will also affect the overall number at any given time.

If that is true, the measures that we have to gauge where people are going after their degree are becoming outmoded by a change in the structure. In other words, we were asking people within the first six months of their degree, and it may be that six months after their degree many of them are still in a holding pattern and they really won't have their outcomes confirmed for another one, two, or three years, or even more, when they emerge from their postdoc.

There were many people who took postdocs before for reasons that were entirely consistent with further graduate training. They did it voluntarily. So, I am not trying to imply that all people who take postdocs are doing it because of a distressed market. However, if there is a relatively rapid run up in that proportion at the same time as the job market is deteriorating, it is probably reasonable to conclude that at least a hefty proportion of that run up is due to people who are extending their career both to polish their CV and also to wait for conditions to improve.

We do a study of our society members and ask people who took postdocs why they did so. I would expect that people who are still in a postdoc four or five years after receiving their doctorate feel that is a very long time to be in a postdoc, and are probably in there involuntarily. But what is interesting to me is that even for the more recent doctorates a hefty proportion take a postdoc, not to further or broaden their education, but simply because they couldn't find a permanent job.

Kruytbosch:
Do you have the number of people in each of the categories there? At two years out, it looks pretty flat, and then it starts to go up rather sharply.

Neuschatz:
About 10 percent of the people in this cohort are in postdocs several years out.

Fechter:
It seems to me that the second and third year is kind of a cut point between first and second postdocs and you are really getting a whole different set of people in those two samples.

Neuschatz:
I agree, which would just reinforce the idea that by the time somebody is taking their second postdoc, they are probably having difficulty finding a job. The trouble with the way we asked this question in '94 was that we didn't ask why they initially took a postdoc. So, it is possible that this includes people who initially took a postdoc to further their career, and currently re-enlisted because they couldn't find a job.

Kuh:
Did you see a super collider effect?

Neuschatz:
I don't see one. I think that that was a very big event, but when you are looking at an entire discipline, I don't know that one event would be big enough to show up. Though maybe this would be big enough. The super collider was canceled roughly around '93, so it is possible that some of this jump could be people who had initially taken a postdoc expecting to work in the super collider and then found that that wasn't possible.

Fechter:
The strongest evidence you could probably put there would be time series data stratified the way you have it stratified now. Does AIP intend to do that kind of collection?

Neuschatz:
We do as soon as we get a consistent set of questions. This is the first year we asked this question. For the current survey, which is just coming in, we altered it to say, when you initially took your post doc, were you looking for a permanent job? Assuming that we can keep that question for a couple of rounds, then, we will have times series data.

Another area that is very important, but not well explored, is underemployment. We asked people whether they considered themselves underemployed. This is the purely subjective measure. We also obtained more objective measures like whether they are working part time, but wish to work full time or working in a job with no advancement, but wish one that had advancement.

Again, what is interesting to me is that when postdocs start out, many of them, even if they were there because they couldn't find a permanent job don't regard it as being underemployed. This is probably because it is such a common thing to do these days and it is a respectable and accepted part of the post Ph.D. career track. It is only when you start to stay in a postdoc status for a long time that that begins to be an issue.

But then there is this other question. I go back to the point that there are 2,800 to 3,000 postdocs. There may be, after all these years of a climbing number of postdocs, a saturation effect. There may simply be not enough postdoc positions to go around.

I don't know if this is true, but I find this to be a very interesting data point, that a quarter of the people who didn't get a postdoc, who got what we call jobs, consider themselves to be underemployed.
Of course, underemployment may go down as people find jobs that satisfy them and leave the status of being underemployed. I think 12 percent of all of the members of member societies who were in postdocs consider themselves underemployed.

We have done studies that ask people what kind of skills they use. One of the things that we felt was quite interesting is that interpersonal skills and technical writing seem to rank as high as knowledge of physics for all work sectors, not just for the ones you would expect, like industry.

That is the background information. We talked earlier about some of the more specific information we have. We asked departments this year, for the first time, whether they are consciously limiting their first year students, since we had noticed this extended decline in first year enrollments.

Only a sliver of departments say that they are consciously limiting enrollments because of the employment market down the road for their graduates. I think the number is under 5 percent. A much larger proportion, almost half, say they are limiting enrollments because of externalities like funding or a drop in faculty, which may be related to funding.

We gave them a number of reasons, and we just said "such as," and about half of the remainder checked that box. The other half said no, we are not limiting, we are limited by the people who are applying, and we would gladly take more graduate students if more qualified graduate students were available.

I was very surprised by that finding actually. In a period of falling enrollments, what it means is that the drop seems to be half driven by departmental policy, but not by desired departmental policy, but an involuntary response to external conditions, and the other half appears to be due to factors within the student population itself, that are really beyond the department's control.

Now, of course, they could redefine their view of what is a qualified graduate student, maybe opening up their pool. We are currently trying to gather data on how many people applied, how many were accepted, and then how many of those who were accepted actually enrolled.

Fechter:
If you look at it by type of department or type of program or quality of the program, the departments that say they are not taking any more because they don't have the funds or are losing faculty, may be doing pretty well.

Neuschatz:
We didn't get a chance to do it by quality, but I did it by size, which is highly correlated with quality, and there was no difference.

Fechter:
No difference.

Neuschatz:
I was surprised also. That is why I had the run done. In fact, the distribution seems to be the same for the largest departments, middle size departments, and small departments. There are slight differences, but they are really marginal.

But we were looking just for that kind of effect. Maybe when I go back, I will rerun it by quality to see if it holds.

Fechter:
You have data from NSF on R&D funding. You might want to look at it in terms of what are the trends in funding for those programs over a period.

Neuschatz:
Again, remember, these are the departments' subjective viewpoints, and they are limited in terms of time. We are talking about one year to the next essentially, and things can fluctuate.

Shettle:
The next presentation is by Mary Jordan from the American Chemical Society.

Jordan:
Unlike Mike, I was trained as a demographer, and we are trained never to make forecasts. We are the people who didn't forecast the baby boom, remember, and we learned our lessons well.

As most of you know, I am the new kid on this block, so I haven't tried to go in depth too much about what is going on, but we have had some indications of where things are going. 1995 was a miserable year for Ph.D.s, and the first overhead I will show you is the one that got everybody excited.

As you see, this is our fall unemployment rate. With the CPST project, we are all trying to iron timing issues out, so that we can have some equitable measures. Our survey is from July through June graduates. We send it out in August. We start running results in November. We see, for the Ph.D.s and the master's, '95 was an extremely tough year in terms of the fall unemployment rate. This does not mean they didn't get a job ultimately, but as of August or September, they still did not have a job, or postdoc.

This caused a lot of comment, as you can well imagine, but on the other hand, we do have a whole lot of BS graduates, and they had had a whole decade of pretty miserable times. In 1995, the BS salaries increased, their unemployment declined, and something else very unusual happened. This may support part of the findings for physics: for the first time since we have been keeping a chart on this, 42 percent of the graduates decided to go to work. Normally, that runs from 33 to 36 percent or at least in the 30s. So it was a large increase in proportion of graduates who did not go on to graduate school.

This [percent of bachelor's chemists who go to other graduate programs] is the number that has run around the mid 30s. This has changed throughout the years, but this has usually been about 33 percent right here. So we have had a real downturn in even those going on to medical school in the last year, in '95.

Now, we have been assuming it is because perhaps they can't get in. Well, it could be a combination of things, including that the jobs were there and they were paying more money. So students, if the jobs were there, and they were getting some offers, may have opted not to go on. This is the fall. As I said earlier, we think it has eased up and perhaps late last year or sometime during the winter this year the dam broke.

During the nineties, we have gone from where Peter said, six plus years to Ph.D. to over seven years for their Ph.D.s, from 6 to 7.3, I think it is, in '94.

That was the pipeline of PhDs filling. They were staying longer in the Ph.D. program. More of them went off to postdocs. We have a hard time tracking where all our postdocs are because, unlike most of the sciences, chemistry postdocs also go into government, industry, medical schools, agricultural colleges, etc.

Another thing that is going on with chemistry is that we have a large proportion of new grads going out to very small companies. That is where the breadth is required. Procter & Gamble can hire a Ph.D. who is doing one little finite thing over here, or a pharmaceutical company wants an analytical chemist that can do combinatorial chemistry that interfaces with the biologists and get very, very specific about what they want, but the small companies want a Ph.D. who can do everything. We had 40 percent of our Ph.D.s going into companies with 500 or under 100 employees, where they want the one or two Ph.D.s that do everything.

As I said before, it seems so much calmer this year. Even the employers -- because I talk a lot to employers as Joan can tell you -- that call and want to know about the salary surveys and will start chatting with you about why they want to know this different information -- and they were sounding optimistic.

My favorite story is the fellow, who will remain nameless at his request, who had 80 employees for about a decade now, and is going up to 120 employees, and most of them will be chemists this year. Of course, everybody wants to know where they can send their resumes, and he went through our department. He is in the middle of a lot of different industries that are chemically related, and he was very optimistic about everybody around him, too.

So in January, I started feeling that maybe things were going to be better.

One of our first indicators that '95 was going to be bad for the new grads, though, was in our regular salary survey, which in '95 was a survey of all the ACS membership, and in that survey, we saw a very large shift in the age of the unemployed.

Our department was set up much in answer to the downsizing in industry and the unemployment of older chemists. In 1995, we saw a huge shift in the age of the unemployed. The largest unemployment rate was in the ages of 25 to 29, but those under 30 had the largest unemployment rates at all degree levels, and they had the largest proportions that had some unemployment in the prior year.

That was perhaps our first indication, so we weren't totally surprised when the graduate student study came in, in the fall, that they were having the same problem.

Now, another answer that we have coming out is on the last page. One of the things that we hear from chemists in the field, we hear from graduate schools, we hear from all kinds of folks, is what else is there to do if you are trained for a very highly specific field.

This is the outline of a book that is coming out of our department, talking about alternative careers. We are not saying you should go out and be these. We are just saying there are more options at all degree levels. I know there are Ph.D.s in this book, that somehow maybe started out working in a lab, and they have gone off into other areas, and they are pretty happy about the transitions they have made and in how they made the transitions.

So that is the latest product we have. It is to give graduate students, chemists in general, or to incorporate into our talks, just to give them the idea that there are more than two or three places that you can work in this world, and that we have growing numbers of ACS members who are in those alternative areas.

Lately, I have been requested by other professional societies to send off all of our literature and I will give you the sales pitch here. You might want to pick this up if you are interested in careers, because we are pretty well established on this issue of careers, and we have all kinds of services that we offer our members and graduate students.

One of the interesting things, as some of you know, is that we do have a program on resume writing skills, interviewing skills, and those related skills that goes out into the graduate departments. In the first year that it was offered, which was last year, we were turned down or not replied to. We are rotating through the list. We have a three-year rotation through the chemistry departments that offer graduate degrees, because the career department handles graduate degrees and above, and we were turned down by those departments or not replied to. So we have had to really work to get into some of the universities.

Nelson:
Any evidence why?

Jordan:
Well, I really don't want to get into the politics. Yes, we do have some evidence. Just somehow some people just really --

Nelson:
Our people don't need it?

Jordan:
Right. And actually, some of the departments that you would think don't need it, are the ones that bring them in, and that's why their students don't need it, is that they cover every avenue.

O'Neill:
Maybe they think it competes with university career services.

Jordan:
I think we always work through the university or departments' career services if we can. I mean we try to combine all kinds of services. You know, some of the very large departments do have career consultants within them, and we try to work very closely with those people. Elaine Diggs, who is here, handles the consultant program. We have career consultants throughout the United States also who will go into universities and talk and review the resumes. That is a free service we give any ACS member or student. Any other questions? Yes.

Fechter:
You may have said it, and I may have missed it, but the very last table in your handout, which is Career Alternatives for Chemists, where did you get the information for this?

Jordan:
These are actual people represented. This is the outline from the book, and these are the people who have been interviewed and their two or three page synopsis that they wrote themselves. It is a compilation of interviews and life bio sketches.

Fechter:
How did you select the people?

Jordan:
A lot by word of mouth. It wasn't a random selection. To get at it, we would have had to know somebody who knew somebody, and these were ACS members that went out and found these people or knew these people and interviewed them, and then it was written up. It was sent back to these people for their review.

Kuh:
I have a quick technical question. For the 2,500 people who answered your survey, is that from sending out the survey to all 9,500?

Jordan:
Yes.

Kuh:
Do you have any way of knowing if these people are special in any way --

Jordan:
No.

Kuh:
Or do you think they are representative?

Jordan:
I do actually, I could compare by basic demographics. I just haven't analyzed it.

Kuh:
The main question is, if you answer the survey, are you more likely to do it if you are employed or if you are not employed?

Jordan:
No, we don't have that measure. This year I am going for a much higher response rate.

Shettle:
Thank you.
Our last speaker in this group is Catherine Gaddy from the Commission on Professionals in Science and Technology. As many of you know, CPST is an umbrella organization for many of the societies as well as corporate and academic employers of scientists and engineers.

Gaddy:
Let me pause for a minute and let you read my first slide before I go on: Graduate programs/advisors are responsible for ensuring that their graduates find employment OR Graduate programs/advisors are not responsible for ensuring that their graduates find employment

There is an excellent book on preparing one type of scientist entitled, "Preparing Psychologists for the 21st Century," (edited by L. Bickman & H. Ellis; Hillsdale, NJ: Lawrence Erlbaum, 1990). In particular, there are two sections entitled: "Departments Should Prepare Students for Careers, " and "Should Departments Be Held Responsible for the Marketability of Their Graduates? A Con Position." (Note these chapters tend to address doctoral-level education and training.) These sections got me thinking about what sort of data do we need to try and find the happy medium in this debate, and the two psychologists who wrote the chapters did a good job of taking the opposing positions on these issues.

What I listed at the bottom of the slide are some of the considerations that you might imagine the section authors debated, such as academic freedom, societal responsibility, equipping students for life, making a living, changing job market. How do we balance academic freedom with societal responsibility? What are we educating students for? Are we equipping students for life? or for making a living?

With regard to data needs, what level of information do we need to know about curricula and education and training, and what sort of information do we need to know about the job market to do any kind of reasonable matching? I am privileged to work with staff from ACS, AIP, AMS, and APA who are, to my knowledge, the only four societies in this country, maybe in the world, who have collected survey data directly from recent doctoral graduates in their respective fields. Of these four societies, three currently have questions that provide bridging information about the match between education and training and employment: Mike Neuschatz from AIP presented one slide today that addressed skills that are used, Mary Jordan from ACS asks about team training among other things, and Jessica Kohout of APA has a couple of questions on their questionnaire. These kinds of bridging questions I think are extremely important and very challenging.

So my second of three slides is just to get us thinking a little bit more about this. In graduate education -- and I believe Charlotte brought this up -- we want to give people this incredible broad lifetime worth of skills and knowledge in five years. It is a real challenge. Do I put a course in on cultural diversity and bag multivariate regression? There are some real issues here. It is very difficult. How many options are there? What is a core for disciplinary identity?

In any case, we have this finite period of time where we capture ostensibly the "best and the brightest" students, or at least S&E's fair share of them, and indoctrinate them into the science and engineering professions. They have some initial and subsequent employment, and we hear various guesses about how many jobs and careers people might have. We also have this concept called life-long learning or continuous learning that gets batted around, but I don't believe there is a lot of formality about it yet for many of the fields.

Also, there is the issue of on-the-job training. Do we want a Ph.D. program set up to train scientists to work in one specific company's lab? Probably not. What are the employing organization's appropriate obligations and responsibilities for training?

One pharmaceutical company spends a year or two indoctrinating their scientists to train them in "their" way. Should some of that have been done in graduate school or not? A major automotive company has a core curriculum for their engineers - if they don't arrive with it under their belts they have to take it then.

One thing we can do is to ask S&E graduates some of the kinds of things that Mike, Mary, and Jessica do: what did you learn that you use a lot, what is helpful to you in the job that you have now, what do you wish you had learned.

Of course, a big factor is when you ask. I think that the U.C. Berkeley study that Peter Syverson described of PhDs 10 years out will be very interesting. While it has its limitations, as all research projects do, I think it will be very interesting to hear the retrospective reports about graduate education as another piece of information.

As an example of one profession that has a more structured approach (and perhaps has a more structured set of job requirements than some scientists and engineers), nursing did an interesting study where they asked the supervisors and managers of nurses what they thought about the training of those that they supervised. There are very few professions or disciplines that have actually done this, i.e., rather than asking the person, go to the boss (or "clients") and say, how did we do training him or her. So there are other ways to get at it.

In some of the work we are doing now as part of the Sloan project I will discuss later, we are thinking about what kinds of things we want to ask people, when do we ask them, and what would be most helpful.

In preparing some Congressional testimony on contributions of scientists, I was really sobered that few on Capitol Hill cared about publications and citations per se. That is about the only outcome measure we have for a lot of the doctorates. I think we need to do some high-level conceptualization of what we are training them for, some high-level conceptualization of what it is they are doing out there, and then the third hard piece, which Maresi Nerad and Joe Cerny are tackling, is how do we quantify that, which is yet another step that I think will be very important.

As you can see from this third slide, I think that societies can help in some ways to do this transitioning between higher education and employers, whether the employers are in academia or in industry, and perhaps the data can be put on the table to elicit discussion. As Mike and Mary and others have pointed out, it is not perfect data, but it certainly does focus discussion, so that we start to talk about these kinds of things and what they mean. I think the societies' sensitivity to field-specific issues is extremely important, and something that is probably out of the scope for most Federal agencies; trying to keep up when you are in an association or in a field is hard enough.

Fechter:
If you are in NSF designing a questionnaire to go across all fields, you really can't design a questionnaire that will get at field-specific information.

Gaddy:
Right.

Shettle:
Especially on things like skills.

Fechter:
Yes.

Gaddy:
So having expertise in the hot technical areas, either by members or staff is important. There are lots of volunteers among the membership who have field-specific expertise in education. Experience in a broad range of employment is also important and represented among their members. I sometimes think we forget that academe is also an employer. We need training for teaching, as well, which we don't often stop to think about.

It seems to me that there is a mechanism for providing national-level feedback between "school" and "work." Perhaps we can work in concert with folks doing Federal data collection to coordinate this, but I think it would be very useful. The key is that this is a school-to-work transition. We use that phrase when we talk about the high school students, but that is a pretty apt description for this transition too. What feedback can we get going, what can we tell academe, what can we tell graduate educators that is helpful to them, that they can do something about and use in their curriculum? How can we get that feedback going? I see that as a really important need that has been scratched at, but not systematically dealt with. This is something I don't think we know a lot about.

Thanks.

Shettle:
Thank you. I think it was a good session this morning. See you this afternoon.

AFTERNOON SESSION

Shettle:
This is the first time Rocco Russo has done a presentation for us. He has a very interesting topic. He sent me some little blurbs about it ahead of time and I thought it was really exciting. This is something we haven't heard about in these sessions before.

Russo:
Thank you, Carolyn.

AAU/AGS PROJECT FOR RESEARCH ON DOCTORAL EDUCATION

Russo:
The project that I am going to introduce you to today is called the AAU/AGS Project for Research on Doctoral Education. AAU, probably as most of you know, stands for the Association of American Universities and AGS is the Association of Graduate Schools.

In my short fifteen minutes, I want to give you an overview of the project and also give you some insights into the data summaries that we have published and the data that we have put together.

The project was initiated in 1988 at the University of Rochester. The goal was to develop a national longitudinal database that allowed us to track the flow of students into and through doctoral programs in the arts, sciences, and engineering.

That goal was intended to provide data that addressed policy issues and questions related to some specific goals, the first being to provide information that would improve our understanding of national and institutional trends in doctoral education. A second goal was to provide data that individual departments at AAU institutions could use to compare the features of their program to other programs.

Finally, we wanted to help policymakers understand the forces affecting the flow of doctoral-student talent through admissions to completion or attrition.

The database that we are developing is based on voluntary participation by institutional members of the Association of Graduate Schools within the AAU. Currently, there are 60 AAU institutions of which roughly 40 are participating.

The data from each of these institutions are collected on an annual basis. The data are computerized student-level data that are actually transferred to the project by institutions. It is important to understand that we don't have an individual survey form that students fill out.

Rather, we ask graduate schools at, let's say, U.C. Berkeley to provide us with a computerized disk containing data about their students in each of ten doctoral program fields. We have five data areas that we are addressing -- demographic data, background data, academic talent data, student progress data, and financial aid data.

In '88, we initiated a field test, or pilot study, of students in one program, economics, to begin to develop and test our procedures. In the fall of 1989, we collected data about applicants and students in five fields: biochemistry, economics, English, math and mechanical engineering.

In 1992, we expanded the project to include the original five fields and an additional set of five fields. We selected those fields based on factors such as growing, emerging fields of study. We also wanted to cover the broad five areas of doctoral education as well as a mix of programs with varying levels of participation by women, international students, and so on. There were a variety of criteria that our governing board for the project, which is a steering committee of roughly 15 graduate deans, worked through to select these fields.

As I mentioned, a three-year cycle of data was collected in '89, '90 and '91 for applicants. We have datasets related to applicants and applications to the doctoral programs in the initial set of five fields. What I am going to be talking about today are the student datasets across the ten data fields that we have studied.

There have been a variety of summaries that have been prepared and disseminated primarily to our participating institutions. Initially, this was the main reporting focus for the project, since the AAU institutions and the AGS were funding the project. We spent the first few years developing reports that provided institution-specific data back to the participating institutions. We now have moved and expanded into more global reporting cuts of the data while still maintaining an individual reporting format for the participating institutions. Over the years of '89, '90, '91, '92 and '93, we have a core set of 28 institutions that have provided data to us in the five programs listed below for the original five doctoral fields. I would like to walk through a few summaries of those data.

We are currently completing the '94-'95 student year. That dataset will be ready in about four weeks and that series of reports will be coming out in about six weeks. So although I reference '93, it is the '93-'94 academic year.

Data for the '95-'96 academic year, the year that we are in, are due to the project at the end of June. We capture information for a graduate student's or a doctoral student's program for an entire academic year. That is part of the reason for the time lag.

By and large, I think what we see from this chart on average doctoral student enrollments by field is a continual growth in doctoral programs that has leveled off over the last two years. I expect the data to show the same leveling in '94. I think that is probably consistent with what you have found in physics.

Looking at U.S. students only, we see the percent who are U.S. students has increased slightly, particularly for the fields here on the lower end -- economics, mechanical engineering and math. It is pretty stable for biochemistry and English. Of course, English is on the high end so we wouldn't expect too much of a change upward for that field.

Looking at the percent of female enrollments; again, this is very stable across the years. This finding is present regardless of citizenship. I took a cut at the data looking at U.S. females only; again, it is very stable but 2 percent higher than the prior overhead per field. So instead of 59, we may have had 57 when we take into account both U.S. and non-U.S. students.

What about minorities? Looking at percent black Americans -- I do want to caution that the numbers that we are working with here are extremely small, in some cases less than 35. Black Americans have done best in making gains in English. We get mixed results but maybe positively slanted upward for the other fields.

Hispanics do a little bit better. You don't have a copy of this particular overhead in your materials. Here we see a more positive increase. Again, the numbers are extremely small. I wouldn't put much statistical reliance on what we are looking at here but at least it is in the right direction, and we are seeing an upward trend.

I think we see more of an increase in percent Asian in those fields, but again, the stabling-off is due to the leveling off of enrollments, English being down at the bottom, of course. Asians are not under-represented in English, so we would probably speculate that we wouldn't expect to see much happening there.

Mechanical engineering is all over the place. The numbers are just so small in some cases that three or four people will make an impact on the percentages.

That is a look at some of the trend information. What we are currently doing is taking a look at these trends and cutting them by program size, by quality, looking at some of the NCR quality measures as well as an idea that I received here today which was the R&D expenditures. So we can look to see if there are different trends in doctoral programs that are on the top end of quality versus some of the other -- among the cohort of 28 institutions.

Here I took the newer fields which we started studying in 1992 and selected out some of the science and engineering fields. Here we see that physics, again, is pretty stable and the numbers there don't really reflect a growth or reduction. From '92 to '93, in economics, we see a little bit of a move down; math, possibly; chemical engineering being pretty stable; mechanical engineering showing a little bit of an increase. But, again, year-to-year differences -- I think the trend data provide a better look at that.

Looking at those same fields in terms of citizenship status, the top is the fall 1992 cohort. Again, there are very, very stable patterns across years by field. These have been very consistent over the years.

We have also looked at gender. Mechanical engineering is on the low end for female participation. Economics. math, and chemical engineering have a little higher level of involvement. And physics is somewhat in between.

Kruytbosch:
How do you know these are doctoral enrollments?

Russo:
I have asked each institution to provide on October 1 or thereabouts an individual record for every doctoral student they have enrolled in biochemistry, in mechanical engineering, etc.

Kruytbosch:
What happens with somebody who enrolls in graduate school and doesn't specify whether he is going to go for a doctorate?

Russo:
It is based on the institution's definition. We have programs at the AAU institutions where everybody who comes in is a doctoral student from Day 1. We have others that indicate that they need to complete a master's degree first, then they have a formal or an informal entry process into a doctoral program.

A quick look at minority status. We can see these include Asians, blacks, Hispanics, and others in those fields.

We have also taken a look at some of our academic talent measures. I am presenting data for the 1993 cohort on this. The top graph is for public institutions. The bottom graph is for private institutions. Across fields, we see similar patterns. Within fields, as you can see, the blue bar would give an indicator of the range of GRE scores.

What we see is a very slight - 20 or 30 points difference -- on the GRE. The public schools are a little lower overall, as we can see here, except for physics.

By and large, if you look at the 50th percentile, for the private schools, we have just a slightly higher mean and less spread. A good indication, here, is in math. We can see at the very top end the private schools are creaming off a little higher segment of talent than the public schools. This is just to give you a flavor of some of the analyses that we have undertaken.

Finally, here is a look at the analytic score, again reflecting some of the same patterns. I don't know how much credence we want to put in 20 to 40 points difference on the GRE scores, but that is what we see there.

As we noted, part of our intent is to look at time-to-degree for students. We have pulled off, for this analysis, a set of, I believe, 14 or 17 institutions, roughly 4,000 students that we identified in fall 1989 and looked at their time-to-degree patterns.

Their representation by field, gender and citizenship status are there on the pie charts.

Kuh:
How long was it since they enrolled? Is this the '89 cohort?

Russo:
No. This is not by cohort. This is anybody who was there in 1989. We are just beginning to take a look at some of the time-to-degree analyses.

Fraumeni:
I am seeing more resident aliens in my Ph.D. classes. Do you have any information on resident aliens as opposed to U.S. citizens?

Russo:
In our analyses, we include the resident aliens and U.S. citizens together.

Fraumeni:
So even though you are saying U.S. citizens, that includes -- is that the general convention that I have been seeing today for the other information -- resident aliens are included with U.S. citizens?

Syverson:
Yes. There are some surveys that can give you information about resident aliens broken out.

Kuh:
But they are part of the U.S. talent core. There is a reason for doing that, I think.

Fraumeni:
Yes; that is why I was objecting to thinking that the resident aliens were put in "other."

Neuschatz:
We don't do that. One of the reasons is because it is so subject to legislative changes. For example, the Chinese Student Protection Act suddenly created a bubble of what appeared to be resident aliens who were temporary students before. If you don't separate them out, you can't make sense of that. There are arguments for both sides.

Kuh:
But they are available for employment in the U.S. -- right?

Russo:
Right.

Fraumeni:
That probably should be clarified for people who don't know your conventions.

Russo:
Okay. These are very initial, preliminary cuts, but I did want to share them with you today. There is more work that we need to look into. Basically, we see that non-U.S. citizens take a shorter time to complete their degree than U.S. citizens. However, in the first two years, what we are picking up is that the distribution is pretty much the same.

Our speculation as to why that is happening is that those who are coming in with a master's are finished in two or three years.
This is across all fields; yes.

Neuschatz:
First enrollment meaning the first time they appeared at that department rather than the first time in the graduate school?

Russo:
Not at the department; at the doctoral program at that department.

Kruytbosch:
Could it also be that there are more non-citizens in mechanical engineering than there are in English? The average length of time-to-degree in those two fields is rather different.

Russo:
Yes. We will be working through some of the field-specific ones here but we just did some initial cuts.

This is the same cut by field; basically, English taking longer, mechanical engineering behind biochemistry, math, and economics; math having the shortest, very close to economics.

We look at this by field and citizenship, biochemistry, non-U.S., having a slight edge over U.S. economics; we have a weird distribution here -- almost looking bimodal. We are delving in and looking at what is happening there in the first two or three years.

Syverson:
You could have a couple of large departments that require a master's degree.

Russo:
Right. That is what we might have.

Syverson:
So what you have here are early completers who have gotten their master's degrees.

Russo:
They have been there for five years, but they have to finish the master's first, formally apply to enter a doctoral program and then the clock begins ticking when they formally enter a doctoral program. That is what we feel is happening.

In English; a big, big difference between non-U.S. and U.S. We also have big differences in numbers. Math is looking very similar for both U.S. and non-U.S. The numbers are fairly close as well. And then the final one is mechanical engineering. Sometimes I wonder what happens --

Syverson:
There is a case out there; right?

Russo:
Yes. Going back to the whole group and looking at gender, females take a little longer than men but, in the first two years they are very similar.

Let's look next at gender by age. Patterns are very similar for men and women to about the age 30. Between 30 and 38 women complete more slowly than men, and, after that, women complete faster than males.

The final one, which probably shouldn't go up since the N's are so small, is a look at ethnic groups. Again, it is only U.S. citizens. The differences are not significant. Likewise, as in the male/female break across all fields, the differences were totally explained by field differences. So there were not any statistical differences by gender. I was trying to keep on time so I hope I didn't go through this too fast. I would welcome your questions.

Kruytbosch:
We keep talking about the difference between longitudinal databases and time-series databases. What you have here is mostly time series except for the time-to-degree. Are there any other questions that you can address with your data such as what happens to individual students as they move through the program? I would call that longitudinal.

Russo:
There are several datasets that we have. One is the applicant dataset which takes into account, on a yearly basis, the number of individuals who have applied to programs across our set of institutions. We have put together a dataset that takes into account multiple applications. We also have an application dataset that just counts numbers of applications to economics and so on. On the student side, we have the individual datasets for each field by year and the number of institutions that participated in a given year varied.

Of course, not every one of those 40 participating institutions offers a program in English, for example, Cal Tech. So the number of programs in a given year, per field, varies. The initial set of data, the graphs we talked about for the 28 institutions, took into account the same programs of varying students across years.

The very last set is the longitudinal dataset that we are creating. We made the link from Fall 1989 to 1990 looking at the student who was here in '89, were they here in '90, were they here in '91, '92. So it is an individual record-based data set. The others are more field-oriented, but include student-level data.

In a roundabout way, I hope I answered your question.

Shettle:
I think we are going to have to cut questions now because we are running well behind. But we will have an open discussion later. I will turn the meeting over to Al Tupek who is going to take over for the rest of the afternoon.

End Session II