SRS Professional Society Workshop

DATA NEEDS FOR ASSESSING THE RELATIONSHIP
BETWEEN GRADUATE EDUCATION AND EVOLVING TRENDS
IN THE S&E LABOR MARKET

SESSION III

SESSION III: ALAN TUPEK, DEPUTY DIVISION DIRECTOR, SCIENCE RESOURCES STUDIES, CHAIR

SRS data relevant to the policy issues surrounding graduate education and its relationship to trends in the S&E labor markets

Mary Golladay, Education Program, SRS
Susan Mitchell, National Academy of Sciences
Carlos Kruytbosch and Mark Regets, Personnel Program, SRS

Open Discussion

Update, Sloan-supported project on the S&E job market for recent graduates

Catherine Gaddy, Commission of Professionals in Science and Technology

Around the table -- Science and Engineering Indicators

Jennifer Bond, SRS Science and Engineering Indicators Program

Closing Remarks

SESSION III
Tupek:
We are going to be looking at data from the Science Resources Studies Division, your tax dollars at work, for the next hour or so. Well over half of the $12 million a year the Division spends in data collection efforts goes to data collection efforts related to the education of scientists and engineers and the work force.

So we are going to hear a little bit about what you are getting for your money in the next hour or so.

The first speaker is Mary Golladay. Dottie Jacobs also worked on this, but I believe Mary is going to give the presentation. Mary is the Program Director for the Education and Human Resources Program within the Division of the Science Resources Studies. She is going to be talking about the Graduate Student Survey.

SRS DATA RELEVANT TO THE POLICY ISSUES SURROUNDING GRADUATE EDUCATION

Golladay:
There are two surveys that are handled out of our program that deal with aspects of graduate education, so they are right at the beginning of the period that we are looking at today. I just want to call to your attention to what they have to offer on some of these issues.

The first is our Survey of Graduate Students and the second one is the Survey of Earned Doctorates, which is taken at the point when someone completes his or her graduate education. Let me start with our Survey of Graduate Students and Postdoctorates. This is a very comprehensive survey, as you can tell. It picks up every student enrolled in an accredited science, engineering or health program. We are joined by the National Institutes of Health as a sponsor and we find that additional area very useful to us, especially as we look at the time trends. So we have made cooperation serve us well, there.

Just to give you a sense for the scope of the effort, the survey covers 433,000 students. About 100,000 of these students are part-time and 333,000 in the group of full-time students. We go to over 11,000 departments and we do get data at the level of department. The data are all publicly available so we can give you back any data on institutions or actual departments in which you might be interested.

About 604 institutions are participating in the graduate education endeavor in these fields. Again, that is all the institutions involved across the country and we have time-series since 1975. Some data were collected before that, but our reliable time series start in 1975.

So this is a survey that gives us a good steady basis against which to look at some of the issues that are before us. It consists of three statistical snapshots. Unfortunately, they don't always interface. The first one gets students recorded by the primary type of support that they get as they go through school. It just looks at the financial status and it is a force fit; that is, we ask departments to select a single mode and report students by just one source so that any one student appears only once in the matrix.

Incidentally, I did bring copies of the questionnaire. If you are interested in seeing it or having a copy, I invite you at the break to come see me. As I say, one of the important matrices is the mechanism of support -- that is, fellowships, traineeships, et cetera -- by the Federal sources and then non-Federal sources, so mechanism by support is snapshot number one.

Snapshot number two takes that same student population but gets a different cut at it, this time by race/ethnicity, and full-time, part-time status. I am very pleased to report, we actually have new information in '94 that we have never had before and that is race/ethnicity by gender.

It took a lot of time in coming but we do have that information. We only have one year at the moment, but we will be developing our time trends. It is a separate snapshot and so we cannot tell you what the financial assistance patterns are for particular demographic groups at this point. That would be another dimension which we don't have.

Then the third snapshot is an account of postdocs and non-faculty research staff with doctorates. What is in that latter group is, perhaps, a bit uncertain but it is distinguished from postdocs which we regard as a particular type of educational research experience. But we have some information on that which we will share.

Because this is reported from the academic departments, there may be institutions where postdocs are missed. If the postdoc is with some affiliated body that is not tied to an academic department, they may not show up. So we are a little suspicious that we have a bit of an undercount, but, nonetheless, we have a time series which we regard as very important. I am only going to show you these two absolutely exciting charts that you have in front of you. The first one is major type of support. I am not going to do anything with the demographics just because our time is too short. The numbers for all of these are included in the tables that you have that, in fact, go into a lot more detail.

The major type of support; the top line, is research assistantships. Basically, support is virtually flat over the eight data points shown here. There is nothing much going on except for research assistants, students supported on research-assistants money.

That is the one message from that. There is an "other support" category which has a lot of students in your table. But it is a high enough number that it would have scrunched all of these other trends if we included it in the same graph.

So starting from the bottom is the traineeship number around 10,000. The second one up is fellowships. The third one up, around the 60,000 mark, is teaching assistantships. As I say, the top one is research assistantships. Those are your four numbers.

Syverson:
Can you speculate for a minute about what is in "other?" Is it loans? This is a departmentally based survey; right? So the departments are filling this out and are telling us that the largest source of support is "other." In other words, there are more students in "other" as their primary source of support than any other source; is that right -- 128,000 versus 91,000 on RAs?

Piekarz:
Roughly a third of those students report self-support.

Golladay:
A lot of self-support.

Syverson:
Is that what is in "other?" Self-loans?

Golladay:
Yes.

Kuh:
Do we know the mix of these by degree objective?

Golladay:
No; we do not. We keep asking the institutions, do you know the difference between a master's-degree student and a Ph.D. student? Some institutions will say, "Absolutely; no question." Others will say they don't have it.

So, at the moment, we do not collect that and we might try even to figure out if they could make those demarcations, would it then improve our --

Kuh:
In particular, if you have a problem with the "other" category, and most of those other people are MA or MS students, then you might worry about that less than if that big "other" were Ph.D. students, since that would help explain how long degrees take.

Golladay:
Right. We, unfortunately, do not get that. That is a real problem disentangling what is here. There is a lot of information. One of the tables gives you some '94 information on the type of support by source; in other words, the agency by type of support.

We did just one chart investigating the phenomenon of postdoctorates to try to see what was going. The pink one at the top is biological sciences. The orange one next is health sciences. This was important enough that we thought we would show health here even though it is not typically reported as a science and engineering field.

Notice how closely those track. Engineering is the bottom one. And physical sciences is that number just below the 6,000. You may ask, what about the social-science disciplines, about some of the other scientific disciplines. The numbers are in Table 28. We didn't graph them because the numbers are all so small. They are all below 1,000. Look at the mathematical sciences, computer sciences, agricultural sciences, psychology, and social sciences. The numbers are so small that even though there might be some increases and decreases, apparently it is not part of the culture in some of those disciplines.

Kruytbosch:
But you are lumping together masters and Ph.D.s.

Golladay:
These are postdocs.

Kruytbosch:
I thought what we have seen in the survey for doctorates is that, although it is low, it has been growing quite rapidly even in those fields. Am I wrong?

Golladay:
That's true. For example, 99 postdoctorates in computer sciences were reported in '87 and we are now up to 182. If you want to do that in percentage terms, obviously, you are going to get a real dramatic chart, much more dramatic than this one.

So yes, you are definitely right, the numbers -- psychology from 459 to 544 -- are fairly dramatic. If you were to convert it to a percent or rate of increase, you would see something much more dramatic there.

So here you have the numbers and you can look at them and see whether they do or do not indicate some changes. That is food for discussion. I would call your attention to a chart on postdoctorates that is in the new Indicators report, which many of you have, which takes these -- I think it takes these numbers; is that right, Mark?

Regets:
Yes.

Golladay:
And puts them in the numerator, the denominator being the number of doctorates; is that correct?

Regets:
Right.

Golladay:
And, again, shows a pretty level situation, if you will, even somewhat more level than you will see here with the absolute numbers.

Fechter:
Under health sciences, are they restricted only to Ph.D.s or do M.D.s and D.D.S.s --

Golladay:
They are supposed to be only Ph.D.s. We constantly work with the medical schools and all. We are looking at research activity and we sort of shove the M.D. activity aside, whether clinical or study or whatever.

Fechter:
But I can tell you from personal experience that there are M.D.s who are engaged in research on campuses in this country. The question is, are they included or excluded from your figures?

Golladay:
They are supposed to be excluded, but whether they really are, that is --

Fechter:
The other question is we have been hearing from the societies this morning about postdocs, at least over the last ten years, having become a more prevalent phenomenon and, somehow or other, except for the biological sciences and the health sciences, your figures don't seem to show that.

Golladay:
That's true.

Fechter:
Why is there a difference?

Neuschatz:
I have one possible explanation. You have to remember that these are restricted to postdocs at Ph.D. universities. They don't include postdocs who are at national labs or in industry. The proportion of those varies from field to field. In some fields, that is a substantial proportion of the postdoc students.

Fechter:
Especially in the physical sciences, that would be very important.

Golladay:
Right.

Jordan:
The other thing is it is not just the number of postdocs, it is the proportion who are staying there for longer than the two and four years. Maybe, the overall number isn't going up in academic institutions but you may have gone from a two-year stint to a four to a six to a eight, or going from institution to institution and that proportion has gotten larger.

Fechter:
I wouldn't expect that to raise the total numbers, though.

??:
The other point that Mary made is that this only captures the ones in academic departments. To the extent that postdocs are going into research institutes or some other areas outside the academic department, that is where the growth has been, that is not going to be captured in this data.

Golladay:
We may be up against the capacity phenomenon that Michael was talking about. There may not be more places in those academic institutions or this may be pushing even that limitation.

Fechter:
But this does not pick up the research institutions within the universities.

Golladay:
That's right, unless it is affiliated with an academic department.

Fechter:
I have a follow-up question. Has any attempt been made to try to reconcile the differences? Is there any attempt to resolve the difference with respect to your estimates? If you were to pull out, for example, from the professional society data the numbers of people who are outside of academia, how much closer would it come? Is there any thinking that has been done about --

Golladay:
We would like to do this. It really, almost, needs to be done on an institution-by-institution basis. We have had a couple of offers to go to an institution, as we have done with the Survey of Earned Doctorates, occasionally, and look at detailed data and see how institution records track with other records, say, in the case of support. . unfortunately, we have not been able to pursue this.

Neuschatz:
I have a partial answer for that, too. Our data for '94 is fairly close to her data when we just look at research departments. I think we have 2,000 and she has 1,841. And ours may include some astronomy too. So they are fairly close.

Kruytbosch:
In the '95 survey of the scientists and engineers, we have added a question on postdoctoral positions and we will be able to determine whether they contain these characteristics or not. We will be able to mix and match whether or not they are nationally competitive postdocs, whether they are postdocs on a grant, whether they are research associates but they call themselves postdocs. That should help.

Fechter:
A similar question would be between those numbers and the numbers that might come off the survey of doctoral recipients where there is a big discrepancy between the levels of those numbers. The question is why. What does it mean?

Golladay:
Probably, there are places that we are not measuring here. That is my uneducated --

Fechter:
These are higher.

Golladay:
Oh; are they?

Syverson:
Also, it is a definitional question that we keep battling.

Fechter:
It is a slippery animal.

Golladay:
And we have tried to keep this in a slightly different category than we have what we call the "non-faculty research doctorates," which is to say employment of a Ph.D. at an institution or at a department where it is not exactly what we call the traditional postdoc.

I have a couple of those tables in there, I think a male/female cut, which you can look at. Those numbers are not changing a whole lot either.

Kuh:
But they are. In the biological sciences, that is the most rapidly growing category, this strange non-faculty thing. And we don't know what it is. We would love to find out because that seems to be where they are going -- especially if you look at people more than seven years out.

Fechter:
The focus of my question is based on the fact that this morning we heard about involuntary versus voluntary postdocs and the whole idea of wondering whether or not we are utilizing our Ph.D. production properly. If you can't measure the postdocs very well, what can we say for policy purposes about this issue?

Tupek:
Thank you, Mary. I think we should move on to the next speaker.

Golladay:
Our next speaker is Susan Mitchell from the National Research Council.

I notice that on the minutes from the last time this group met in December, I was announcing the arrival of data on Ph.D. awards for 1994. Here we are at the end of May and I am delighted to tell you all that we now have selected data on Ph.D. awards for '95, which are being released as of today. Obviously a lot of time and effort went into speeding up the release date this time. A number of people in this room, Susan Mitchell and Peter Henderson, deserve a lot of thanks as do the NSF folks. It was a lot of pulling and pushing from all directions, but we are very pleased to have 95 Ph.D. numbers for you.

Susan Mitchell of the National Research Council will talk about that Survey of Earned Doctorates.

Mitchell:
On March 29, the NRC held the first meeting of individuals who use data from the Survey of Earned Doctorates. This group is aptly called the SED Data Users Group.

The purpose of the meeting was to ask the user community to help us in the redesign of the survey questionnaire. Many of you were there and we really appreciate your assistance.

The goal of the redesign of the questionnaire is to help us improve the quality of the survey data and to help increase its relevance to current policy issues. The meeting was quite productive and a number of changes to existing questions were suggested as well as potential new content areas.

The handout I am circulating summarizes some of the key points that were made and lists, in particular, new questions that were suggested. I need to say at this point that I offer these merely as a reporter and not as a recommendation. Certainly, that will come later. We are very much in our preliminary stages but it should be noted that because of budget uncertainties we face and the fact that we haven't yet met with NSF to determine their interest in these questions, a critical step, we don't know which, if any, of these changes will actually end up on the final survey in its new form.

So just consider these food for thought. If you do your own survey work, you might find some of them of interest. Because time is short, I am not going to read through this but, even as I say that, I find it hard not to.

I will tell you just generally the main points. One is we were strongly encouraged to improve comparability between the SED and the SDR which is the longitudinal follow-up survey and between the SED and the surveys conducted by the professional societies.

We were encouraged to review other survey databases for possible alternative sources of data as a way to eliminate duplication, to consider other ways of collecting data; for instance, using administrative records at the degree-granting institutions for type and source of financial support, for example, and to experiment with using the Internet for data collection.

As we revise questions, we need to consider the tradeoff between making improvements and disrupting the time series and we need to use terms that are understandable to foreign students because this is a growing segment of our population. We also need to make better use of the SDR to publish data on new Ph.D.s, which is the population of greatest interest these days.

Now, let's look at post-graduation plans. There was a lot of interest in finding out more about the types of jobs people have accepted and their reasons for accepting the jobs that they did. Additionally, we want to find out, for example, what their starting salaries will be, did they have other offers, how aggressively did they look for work. If they are still seeking, why are they still seeking.

The graduate school experience section, which is something new for the SED, is where we are actually trying to get qualitative or attitudinal data. Whether or not we choose to pursue this or not isn't known at this time, but some of the questions that came up were: why did you go to graduate school; how would you rate the quality of certain attributes of your graduate education; how well did your graduate education prepare you for your intended career and, at this point in time, would you have chosen a different field or chosen not to pursue a doctoral degree; what about your graduate education might have prepared you better for your first job; and how do you view the job market. A number of these questions are asked on some of the other surveys and, to the extent we can achieve comparability, we will have a much richer database for analysis.

Our next step is to convene a meeting of what is called the SED advisory panel which is an NRC committee recently created with the express purpose of coming up with a new SED questionnaire. That group consists of survey specialists and subject specialists. That meeting will happen on June 20.

Our schedule is to finalize a questionnaire and have one ready for pretesting in October, spend three or four months in the pre-testing phase, and be ready to field the new questionnaire in the 1997 calendar year.

Thanks.
If you have any comments on what you see here, please let me know. We are still collecting comments and using them to design our questions.

Tupek:
Thanks, Susan. We can take one or two questions before moving on.

Fechter:
On the graduate school experience part of your exploration, did you include a question, "Did you have periods of part-time enrollment?" That is a very important question because a lot of people think that some of the increasing time-to-doctorate data may well be related to the fact that people are, in fact, more likely to be taking part-time enrollments now than in the past.

But you don't ask the important question. To me, the important question is over what period of time were you employed part time? How long were you employed part time? And when was that during your graduate education. You only ask for why. I think when is a very important part of that.

Mitchell:
Thank you.

Tupek:
Anything else? Let's move right on to our data on the science and engineering personnel statistics. We have two presenters. Carlos Kruytbosch is the Program Director for the Science and Engineering Personnel Program. I will let him start and introduce Mark Regets when that comes around.

Kruytbosch:
I want to try to do a bit of a selling job today because I think that what we have been producing over the last couple of years is a valuable national resource into which a lot of money has been put. It needs, really, to be mined. There is "gold in them thar hills" for you all.

What I would like to spend a little time on today is the idea of the education-occupation matrix. This data that we have assembled is very, very powerful in terms of allowing the analyst to explore different patterns of educational experience, occupational and career patterns.

There is a big, blooming, buzzing, confusing world out there. You don't just run this by that and you have got an answer. As an example, the Engineering Directorate of the NSF has a study underway entitled something like Profile of the Engineering Workforce. They have been using primarily the SESTAT data. It has taken them about four months to come up with what they believe to be a fairly reasonable picture of a variety of major patterns of educational/occupational connections among engineers.

Individuals may have up to five or six degrees. Some of them are in engineering. Some are not in engineering. Some are in business. Some are in physics. Some are in chemistry.

Who is an engineer? How do you define your field? Anybody who has any engineering degree. You have got 2 million people or 2.5 million people. Which ones should we look at? How are we going to divide it up? How do you divide up that technical community into manageable chunks that have policy relevance and intellectual interest.

That is no mean job. It has been fascinating to participate closely with them because I have learned a tremendous amount about how to better organize our data and what kinds of questions people who are concerned with the health and welfare of that community want to ask of it. These questions would be similar in principle but very different for each one of the communities that you represent. I would love to see each of you undertake a profile of your community using this very, very rich database.

Who knows? The Chemistry Division might be willing to put in a few dollars to assist on that. I would certainly support such applications on physics or mathematics. Jointly, we can give a lot of help. We can't afford to do it ourselves because there is just far too much to do and we can't afford to pay for it all, either. But I think that you would find this very solid information about the relationship between education and occupation in your communities.

Three surveys are included in the database that are aligned together statistically. There is the SDR for the doctorates, and there is the National Survey of College Graduates, and there is the fill-in of new graduates that is done every year in science and engineering and added into the system so as to have the younger people.

One of the sorrows of my position here is that we have lovely information on all college graduates in the United States and it is essentially sitting there. We didn't follow all of them up for '95 because it is not the mandate of the NSF to look at non-science and engineering. It is such a rich information database, but there wasn't any money anywhere else to follow them up. It is a great shame. But the data is still there and it is still, certainly, usable.

We have developed a web access system that is functioning, currently. We are doing runs to answer questions, but we have still one problem. We are still tinkering with one of the datasets, the SDR, to ensure that the confidentiality of the respondents is sufficiently guaranteed because we promised them confidentiality. We hope that this will be resolved very soon and that we will be able to open the doors to reasonably user-friendly, immediate, public access over the web.

Until then, and Mark will tell you a little about that, we have one of the component surveys, the National Survey of College Graduates, with a large number of bachelors and doctorates, available in CD-ROM. You can sign up for it outside. It is available for free.

Gaddy:
Are there plans to put any historical data on or are there problems with comparability?

Kruytbosch:
The historical data will come on within the next few months when we get the '95 data so there will be history between '93 and '95. And we are dealing with exactly that question.

Audience:
What about the NSCG. I know for the SDR, there was a change in methodology. Is it the same issue with the NSCG?

Kruytbosch:
Yes. There was the big reorganization of the whole personnel system. '93 was the first one. Certainly, in the SDR, there are some time series, and Rolf is the great expert on that. We might be able to put together some limited variable time series on the other ones. But right, now, we don't have that.

What I thought I would do is just give you a few examples of the big picture of the education-occupation connections here. What I have done is run everybody whose highest degree was a bachelor's degree in science and engineering by the field of their occupation. I have cut out the people who were unemployed or out of the labor force, so the numbers are not going to add up to 11 million. What I was interested in primarily are the diagonal cells there that show that about 40 percent of those with math and computer science bachelor's degrees are actually working in math and computer science jobs.

Life sciences are really quite surprising. Only 11 percent are working in what we define as life science occupations. Physical science is a little larger. Nobody with a bachelor's degree in social science is working in social science occupations. There are probably not too many social science occupations that are really open to bachelors.

You can see that engineering is vocational in the sense that it is a licensed occupation and you can only get into it in many areas if you have the degree, and that is reflected there. With regard to the social scientists with bachelor's degrees, there are only 2 percent working in the social science field. They are spread all over the map. About 20 percent are in management-type jobs. Teachers, 6 percent. Sales and marketing; there are a lot of people out there pushing the products. Here we have our construction workers and miners. That adds up to about 20,000 people with a bachelor's degree in social science. So there are quite a few hard hats down there. But there are obviously not too many people with bachelor's degrees who would be qualified to occupy jobs in social science. So that is kind of an artifact of the terminology with regard to the occupations. Let's turn to the masters.

Gaddy:
This is new to the '93, that people do this analysis.

Kruytbosch:
Yes; absolutely. The rigor and the extent of the occupational classification is new here, and the extent of the educational classification is new. So you can do quite fine analyses. And, because the numbers are so large, you can get quite good results from quite small fine categories, although there are frustrations in there and there are many things that you are not going to get exactly the way you want it.

Neuschatz:
The managers at the bottom; is that a subset of non-S&E?

Kruytbosch:
Yes, because they are classified as non-S&E in the way that we do it, although, as I will show, most of the managers feel that their degree is either directly related or partially related to their work.

Here, with the masters, you get a somewhat higher percentage of people in each of the diagonal cells. Engineering doesn't change very much, actually. More of the social scientists are working as social scientists, and the life scientists, and so on.

Individuals with one S&E degree or employment in S&E whose highest degree is a non-S&E degree are particularly interesting because this would include the MBAs. Nearly half of these are MBAs. The people in engineering are people with bachelor's degrees in engineering and MBAs. There are a heck of a lot of those people in industry.

That is a career pattern. You move into management that way. This is what they are studying now to see how many people are taking this track and how are they contributing to the economy in that way.

Syverson:
If I am teaching psychology in high school and I have a master's degree in psychology, am I in S&E or not in S&E?

Kruytbosch:
You are not in S&E.

Syverson:
I didn't think so.

Kruytbosch:
You would be a pre-college teacher.

Syverson:
So pre-college teachers, no matter what their subject area, are not in S&E?

Kruytbosch:
They are in the science and engineering database because they have a degree but they are not in the science and engineering occupation.

Costello:
Even though they are teaching in college? If you have a master's degree in a social science and you are teaching at the college level, you don't show up as having a social science profession?

Kruytbosch:
If you are teaching in college, you would be included. Post-secondary teachers in the life sciences would be included here; yes. I will show you the data for the doctorates. I didn't do it for the masters, but the data for the doctorates are right here.

Fechter:
You mentioned the engineer, the bachelor's degree engineer who got a master's degree in business administration. On that chart, where is that person?

Kruytbosch:
In here. If his highest degree is an MBA, he is in here.

Fechter:
So they classify it by the highest degree level.

Kruytbosch:
This particular table does that. You don't have to do that. I am trying to give you a broad notion of what you could do with this data.

Kuh:
If I want to find out about people who are teaching high school science, they only show up if, somewhere along the way, they got a science degree.

Kruytbosch:
Well, no. In this particular database, the science and engineering database, they would have had to have had a science or engineering degree. That is correct. But you could go to the other database which also will be up on the web, the broader one, and you could identify high school teachers of science who don't have any science and engineering degrees, if you want it. There are about, I think, 600,000 or 700,000 or 800,000 teachers of social science, physical science, and biological sciences at the precollege level in there. So you can do a lot with that.

Let me just show you the doctorates. This is from the SDR. Here, again, you see the diagonal is much more highly populated.

I was quite surprised by computer science. You think computer science is the most free-floating -- that they are working here and working there. But they seem to be the most tied into that particular set of occupations.

Maxwell:
Carlos, can you separate math and computer sciences?

Kruytbosch:
Yes; I can. Not here. I could if I had the machine. It would take me three minutes. This is that same table, but this is just giving you the percent of the total in any given column that were post-secondary teachers. You see, it is interesting that computer science has the highest proportion of their total who are actually post-secondary teachers. Social science is also reasonably high there. The rest are employed elsewhere.

Again, you see a somewhat higher proportion of managers among the doctorates. And the engineers, also, like to get into management. Management pays well and presumably it is a fulfilling job. Sometimes it is not fulfilling as a lower manager.

This last one will lead into Mark's presentation, if there is any time left over for him. This was the data that I mentioned about the relationship of the degree field to the work, whether the individual felt it was closely related or not related. I have marked two. Among the bachelors, a third of those who were not managers and about a quarter of those who were managers felt that they were doing work that was not related to their degree at all. But among the masters and the doctorates, the percentage who feel their jobs are not related to their degree drops to a very low level. Presumably, this would include some cab drivers, et cetera. Those in the manager column would be managers of the cab pool, I guess, and those in the non-manager column would be the drivers.

Mark, you are going to wow them with some career-type information.

Thank you very much.

Regets:
I am going to be following up on Carlos' general theme and taking some of the data and dividing it by the year that they got their Ph.D.

This overhead looks at U.S. Ph.D.s and the percent who are in non S&E occupations with one thing added. We also have a question as to whether or not their job is related to their degree. So this is just people who are both in a non-S&E occupation and who say that it is unrelated to the ir Ph.D.

Mind you, there are people whose Ph.D. is in physics and whose occupation is physicist who say their job is totally unrelated. But this seemed one useful way to divide it.

There are just a few trends there. In general, as might be expected, the numbers are low at the Ph.D. level. Most people with a Ph.D., even if they are in a non-S&E occupation, are doing things that they say are related to their degree. But this percent does increase over time and you do also see differences between fields, with the life sciences being relatively high and the physical sciences relatively low in this.

Kuh:
Do you know what they are doing?

Regets:
Half to three-quarters are top managers, where top manager is defined as someone who manages other managers.

Neuschatz:
What about in the first five years, where you expect fewer top managers. I know in the life sciences you have a very high proportion of postdocs.

Regets:
This is being affected by postdoc numbers, certainly.

Neuschatz:
Would postdocs say they were in a non-S&E occupation?

Regets:
No.

Neuschatz:
Don't you have something like 75 percent postdocs among biology Ph.D.s?

Regets:
I don't believe it is that high even in biology. But, no; you are right. Certainly, a lot of the people in this grouping are probably in their original field when they are in postdocs for the life sciences. But over 15 percent, even in the first five years, are already in some non-S&E occupation that they say is unrelated.

Neuschatz:
Any guess as to what it might be, what kinds of things people put down?

Regets:
It is spread across. The first time I looked at this, I thought, in the life sciences, "Well, these are people who are actually working as health practitioners or something like that." But the people who are in health sciences are mostly saying that that is somewhat related to their degree, so it is not the health-practitioner numbers that are getting into the life sciences.

This is really spread across a wide number of occupations. There was no particular mode.

Kuh:
Would you say most of the people in this room would fall in that category?

Regets:
It depends on how you fill out the SDR. I would have claimed that I am an economist, still.

Kuh:
But this is self-classification.

Regets:
This is self-classification. Absolutely.

Kuh:
But people have a narrow notion of what it means to be a physicist, do you think there are systematic ways of answering this by field where some fields have very narrow definitions?

Regets:
I think we tried to get around that. We had people fill out both a number to match up, "My occupation is," and here is a list to pick from, but also do a write-in. Then we tried to make sense of the write-ins and in some cases, where it was obvious, corrected the numbers. So if a person said, "I am not a physicist; I am a," and then put down something else. Even if it was very strange, I think we would have at least placed him in science.

As I said, a lot of the people are becoming top managers and a lot of the top managers are outside of S&E. But a lot of them are inside science and engineering as well. With the exception, for whatever reason, of math and computer science, you can see fairly substantial numbers of new Ph.D.s become top managers within 20 years.

I don't know if anyone has any thought on what is happening to the math and computer science people. It is probably not surprising the engineers lead the way, but then physical and life sciences are pretty identical.

Syverson:
Mark, could you control for sector, because I think managers are much more in the non-academic sector.

Regets:
Here, I have not. This is everybody.
Speaking of sector -- that was a good segue. This looks, by year since degree, at the percent of Ph.D.s who are working in education. This clearly shows one thing. If you really and truthfully want to be in academia, you should have majored in history. The humanities have the highest percentage of people working in education.

I think this does show that we have to be careful in looking at the percentages that are outside of education as an indication of distress. Engineering and physical sciences have much lower numbers in education, and the lower numbers are not just for the newer graduates but for people in mid-career who presumably graduated during better times.

You do see here, in addition, a blip up for people who got their Ph.D.s in the '60s or very early '70s. I think that fits at least the anecdotal stories that this was a good time to get your degree and go out and get an academic job.

Syverson:
Define, Mark, for us what education means here in this graph.

Regets:
Right here, education is everything including being a high-school teacher. But the vast majority are people at universities.

Syverson: This is sector; right?

Regets:
This is sector. So if you worked in the campus Xerox room, you would certainly be included here.
Knowing that question was coming, I thought there would be some interest in breaking the education sector down into something a little more specific, and that is the percent who are in tenured or tenured-track positions.

Here, once again, you should have majored in English or history, or math and computer science which, you are right, may be the story on the top managers. Towards the bottom are engineering Ph.D.s and physical science Ph.D.s. Really, this chart shows the same story as the chart for the education sector as a whole.

You have a relatively flat line for many groups, where, even among the older cohort, the percentages in tenured or tenured-track positions are fairly low.

Kuh:
You see the postdocs affecting the life sciences very strikingly.

Regets:
Absolutely. That is true. If you are a postdoc, while you would have been included in education in the last chart, you presumably are not a tenure track postdoc.

Kuh:
But that is a very steep slope.

Regets:
Absolutely.

Kuh:
It is a steeper slope than anything else. And that is postdocs doing that.

Gaddy:
Why are such low percentages of engineering and physical science faculty tenured that far out?

Regets:
This is all Ph.D.s who are working.

Kruytbosch:
This is to remind the community that you are only talking about a minority of the whole community of physicists, chemists, engineers, and so on when you are talking about tenure track academic jobs. Most of them are working elsewhere and have been for years.

Regets:
The last couple of times I did this, people kept saying, "But can't you break out my field?" So I just went through and created a table of some of the fields. This, I think, is mostly self-explanatory.

Maxwell:
When was this data taken?

Regets:
This is April, 1993. I suppose if you wanted to do some analysis of this, if you compare the numbers in some of the fields that have high involuntary out-of-field rates, and high measures of employment distress, like sociology, you see some of the greatest changes between people in early or mid-career in the 1960's cohort.

The same is really true of physics although, even there, there is not a majority in academia.

Fechter:
If I remember my data right, Ph.D. production kind of peaked about 1971 or 1972 in the science and engineering fields.

Regets:
Right.

Fechter:
And then it kind of dropped by 10 percent or so over the next three or four years. That seems to be correlated with the comparison between before '73 and '73 to '77. You find that the percentage of the tenure track or tenured positions drops between the two cohorts, the earlier cohort being much more likely to be tenured.

Regets:
Much larger graduate programs with people trying to beat the Vietnam draft.

Fechter:
That was the '60's.

Regets:
Although, as you said, it peaks -- actually, I thought the peak was exactly '73, wasn't it?

Fechter:
I wouldn't argue with you on that.

Regets:
But certainly '72, or '73 was right.

Kuh:
Until the most recent years.

Regets:
We are now either in the middle of a new upward slope or at a new peak, depending on your viewpoint.

Fechter:
My guess is that the latter years are the years when the sectoral distribution of employment of Ph.D.s changed dramatically away from academia into non-academic employment. And that, again, is kind of consistent with the kind of cohort effect you show here.

Regets:
Any other questions?

Tupek:
Thanks, Mark. I would like to have the open discussion between now and 3:00 and then have Catherine Gaddy begin her discussion as scheduled at 3 o'clock.

OPEN DISCUSSION

Tupek:
The topic of the open discussion is what additional data collection and analysis are necessary and feasible to address the needs of the policy community.
I guess, based on some of the things we heard today, we may want to broaden that beyond just the policy community because, clearly, our data are serving other constituent groups as well. So I wouldn't necessarily restrict it to just the policy needs.

Some of the things that we have heard today relate to non-U.S. citizens and relate to the graduate support mechanisms. More information on the master's-degree-level people. We also looked at department policies and a range of other things.

In the past few meetings we have had, we were discussing at quite some length the need for more data on the new Ph.D.s. I would like to think that, in some way, these meetings have contributed to the new CPST effort in collecting more data on the new Ph.D.s from a number of societies.

So I think we can make a difference working together and looking at where the most critical data needs are. I think it is worthwhile to discuss where are the critical data needs at this point and if we have new data collection efforts, should they be Federal efforts, should they be some combination? Are there things the societies can do at this point?

With that, I would just like to open it up for discussion.

Russo:
Just one point of note that may be somewhat of interest. The data that we collect, one of our unique niches, is that we have students coming through the pipeline. They may or may not be successful in terms of completing a degree and walking out with a Ph.D.

What level of interest is there about the individuals who don't make it through but end up employed? Where are they going? What interest is there in these topics?

Syverson:
That was one of the things I was excited about when I saw your original write-up and I was hoping you would have more opportunity -- and I know we cut you a little short -- to talk about that part of it. I think that comes up very much in Ellie's group when we were talking about funding and what happens to people.

We have, in the SED, length of time to degree for people who complete their degree. But I think there is the big issue of what is the impact of the funding mechanism on whether or not people complete at all. Ellie?

Thomas:
I was going to ask about what data you have and if you have looked at them on funding mechanisms.
But Charlotte tells me they are not --

Russo:
We are trying. The funding or financial-aid data is a very tough area for us. The bridges that need to be developed within the institution for us to obtain data longitudinally over the year or even within a given year is often very difficult for graduate programs to pull together.

There isn't a central location for graduate funding mechanisms at a graduate school. It is very different from undergraduate programs. Particularly, a graduate office most likely will have to tap into payroll systems to find out what mix of RA and TA dollars the students are paid over a given year. The degree to which an institution is able to pull that information together for us is quite varied and it gives us lot of concern when we look at our data.

Neuschatz:
Actually, a group of us have done some studies on attrition in graduate school. I think we actually have another proposal in front of NSF to look specifically at the points in time, the gates, if you want, within graduate schools that cause different rates of attrition, when it happens, around what events in the graduate education process and especially looking at it by gender because there is some evidence that there are differential rates of attrition by gender and that it comes at different points in the graduate career.

So there has been some stuff on it, but, it was at selected schools or in selected disciplines.

Russo:
Did you link that to financial aid?

Neuschatz:
No; we didn't. But that is something that could be done.

Russo:
That is what we are trying to do. We do have an initial look at some retention patterns, one year, two years out, and so on. But we want to try to explain what causes a student to leave in terms of particular financial-aid packaging or where the student is in terms of what we call candidacy status which is one cut that we do have. We are very interested in looking at financial aid, but it has been very difficult.

Kuh:
A number of years ago, OSEP did a white paper study about measuring attrition which I think may actually get published in June. What we found was that because of this problem that I think Peter mentioned, every institution counts time-to-degree differently, that it was very difficult to get a uniform measure of attrition because there were different models and you couldn't add them up into a nationally consistent definition.

On the other hand, individual institutions try to track attrition and pay attention to it. So it may be one of those cases where we can't do something nationally but we could get impressionistic data with different definitions locally.

The other thing I should say is that we ran into that problem with time-to-degree as we do it in the Survey of Earned Doctorates. It turns out we ask people when they first entered a doctoral program. We don't ask them which doctoral program.

The computer science programs got especially upset by this when we published the time-to-degree data in the research doctorate study. Interestingly enough, it was the first time anybody had looked at themselves, which goes to show that if you produce data when it is related to something people care about, then they look at your other data, too.

They said, "Well, your time-to-degree measure is much too long. It turns out that there are many of those programs where people stop and do co-op work. That doesn't count in the department measured time-to-degree, although to a student, that is years of their lives.

You have to be very clear just what you are measuring. Again, it might be very interesting to ask departments what they think time-to-degree is as compared to what students think time-to-degree is.

Kruytbosch:
We will have, in '95, a sample of people who got bachelor's and master's degrees in '91 and '92, and correct me, Linda, if I am wrong here, who were surveyed in '93 and will be followed up again in '95. So we have a national sample, reasonably large here, across all those science and engineering disciplines, to see whether people were enrolled full-time, were enrolled part-time, went directly out to get a job, where are they in '95?

Of course, we hope to follow them again in '97. So we will have a little glimpse into the black box of graduate education in there. That is something to look forward to. We are even thinking of augmenting that sample somehow, depending on how it works out.

Regets:
Another area that we have talked about at our other meetings, and Steve Nelson brought it up indirectly in his talk, is the lack of demand side data. I know a few of the associations capture a little bit of that, but that is really something we have tended to ignore, and, I suspect, will continue to ignore unless there is some groundswell support for it.

Fechter:
You have the occupation/industry matrix which is kind of a snapshot of employment at a moment in time. What you don't have, and it would be very nice to have, would be the flow data, your replacement demand, new positions that are opening because people have retired or died or moved on to different occupations and new positions that open up because of growth in the employment demand, or some measure of demand.

Nobody, as far as I know, has that kind of information around. That is really the number you need if you are going to start talking about matching demand with supply, if you are thinking of supply as the new Ph.D.s or the new graduates coming out.

Tupek:
Steve brought up the question of how many scientists do you need and for what?

Fechter:
I didn't say it to him, but I will when I see him; need, like beauty, often lies in the eyes of the beholder. So your definition of need may not be my definition of need. It is a very difficult question to get a handle on.

What we know well is what people are willing to pay for in the way of positions. That may be constrained by budgets, but that is life. We all live with that. So maybe these employment numbers are, in fact, what we need to look at, at least for that dimension of demand.

Neuschatz:
I know that our problem is that supply is very easy to measure because there is the stamp of the passage through the academic system. Demand is incredibly slippery because it is very hard to determine for what the demand is.

In other words, what is a position for, say, a physicist or a chemist in industry. There are many positions which a physicist or a chemist or an engineer could fill. Where do you assign that position to? It is easier in academia because you have departmental boundaries. But when you get outside of academia, even conceptualizing what, exactly, a specific discipline's demand is is very difficult.

So then what you get is a great aggregate. There, we might be able to come up with what is demand for all science and engineering doctorates, or something which may be a little closer. But then it is so widely aggregated that it is not going to help in terms of a disciplinary match to production.

Fechter:
Michael, that is a very important point about the difficulty of matching occupation to degree level. Some of the pictures that Carlos put up, the slides where you saw a very large proportion of people outside of their fields working in occupations that are not considered their particular degree field.

That kind of information might be able to give us some clues about what is going on there and some idea of which groups of fields are aggregatable.

Neuschatz:
I guess I am saying it is almost like Schrodinger's cat; if you fill a position with a chemist, then it is a demand for a chemistry Ph.D. If you fill it with a physicist, then it is a demand for a physics Ph.D. It is impossible to tell beforehand which it is.

Kuh:
But our data collection is built on being able to identify slots as chemistry slots or physics slots. One of the fundamental problems, I think, in the way we collect data right now is that, as our economy is changing, dare I say, there are a number of general problem-solving skills that Ph.D.s have. When you look at industrial demand, what may be demanded is not so much a physicist or a chemist, but the ability to pose and analyze problems in tractable ways.

In that case, getting back to the question about what beauty is; there are a lot of beautiful people out there and people do not care whether they are blondes or brunettes or whatever.

Neuschatz:
But that is very difficult to explain to the physics department.

Kuh:
But that means when you are trying to answer a question from somebody, like, are there too many Ph.D.s, or are we producing too many physicists, or something like that, it would be awfully nice if we could just be brave and say we can't know.

Brown:
To change the subject a little bit, I noticed in Charlotte's presentation, she was giving all of the questions. At least one of them had to do with foreign graduate students and their support. Alan said he was writing a paper in which he was discussing the trends in the number of foreign graduate students.

Do we have enough data to even know how many foreign students there are by country of origin or by field and, if not, do you feel a need for more such data to answer the questions that you and others, of course, have posed.

Fechter:
My colleague has put together, based on stuff from Mark Regets, I suspect, material on how many foreign Ph.D.s and so on we have and what fraction of the workforce they represent.

Brown:
That is after they have got their Ph.D.s.

Fechter:
Yes. We have, of course, the data on the graduate schools and how many foreign enrollees are in graduate programs and how many get Ph.D.s. From the SED, we have information on what is going on there.

The question is how much is too much, or how much is not enough. The fact is, if the number is going up or going down, so what? It is all a matter of perspective and the perspective that you need to have is what is it you are trying to achieve as a policy objective.
The paper that we are writing will show that there are lots of objectives out there controlling supply. Immigration policy is one of these policy objectives. There are equity issues. There are issues of making sure that our science and engineering enterprise is healthy. That is another objective.

Frequently, these different kinds of objectives for a given policy will meet some of these objectives and be counter to others of these objectives. So it is not clear how to set policy unless you first specify what it is you are trying to do, what your objective is.
If you don't know your objective, if you don't have a goal, if you don't know where you are going, any policy will get you there.

Brown:
But you seem to saying, for the policy questions you are interested in, you seem to have the right amount of data. You see no big lack.

Fechter:
The kind of information we have can be enhanced and refined, but, by and large, we can make some meaningful statements or at least point out where there are conflicts that need to be addressed with the data.

Obviously, more is always better but it seems to me it depends on the price you are willing to pay and what you are trying to do. If, in fact, the objective is to take care of the science and engineering enterprise and if, in fact, to do that you need the very best talent you can get to come into science and engineering, then it shouldn't matter whether they are foreign citizens or Americans.

Kruytbosch:
Especially if the best ones stay here.

Fechter:
That is one of the things we are going to look at. That is exactly right. So, for that objective, the question of citizenship status is irrelevant.

Kruytbosch:
I would like to raise a question, Mike Neuschatz' very interesting question on skills because it has come up a number of times here. Wouldn't it be nice if we could get a list of skills somehow, one of them being cutting edge knowledge of your discipline or your field.
That sounds like a nice question to have in the SDR. I wonder whether anybody thinks that it is feasible to generalize that kind of question because we have never really had that.

Fechter:
That is the kind of thing that the societies are probably in a better position to take on. I used to push for the SDR. I really changed my mind on that one because it seems to me when you are dealing with a questionnaire that is supposed to be serving all the disciplines, it really does constrain the degree of specificity with which you can go on and ask questions about a particular view.

Kruytbosch:
The way that he did it was -- in fact, he even had a little space there for whatever your field knowledge is. So then the others were clearly more generalized kinds of skills where you have been taught to teach or you have worked in a team, or whatever.

I don't mean, do you know how to do a Fourier transformation or what.

Fechter:
Exactly right. I'm sorry. I misunderstood your question.

Kruytbosch:
That would be, indeed, a completely different can of worms. But would that kind of stuff be of interest across the board, because I think there was some interest in that particular table. So if you could give me a copy, I would appreciate it.

Tupek:
We are going to have another chance to have "around the table" at the end. Why don't we move on to Catherine Gaddy. She is going to update us on the Sloan project measuring workforce experiences of new Ph.D.s.

UPDATE, SLOAN-SUPPORTED PROJECT

Gaddy:
At the last meeting, which many of you were at, I gave a presentation on this particular project. The Sloan Foundation has provided initial funding and most of this funding is being passed on to six fields, represented by professional societies, to improve the data collection of the employment experiences of recent graduates. The reason those particular fields were selected initially was because they had already collected some data from or about recent doctoral graduates. Only four societies -- AIP, ACS, AMS and APA -- actually have experience collecting data from doctoral graduates.

While the AAU/AGS (Rocco's data) is looking at experiences of graduate students, and the UC Berkeley study (described by Syverson) is following up doctorates ten years after graduation, we are following up six to nine months after graduation -- a critical time frame not currently covered. At the Federal level, of course, the Survey of Earned Doctorates surveys before graduation, and the Survey of Doctoral Recipients samples (about 11%) the entire universe of doctorates every two years. So we have this mosaic of people trying to fill different gaps and coordinating their efforts.

We are now trying to obtain funding to bring six additional fields into the effort. Three are in the social sciences: economics, political science and sociology; in the life sciences: physiology and microbiology; and for engineering, ASEE will choose several subfields. We are trying now to get seed money for these fields/societies who have never collected data from recent graduates before. Some of them have done surveys but they have never done surveys of doctoral graduates, which has its own special considerations; not the least of which is tracking down addresses from a relatively transient group.

With the portion of the project currently underway and funded by Sloan, the user needs analyses are now completed. We went to faculty and students, and, to some extent, to policymakers to ask what kind of data would be helpful, i.e., to fill this gap for time-critical data in between the SED and the SDR. We have gotten some useful information, even things as practical as what would students be willing to pay for the data (if anything), where would they like to see the data (on the Internet or publications), what kind of data do they think they want, etc.

In a previous meeting, we collated all the variables that the societies have asked about before. There were about 30 employment variables as well as some demographic and education variables. We prioritized those. Yesterday, we started compiling survey questions to tap those variables. On this slide, I listed the variable names for which we are currently writing survey questions. What we are trying to do is a core set of variables, a core set of survey questions. Then the societies can add. But these core questions will be the ones that the societies will give to CPST for meta-analysis across all the fields. We are trying to get some consistency but recognize that there is going to be lots of field-specific detail that the societies are also going to want to get.

Even for some questions, like sector and work activities, we are trying to do a data aggregation strategy that might map to the SDR, but then let the fields have much finer detail for their purposes.

We have gotten some really good input from statistical and survey experts thanks to some leads from NSF people. One of the biggest issues in survey methodology is sampling. The good news is we are not going to sample. We are trying to get the whole population. But another major consideration is response rates. Our mantra has become "Get a better response rate."

It is tricky, though, because, for example, Roman Czujko of AIP has pointed out that some of the graduates never get the questionnaire because we cannot trace down their current address. So are they really "non-respondents?" How do we treat this?

Some of the societies now compare demographic characteristics of their sample to demographic characteristics of SED respondents, so there are some ways to calibrate it which I think will help us to some extent. I also think we may try to increase response rates from 50 to maybe 60 percent. We haven't decided if we are going to do the Carolyn Shettle "incentive" routine, i.e., put some money in along with the survey and you get a certain percent increase -- we are looking at the cost benefit of doing phone follow ups, and the AIP staff follow up with the faculty advisors for at least some information on graduates' employment if all else fails.

So there are lots of creative ways we may be able to nudge up the response rates. But I think one of the other main reasons that the society data has been questioned in terms of any Federal decision making is that it has been inconsistent. People are measuring different things at different times, which we will try and take care of in this project.

Ultimately, the societies will provide field-specific analyses in their publications. We will do higher level broad field analyses that might help people like Jenny Bond doing the S&E Indicators and so forth.

The self-supporting part to keep this effort going over time is a little tricky. In some of the societies, it is institutionalized. Once it gets institutionalized, people don't question it as much. When you are trying to introduce it, they say how much is this going to cost and what is the benefit?

Some of these place where this data collection on new grads is institutionalized have pub sales, like student guides, to offset it. This is an issue for some of these new societies coming on board to consider. So we are trying to work with them to think of entrepreneurial ways to get revenue from Internet or other means to keep this going because it would be nice, at least on a biannual if not annual basis -- the goal is annual -- to be able to update this in a consistent way.

Fechter:
The startup costs of this kind of a survey are much higher than the continuation costs. So the idea is that this initial support from Sloan and from other outside sources, if it comes through, is to cover the fixed costs of developing the instrument and the methodologies and then, hopefully, the incremental costs of continuing that survey in the remaining years will be small and could be covered by appropriately imaginative entrepreneurial activities.

Gaddy:
In Salant and Dillman's latest book on surveys which Mary Jordan of ACS recommended to us -- she was one of his students -- it says it costs about $7,000 to get 500 responses for something that is already in place. So, again, we are hoping that people get their databases and everything set up. Thanks for your interest.

Tupek:
Thank you. I am going to turn over the next session to Jennifer Bond who just led us through another production of Science and Engineering Indicators, and is currently basking in all the media contacts and coverage.

If you must slip out early, I would like you to pick up a copy of our proposed agenda for next time so that you can give us some comments. We will spend a couple of minutes at the end talking about it.

AROUND THE TABLE

Bond:
Basically, I wanted to tell you that Indicators is out. If you haven't picked up your copy, please do. It is also already on the Internet. We are pretty proud that we had it on the Internet 24 hours after the President released it.

We are trying to address users' needs in a variety of ways. Even though we just published the S&E Indicators report, we need to start planning for the next one. So my goal is to hear from you; not just the issues that we talked about today that are very important, but the broader issues, too.

Are there other areas of concern from your perspective? I would like to go over a few of the emerging trends that we started to follow in this Indicators. The report is very extensive with a lot of issues and data, but we see some emerging areas that we think need to be followed.

I think most people know this, but we are responsible for helping the National Science Board provide a comprehensive report every two years which is essential for anyone who wants to know about the status of science and technology in the United States.

So we are trying to be comprehensive. We are also trying to address policy issues. So that is what I want to hear from you today because it takes time to develop data and indicators for new issues.

In this last Indicators; we are trying to follow the whole pipeline of scientists and engineers from pre-college to higher education, and then scientists and engineers in the workforce. We also have a special chapter on academia because this sector is very important in the research context.

Also, we are looking at national patterns of R&D resources, linkages and characteristics of the S&T system. We also did a special chapter on technology and diffusion. In this volume, we have spent a lot of extra care trying to look at social and economic impacts, in two different ways, looking at public attitudes and understanding and also looking at some economic and social returns on R&D investments. This is an area we would like to investigate further and it is one of the topics we hope to continue analyzing.

We spent a lot of time this morning looking at graduate education and S&E employment trends. We know that the R&D expenditures, as was mentioned this morning, make a differential impact in various fields and technologies. We want to both be tracking what the R&D expenditures and trends are and also examine their impacts.

R&D funding by the Federal government has a major impact on graduate education. It really makes a difference which agencies' budgets get cut. For example, the Defense Department has a major impact on engineering. So we want to be following R&D trends and all of the points we raisedrelated to employment.
We are also interested, as I said, in the social and economic impacts. I would say that we don't have a very good handle on this topic. We have started, and we are really interested in continuing to look at public understanding of science and technology.

Dr. Lane is very interested in looking at the role of scientists as communicators. He feels they should be able to address the public, explain their science, and why it makes an impact.

Information technologies is a major new area. We have found that that is where the S&E employment growth has been in recent years. Information technologies are making a major impact on scientific publication patterns and modes. I would be interested to know from the societies' point of view, if the publications in your fields are starting to publish in electronic form? Astronomy, for example, is doing this.

New information technologies are going to have, we think, different impacts on the way we measure the outputs of science and inter-exchange and communication patterns of nonscientists as well. We found that about 55 percent of all adults have access to a computer at home or at work. If you look at college-educated Americans, the proportion is over 80 percent.

If you look at the academic Ph.D.s alone, it is in the 90's. So this is a key area where we are quite different from almost every other country. So we need to have a better handle, I think, in this area and follow that.

The service sector, we know is where much of the growth in the nonacademic jobs in science and engineering is now. So we decided that we better get a better handle on what is happening in services, and we have greatly improved our industrial survey for R&D expenditures to try to find out what are the growth areas in services, where are we spending R&D funds and what is happening there.

I mentioned briefly international comparisons. In each of the chapters, we have greatly expanded the international coverage. We know that economies are increasingly global. Science and technology, in fact, always have been global endeavors and these trends are increasing. It is easier to interact internationally because of the Internet. We are very interested in looking at S&T cooperation. In the new Indicators, we found that cooperation and co-authorship have gone up dramatically in the last couple of years.

Also, we have started following not just the G7 countries, which have been our traditional areas of focus, but we have begun to examine some of the newly industrialized countries like Taiwan, Korea, and starting now Latin America. These economies are rapidly increasing their S&T capabilities. So, both from the partnership and cooperation, as well as the competition point of view, we want to be able to follow those.

These are some of the areas in which we are interested. We spoke about foreign participation, foreign immigration. Mark Regets of SRS's Personnel group was at a Congressional hearing just this morning concerning what changes possibly should be made in S&E immigration regulations. Mobility between the different countries is a major area of interest. It is a really difficult topic to understand if you don't do it on an international basis.

These are some of the topics that we think are important to follow. I would really like to hear back from you. Do you think these are some of the key issues to be following? Are we leaving out major areas that you see would be key areas that we need to start to plan to address?

Charlotte, I know you have something to say.

Kuh:
Actually, I was thinking about globalization. One of the things that would be awfully nice and it is really apropos to something Alan Fechter said earlier is if there were some way to talk about the flows of scientists and engineers and students in both directions.

It just seems to me that the U.S. isn't the only game in town anymore and that if we are going to be keeping track of both competition and cooperation, I think we may have to talk about flows to and from, say, Taiwan and Australia because those are students who not going to the U.S.

That is a science and engineering capability that it is important for us to know what it is and how it is growing. So I think that it would be good to learn more about the international flows -of-people.

Fechter:
You are looking at international flows of R&D dollars, so you know something about how many dollars come in from foreign countries to American-based enterprises at least and we have some sense of how many American dollars are going out overseas. We need an equivalent to that for the human resources.

Kuh:
Wasn't COSEPUP going to do some kind of a look at transnational migration of science and engineering people, maybe at the doctoral level? There was some talk about that last year.

Fechter:
I have been waiting for a report that was supposed to come out in August, I think, of this year. I am not sure it was a COSEPUP report, but there is something going on at NRC.

Bond:
But it hasn't come out yet.

Kuh:
There is a report on engineers that is going to come out in the next two weeks.

Fechter:
That may not be the one, then.

Kuh:
There was the conference you did, and that has come out.

Fechter:
Oh.

Kuh:
That came out.

Fechter:
No; then that is not what I am thinking of. I don't know where it is going on but it is a multi-volume publication, report, where several volumes have already come out. There is one more volume left. I think it dealt with transnational issues.

Bond:
We are waiting for several Sloan Foundation studies on international flows.

Fechter:
Yes; that's right.

Bond:
So we are hoping to be able to utilize those studies of foreign participation.

Brown:
And we have compiled a lot of data about the production of Ph.D.s in other countries. We don't know where they go after they have their Ph.D.s.

Bond:
I think that NSF has made a major contribution in this area to look at the other countries and really try to, in a very comparable way, examine their production of S&E students in their own countries and their production also in the U.S.

Also, looking at some of the information on foreign students in the new Indicators, we did start to examine the question of the training of foreign students in these other countries. We know that Japan and Australia are going to use that as a strategy. They see it as a strategic advantage for the future.

Lehming:
One addition to this. An approximate view of the question of globalization can be obtained from the portion of the Indicators, in which we discuss international collaboration as evidenced by co-authorship and try to include not just the United States with some other countries but cross-national ties so that you can look at which country is collaborating with which other country.

Bond:
Yes. When you get your report, you will see that Rolf Lehming has done a really excellent job of expanding the analysis of who is cooperating with whom in wide variety of countries.

Syverson:
Jennifer, one very simple statistic that I tried to get pulled together was the number of doctorates, not even only in science and engineering but just the number of doctorates produced internationally by various countries. Is some of that in Science and Engineering Indicators this time?

Bond:
Yes.

Syverson:
Because that is surprisingly difficult. I figured that there would be a report, a source. It is one of these statistics that you think would be easy to get.

Bond:
No; it's not very easy.

Syverson:
Of course, there are definitions of doctorates that are different. There is the whole definitional problem. But I just sort of figured that there would be an easy way to pull this together. And, by golly, there is not.

Bond:
No. I think we have been developing one of the most comprehensive databases at NSF because we actually were in contact with all of these individual countries. UNESCO has something like that but it is at the graduate level; it is not only for the doctorate level.

Syverson:
Exactly.

Bond:
Also, just to mention, finally, the OECD. We have been working with them to do a better job, to expand their activities in the human resources field so that they cover not just R&D scientists and engineers. The OECD and Eurostat are going to try to collect data across all fields, occupations and sectors, which is beyond the doctorates.

But I think there are some good signs for progress in the area of human resource development.

Kuh:
Another thing that I think would really be interesting. There are very different institutional arrangements in different countries about where science is done. In particular, I heard a paper last week by some people from UCLA, who are looking at research institutes and the extent to which they are tied to industry, or tied to universities, or are independent.

So far, they are looking at just one kind of patent which is now fairly pass as an output measure, but the point is that the kind of institutional arrangement and the sort of linkage to universities seems to have a lot to do with the productivity of science.

It would be very interesting to have some multinational comparisons there. They found that the stand-alone research institution is not really terribly productive as compared to the American model. This gets back to the graduate education question because you may say that there are too many Ph.D.s floating around, but if you then say, "Well, okay; let's separate research from graduate education," you may, in fact, be cutting off something that, in fact, is very important to the way we do science here without thinking about it.

Fechter:
You have to adjust those productivity numbers, I suspect, for quality because the differences across countries in incentives are also important. If, in fact, the United States is a country in which publication and bringing in grant money and research is important, then you are going to see high productivity.

If, on the other hand, publication is not a major incentive in other countries which have stand-alone institutions, then they may not publish as much, but it may be better.

Kuh:
You can look at it multidimensionally with patents and publishing, but you need some measure of the output of science.

Fechter:
Right; but you have got to control for differences. It is not easy to compare.

Bond:
I think this whole area of linkages and partnerships, strategic alliances in industry and industry-university-government linkages is an area that is going to bear watching.

Fechter:
You mentioned R&D, once in the context of service industries. I think the other was in the context of foreign countries and so on. I guess, in the context of today's discussion, we may see a larger fraction of our science and engineering workforce going off doing something that is not necessarily closely linked to R&D but that takes advantage of certain kinds of communications, problem-solving skills, that you may have to think about new indicators to look at in addition to R&D and, perhaps, even publications to get at what kinds of productivity is going on, particularly in the service industries where they may be doing something very different.

Bond:
Actually, we found that the service sector is becoming increasingly important in many other countries.

But before anyone else leaves, I want to say three more things. Please pick up your Indicators. When you are reading it, bear in mind that we are in the mode of receiving your ideas. I think it is great to hear them now, but we are very anxious to hear from you when you go back, too. If there is something that strikes you as you are actually using the report in your own work and you find that there isn't data or that there is, and it is useful, we want to hear back both on the positive and negative aspects what information is useful to you and also do you need something else.

Additionally I would like to have the results of all of your work. It sounds like it is pretty exciting. There are a number of studies going on which would be very helpful and useful to our work. We are a user of data as well as a provider of data, so please keep us in mind when you finish your studies. We are very interested and anxious to have those results so that we can enrich our understanding.
So thanks very much.

Tupek:
Thanks, Jenny.

END SESSION III