NCSES Home > Survey of Doctorate Recipients

Updates are shown in order from most recent to oldest. skip to initial notice

Initial Notice: 9/9/2010

SRS has uncovered a data problem related to individuals reporting more than one race in the Survey of Doctorate Recipients (SDR). This problem affects many previously released publications and products—for some the ramifications are significant and for others less so—and SRS wants to alert the community and its data users to the issue as soon as possible. We apologize profusely for any inconvenience this problem may cause you.

The 2000 Census incorporated a new question for race that allowed an individual to check more than one category. For example, an individual could identify himself or herself as American Indian and white or Asian, black and white. OMB required all federal surveys to implement a similar question quickly and SRS introduced collecting data on multi-race individuals in its surveys between 2000 and 2003.

How to report the multi-race data (there are hundreds of possible combinations) has been a challenge for all agencies (and researchers using the data). OMB has provided some guidelines on how to report the data, but has left agencies substantial latitude how to do so. SRS developed procedures to report race/ethnicity data for multi-race individuals, which have been evolving over time. Initially, when there were relatively few such individuals, they were reported in an "other" category (which usually included those whose race/ethnicity was unknown and Native Hawaiians/Other Pacific Islanders, another very small group that was introduced with the 2000 Census). More recently, as the number of respondents reporting more than one race has increased, SRS has been able to report separately on the number of those reporting more than one race in some instances, such as in the Summary Report for the Survey of Earned Doctorates (SED) beginning with the 2007 data.

The SDR is a longitudinal sample survey; the SDR sample is drawn from the SED. For the 2003 survey, to insure sufficient sample for minorities in the SDR, SRS and its contractor, NORC, established procedures to maximize inclusion of minority individuals in the SDR sample. One of these procedures involved detailed specifications for assigning individuals in the sample frame who had reported more than one race in a previous survey to a single race category for the purposes of sampling only to maximize the sample for small race/ethnicity categories. The most simplistic way of describing what was done was to take an example—if I designated myself as multi-race composed of American Indian and white in the SED, then for sampling purposes the specifications we and the contractor developed assigned me to the non-Hispanic American Indian strata.

SRS expected the detailed data files received from the contractor for the 2003 and 2006 SDRs to include all the race detail for each individual so that data for multi-race individuals would indicate each of the races a respondent had reported. However, for most multi-race individuals on the file, this did not happen; rather the contractor provided data files which classified most such individuals using the sampling classification, meaning that non-Hispanics who reported more than one race are generally shown in the files as having reported only one race. To continue the example, for the data file my record should have indicated that I chose American Indian and white, but instead it showed only American Indian. In publications using the data I would be included in the American Indian category rather than the multi-race category. As a result, publications using the race data from the 2003 and 2006 SDRs have overestimated the number of blacks, Asians, and American Indians, and have underestimated the number of multi-race individuals.

The magnitude of the problem is greatest for American Indians/Alaska Natives (AI/AN) as the following illustrates.

This error has substantial ramifications for a number of SRS products. The SDR is a component of the Scientists and Engineers Statistical Data System (SESTAT); thus 2003 and 2006 SESTAT data files are likewise affected. SDR and SESTAT data on variables other than race and ethnicity are not affected.

SRS is in the process of identifying what has been impacted, but at the least the error impacts to varying degrees:

• Chapters 3 and 5 of Science and Engineering Indicators for 2008 and 2010
• Women, Minorities, and Persons with Disabilities in Science and Engineering, 2007 and 2009
• Three SRS InfoBriefs
• Public use and restricted data files provided to SRS licensees for the 2003 and 2006 SDR and SESTAT
• Detailed Statistical Tables for 2003 and 2006 for SDR
• Data provided to CEOSE
• Data provided for one table to the U.S. Statistical Abstract

SRS has just begun to address the problem, but has developed a plan for moving forward. The following is a brief summary of the steps being put in place.

1. The contractor will prepare new data files (this will not be completed until late September) and SRS will conduct a stringent quality control review of the new data files.
2. SRS staff has assembled a list of all products affected by the data file error and the specific text, tables, and figures affected in each product.
3. The problem will be addressed for every affected product but the strategies for doing so will vary based on the magnitude of the problem, available resources, and the availability of alternative sources of the correct information. The options for addressing the problem range from a reissue of an entire publication to posting a notice on the SRS website pointing to a product or a publication where the correct data may be found.
4. SRS developed and begun implementing a communication plan informing a broad range of data users and interested parties (both internal to NSF as well as external) of the problem. The plan will include keeping users and interested parties informed of progress in rectifying the problem as well as informing them when corrected data are available. A notice went up on the SRS website on August 30 and discussions with NSF's Committee on Equal Opportunity in Science and Engineering have taken place. Numerous data users and government officials are being contacted.
5. As part of the outreach to interested parties SRS will also solicit ideas for possible other products related to American Indians and multi-race individuals that could be produced. For instance, SRS will try to expedite release of InfoBriefs for the 2008 SDR and the 2008 National Survey of Recent College Graduates, potentially with a focus on race and ethnicity.
6. SRS has developed an inventory of American Indian groups that need to be notified and solicited for input for new/revised products, and these groups have already been contacted. SRS is continuing to seek out other groups to contact of the data issue.

SRS will keep the community informed of progress in identifying the impact of the problem, and will notify data users of the corrective steps it plans and its implementation schedule. SRS is extremely distressed about this error and will institute additional procedures to ensure that such a problem does not occur in the future.

Lynda T. Carlson, Ph.D.
Director, Division of Science Resources Statistics
National Science Foundation
lcarlson@nsf.gov