This working paper assesses the reliability of institutional reporting of NFRs in the GSS and also the effect of 2010 data collection changes to improve the accuracy of NFR reporting. Overall, the 2010 to 2012 reporting of NFRs was fairly consistent and became more stable over time. Many institutions appear to have definitions and procedures for identifying NFRs. As expected, there was a considerable increase in numbers and percentages of units reporting NFRs from 2009 to 2010; this coincided with the expansion of data collection on postdocs and NFRs and with the appointment by some institutions of postdoc coordinators who were more knowledgeable about their institution's postdoc and NFR data. Comparison of the NFR data from 2010 through 2012 showed that the reporting has been fairly stable at both the school and unit levels since the expansion of the postdoc and NFR data collection in 2010.
However, some issues in NFR reporting remain. One issue is that the definition of NFRs is not consistent across, or even within, schools. The comparison of postdoc and NFR counts from 2009 through 2012 revealed that some school coordinators are still unable to differentiate postdocs and NFRs, resulting in not reporting one or the other or, occasionally, reporting the same individuals in both counts. Another problem uncovered during data collection and from the NFR Debriefing Survey was that school coordinators had data access issues. In particular, several school coordinators explained that their databases do not capture the information needed—such as type of degree (master's or doctorate) and descriptive job titles—to determine whether researchers have the credentials to meet the NFR definition. In many schools, NFRs are simply not tracked well. More than 100 schools reported having postdocs and no NFRs in each of the past 6 years. These schools likely have NFRs but are not reporting them for various reasons. Other concerns with lower potential impact on the NFR and postdoc data include reporting identical counts of NFRs and postdocs, reporting NFRs and no postdocs, and reporting more NFRs than postdocs. The schools with specific reporting issues will be followed up with in the next GSS cycle.
The number of schools and units providing incomplete NFR data is high compared to the other GSS data. Although GSS respondents were able to provide NFR counts for most units, they were unable to provide any NFR data for about 7% of all units in each year. Of the units that reported having NFRs in 2010, 2011, or 2012, only 47% were able to provide complete NFR information in each of these years, and 5% (346 units) were able to report that they had NFRs but could not provide the count. The fact that 29% of these units with NFRs reported a total NFR count in some but not all years suggests that these data are available but may be difficult to obtain within the institution.
Additional work is needed to address these NFR reporting issues. For example, data reviews with both internal and external sources will continue along with further follow-up efforts with school and postdoc coordinators. Similarly, some schools might benefit from the assignment of a different coordinator—especially one who could distinguish between the faculty/nonfaculty status of staff researchers—to report NFRs. Data collection and quality control protocols will also be improved to notify school coordinators when some units are able to report NFRs and others are not; additional information can be shared between the unit respondents about how to access the NFR data.
NCSES is committed to deepening our understanding of this increasing segment of the academic workforce. Through additional qualitative research, NCSES will continue to seek input from the academic researchers and will work with the GSS institutions to improve the collection of NFR-related data.