nsf.gov - NCSES Incentive Experiments: NSF Experiences - US National Science Foundation (NSF)
text-only page produced automatically by LIFT Text
Transcoder Skip all navigation and go to page contentSkip top navigation and go to directorate navigationSkip top navigation and go to page navigation
National Science Foundation National Center for Science and Engineering Statistics
Incentive Experiments: NSF Experiences

6. Slippery Slope?



 

Two of the SESTAT surveys, SDR and NSCG, are panel surveys—most sampled persons in these two surveys have responded to multiple waves of the surveys. Will a panel member who received a monetary incentive in a previous survey round develop an expectation of a monetary incentive for participating in the next round of the survey? Accordingly, will a panel member who received an incentive in the last round be less likely to respond to the survey if he or she does not receive an incentive in the next round? These are the questions that need to be answered before applying incentives to members of panel surveys.

Top of page. Back to Top

6.1 An Ad Hoc Analysis of Expectation Effect

Using existing information from SESTAT surveys, Michael White of the U.S. Census Bureau performed an ad hoc analysis of the possible slippery-slope effect on panel members in 2008 (White 2008). The following is a summary of that report.

The 2006 NSCG sampling frame consisted of the following:

  • Eligible cases from the 2003 NSCG
  • Eligible cases from the 2001 and 2003 NSRCG

During the 2003 NSRCG, an incentive was offered to all nonrespondents toward the end of data collection (see section 2.1). The 2006 NSCG followed a sample of respondents to the 2003 NSRCG; as such, some of the 2006 NSCG sampled persons had received an incentive in the 2003 NSRCG data collection. During the 2006 NSCG, an incentive experiment also was conducted (see section 3.2). Therefore, it is possible to identify four groups of sampled persons from which to assess the possible effect of expectation of an incentive:

  • Group 1: Received an incentive in 2003 NSRCG, and received an incentive in 2006 NSCG
  • Group 2: Received an incentive in 2003 NSRCG, and received no incentive in 2006 NSCG
  • Group 3: Received no incentive in 2003 NSRCG, and received an incentive in 2006 NSCG
  • Group 4: Received no incentive in 2003 NSRCG, and received no incentive in 2006 NSCG

Table 17Excel table. summarizes the number of cases and response rate of each of the four groups.

The analysis indicates there may be an expectation effect of incentives. This can be seen in various ways, e.g., Group 1 versus Group 2, Group 2 versus Group 4, etc. But it should be emphasized that the experiment was not controlled—the 2006 treatment group was not a simple random sample of the 2003 treatment sample (only potential and final nonrespondents received incentives in 2006). Consequently, no statistical test or comparison was attempted. The data strongly indicate, however, that a controlled experiment should be conducted to further assess the expectation effect.

 
Incentive Experiments: NSF Experiences
Working Paper | SRS 11-200 | November 2010