nsf.gov - NCSES Incentive Experiments: NSF Experiences - US National Science Foundation (NSF)
text-only page produced automatically by LIFT Text
Transcoder Skip all navigation and go to page contentSkip top navigation and go to directorate navigationSkip top navigation and go to page navigation
National Science Foundation National Center for Science and Engineering Statistics
Incentive Experiments: NSF Experiences

3. Monetary Incentive as Refusal Conversion



 

Monetary incentives can increase response rates from all sampled persons, as seen above. But offering monetary incentives to all sampled persons incurs high cost; therefore, monetary incentives often are used toward the end of the data collection process in an attempt to convert "potential" nonrespondents. Do monetary incentives work with the "most reluctant" sampled persons? Do monetary incentives work on known refusals? Answers to these questions can be helpful in justifying the use of monetary incentives for refusal conversion toward the end of the data collection process.

Top of page. Back to Top

3.1 2006 NSRCG Refusal Conversion Experiment

The 2006 NSRCG Prepaid Incentive Experiment (see section 1.2) ended in August 2006, 4 months into the 2006 NSRCG data collection process. By then, thousands of sampled persons still had not completed the survey after rounds of questionnaire mailing and CATI calling—these are the reluctant sample members. Another incentive experiment was conducted to test whether monetary incentives also work on these members of the sample. In this experiment, approximately one-half of the eligible final refusal cases (sampled members who had already refused the requests to participate in the survey twice) and two-thirds of the potential refusal cases (other sampled members who had not yet responded) were randomly assigned to the treatment group.[2] The remaining cases were assigned to the control group (see table 6Excel table.). Treatment group members received a $20 VISA gift card that would only be activated after completing the survey.

During the experiment, telephone interviewers continued to make calls to members of the potential refusal group. Interviewers who reached a sample member in the treatment group mentioned the incentive to gain cooperation. Interviewers did not attempt to call members of the final refusal group. The final refusal sample members had to call the U.S. Census Bureau to complete the survey and have their gift cards activated. This may have contributed to the lower response rates among the final refusal group compared with the potential refusal group (table 7Excel table.).

These results (table 7Excel table.) show the significant impact of monetary incentives even on reluctant sample persons. For more detailed information on this experiment, refer to Zukerberg, Hall, and Henly (2007).

Top of page. Back to Top

3.2 2006 NSCG Refusal Conversion Experiment

An experiment similar to the 2006 NSRCG Refusal Conversion Experiment was also conducted for the 2006 National Survey of College Graduates, and the results were similar. NSCG is a longitudinal survey designed to provide data on the number and characteristics of experienced individuals with education and/or employment in SEH or SEH-related fields in the United States. The 2006 NSCG sample was the result of a multiphase, stratified sampling from 2003 NSCG respondents, as well as from 2001 and 2003 NSRCG samples. For more information on the 2006 NSCG, refer to the SRS website at http://www.nsf.gov/statistics/survey.cfm.

In August 2006, 5 months into data collection for the 2006 NSCG, sample members who had not yet responded to the survey but were considered to have good contact information were identified.[3] These sampled members then were divided into two groups: the final refusals (sampled members who had already refused requests to participate in the survey twice) and the potential refusals (other sampled members who had not yet responded). From each group, two subsamples were selected, with one subsample to receive a monetary incentive of a $20 postpaid gift card that would be activated upon the completion of the survey and the other subsample to receive only a reminder to complete the survey (table 8Excel table.). The goal of this experiment was to evaluate whether a promised $20 incentive would encourage response among these reluctant sampled persons. As there is a likelihood that some sampled persons may use a prepaid gift card without responding to the survey, postpaid gift cards were used instead in order to offer a higher value incentive to actual respondents.

During the experiment, telephone interviewers continued to make calls to members of the potential refusal group. Interviewers who reached a sample member in the treatment group mentioned the incentive to gain cooperation. Interviewers did not attempt to call members of the final refusal group. The final refusal sample members had to call the U.S. Census Bureau to complete the survey and have their gift cards activated. This may have contributed to the lower response rates among the final refusal group compared with the potential refusal group.

Similar to the results of 2006 NSRCG Refusal Conversion Experiment, the results show the postpaid monetary incentive works well on potential refusals and final refusals. Comparing table 9Excel table. with table 7Excel table., we see that except for the final refusal control group, all NSCG groups may have higher response rates than the corresponding NSRCG groups. This may reflect the fact that 2006 NSCG members had responded to at least one previous SESTAT survey. They were likely to be easier to reach and more likely to respond, while NSRCG members were new to the survey.

Top of page. Back to Top



Footnotes

[2] Congressional refusals were excluded from this experiment. These are generally cases in which the respondent contacted his or her congressional representative or the NSF to opt out of the survey or was threatening in his or her correspondence with the Census Bureau.

[3] Congressional refusals were excluded from this experiment. These are generally cases in which the respondent contacted his or her congressional representative or the NSF to opt out of the survey or was threatening in his or her correspondence with the Census Bureau.


 
Incentive Experiments: NSF Experiences
Working Paper | SRS 11-200 | November 2010