Incentive Experiments: NSF Experiences
7. Summary and Future Work
NSF's incentive experiments have covered a fairly wide range of features of offering incentives. First, the experiments show monetary incentives indeed improve response rates and data quality for the SESTAT population, scientists and engineers with at least a bachelor's degree in the United States. It seems that higher monetary incentives may yield higher response rate gains. Also, these experiments show that even for the most reluctant sample members, the hard (final) refusals, monetary incentives can improve response rates. On the other hand, nonmonetary incentives in the form of an NSF brochure did not motivate sample members to respond to the survey and may have harmed the data quality.
As to the payment methods, prepaid payments may add an obligation to respond and may turn out to be more cost-effective than postpaid payments. The concern that many people may cash a prepaid check without responding to the survey was not a major issue with the SDR sample. In fact, among SDR sample members, substantially more people did the opposite: they responded to the survey and did not cash the incentive check.
The experiments also show that the timing of offering monetary incentives may not affect their impact: early incentives did not increase the effect of monetary incentives on response rate compared with late incentives. This finding supports the offering of monetary incentives to hard refusals toward the end of the data collection process in order to reduce the cost of such incentives. However, since early incentives result in earlier responses and thus save follow-up efforts and costs, it is important to further compare early and late incentives in terms of cost-effectiveness.
To the panel members of longitudinal surveys such as NSCG and SDR, monetary incentives may need to be used with caution. Limited analysis suggests that if a panel member received a monetary incentive in a previous survey, he or she may be less likely to respond if he or she does not receive monetary incentive again in the current survey. This issue needs to be investigated more thoroughly.