text-only page produced automatically by LIFT Text Transcoder Skip all navigation and go to page contentSkip top navigation and go to directorate navigationSkip top navigation and go to page navigation
National Science Foundation
OverviewHow to Trust a SurveyScience & EngineeringIncome DynamicsSocial LifePolitical PulseQuestionable FutureClassroom Resources
Questionable Future
QUESTIONABLE FUTURE

Declining Response Rate, Rising Costs

Reverse animation of the coverpage

If costs rise and fewer surveys are funded, and if people do not take the time to participate in scientific surveys, we will know less and less about our society over time.

Credit: Michael Greenland, National Science Foundation

The future of surveys as a reliable means to measure trends is in doubt. The response rates for surveys have been declining. E-mail surveys, touted as being convenient, have shown significant sample bias and non-response problems.

Response rates are often used as a measure of the quality of survey data because non-response is often not random. For example, the U.S. Census Bureau finds that single-person households have a much higher “not at home” rate—and therefore a lower response rate—than multi-person households. This type of nonrandom non-response could skew sample data and lead to under-representation of certain groups unless efforts are made to include these respondents. Therefore, researchers take declines in response rates seriously because in general, the higher the response rates, the more reliable the results.

People today seem more likely to say no to a survey taker due to the sheer quantity of requests for their attention, the possibility that a survey may be a sales pitch in disguise, disinterest in the topic or an unwillingness to give honest and thoughtful answers. Telephone sales pitches and phony or biased surveys have also taken a toll on people’s willingness to participate in legitimate, scientific surveys.

In addition, each specific mechanism for collecting survey data has its own problematic aspects. In-person surveys are hampered by a distrust of strangers, along with the new, gated residential communities and security-conscious apartment buildings. Telephone interviewers are hampered by answering machines, caller ID, cell phones and the difficulty of finding people at home. The response rate of mail surveys is affected by the lack of personal encouragement (which interviewers provide), busy schedules, an increasing volume of “junk mail” that gets ignored and the tendency to find longer surveys daunting when they are presented on paper. Similarly, creating random samples for e-mail or Internet surveys is difficult, and people find it easy to turn down requests for participation.

As a consequence of declining response rates and other factors, costs are rising—for conducting surveys, maintaining the resulting data and making them easily available to various user communities. Moreover, continuing investments in long-term surveys are often not as attractive to many funders as new research because the continuing commitment they represent can preclude investments in hot new areas. Consequently, long-term surveys—which become more powerful with each new addition—are faced with reducing sample size or survey frequency, or finding new methods to track opinion.

Any opinions, findings, conclusions or recommendations presented in this material are only those of the presenter grantee/researcher, author, or agency employee; and do not necessarily reflect the views of the National Science Foundation.
Surveys: Tracking Opinion Special Report