Response rate
Encyclopedia
Response rate in survey
research refers to the number of people who answered the survey divided by the number of people in the sample
. It is usually expressed in the form of a percentage
.
Example: if 1,000 surveys were sent by mail, and 257 were successfully completed and returned, then the response rate would be 25.7%.
In direct marketing
, the response rate refers to the number of people who responded to the offer.
In oncology
, response rate (RR) is a figure representing the percentage of patients whose cancer shrinks (termed a partial response, PR) or disappears after treatment (termed a complete response, CR) . In simpler terms RR=PR+CR.
There may be a non-response bias
if the response rate is low.
A low response rate can give rise to sampling bias
if the nonresponse is unequal among the participants regarding exposure and/or outcome.
For many years, a survey's response rate was viewed as an important indicator of survey quality. Many observers presumed that higher response rates assure more accurate survey results (Aday 1996; Babbie 1990; Backstrom and Hursh 1963; Rea and Parker 1997). But because measuring the relation between non-response and the accuracy of a survey statistic is complex and expensive, few rigorously designed studies provided empirical evidence to document the consequences of lower response rates, until recently.
Such studies have finally been conducted in recent years, and they are challenging the presumption that a lower response rate means lower survey accuracy.
One early example of a finding was reported by Visser, Krosnick, Marquette and Curtin (1996) who showed that surveys with lower response rates (near 20%) yielded more accurate measurements than did surveys with higher response rates (near 60 or 70%). In another study, Keeter et al. (2006) compared results of a 5-day survey employing the Pew Research Center’s usual methodology (with a 25% response rate) with results from a more rigorous survey conducted over a much longer field period and achieving a higher response rate of 50%. In 77 out of 84 comparisons, the two surveys yielded results that were statistically indistinguishable. Among the items that manifested significant differences across the two surveys, the differences in proportions of people giving a particular answer ranged from 4 percentage points to 8 percentage points.
A study by Curtin et al. (2000) tested the effect of lower response rates on estimates of the Index of Consumer Sentiment (ICS). They assessed the impact of excluding respondents who initially refused to cooperate (which reduces the response rate 5-10 percentage points), respondents who required more than five calls to complete the interview (reducing the response rate about 25 percentage points), and those who required more than two calls (a reduction of about 50 percentage points). They found no effect of excluding these respondent groups on estimates of the ICS using monthly samples of hundreds of respondents. For yearly estimates, based on thousands of respondents, the exclusion of people who required more calls (though not of initial refusers) had a very small one.
Holbrook et al. (2005) assessed whether lower response rates are associated with less unweighted demographic respresentativeness of a sample. By examining the results of 81 national surveys with response rates varying from 5 percent to 54 percent, they found that surveys with much lower response rates were only minimally less accurate.
As a result of these and other such recent findings, it now seems clear that a low response rate does not guarantee lower survey accuracy and instead simply indicates a risk of lower accuracy. Consumers of survey results are therefore cautioned to view response rates as informative but to recognize that these rates "do not necessarily differentiate reliably between accurate and inaccurate data."
Statistical survey
Survey methodology is the field that studies surveys, that is, the sample of individuals from a population with a view towards making statistical inferences about the population using the sample. Polls about public opinion, such as political beliefs, are reported in the news media in democracies....
research refers to the number of people who answered the survey divided by the number of people in the sample
Sample (statistics)
In statistics, a sample is a subset of a population. Typically, the population is very large, making a census or a complete enumeration of all the values in the population impractical or impossible. The sample represents a subset of manageable size...
. It is usually expressed in the form of a percentage
Percentage
In mathematics, a percentage is a way of expressing a number as a fraction of 100 . It is often denoted using the percent sign, “%”, or the abbreviation “pct”. For example, 45% is equal to 45/100, or 0.45.Percentages are used to express how large/small one quantity is, relative to another quantity...
.
Example: if 1,000 surveys were sent by mail, and 257 were successfully completed and returned, then the response rate would be 25.7%.
In direct marketing
Direct marketing
Direct marketing is a channel-agnostic form of advertising that allows businesses and nonprofits to communicate straight to the customer, with advertising techniques such as mobile messaging, email, interactive consumer websites, online display ads, fliers, catalog distribution, promotional...
, the response rate refers to the number of people who responded to the offer.
In oncology
Oncology
Oncology is a branch of medicine that deals with cancer...
, response rate (RR) is a figure representing the percentage of patients whose cancer shrinks (termed a partial response, PR) or disappears after treatment (termed a complete response, CR) . In simpler terms RR=PR+CR.
There may be a non-response bias
Non-response bias
Non-response bias occurs in statistical surveys if the answers of respondents differ from the potential answers of those who did not answer.- Example :...
if the response rate is low.
Importance
A survey’s response rate is the result of dividing the number of people who were interviewed by the total number of people in the sample who were eligible to participate and should have been interviewed.A low response rate can give rise to sampling bias
Biased sample
In statistics, sampling bias is when a sample is collected in such a way that some members of the intended population are less likely to be included than others. It results in a biased sample, a non-random sample of a population in which all individuals, or instances, were not equally likely to...
if the nonresponse is unequal among the participants regarding exposure and/or outcome.
For many years, a survey's response rate was viewed as an important indicator of survey quality. Many observers presumed that higher response rates assure more accurate survey results (Aday 1996; Babbie 1990; Backstrom and Hursh 1963; Rea and Parker 1997). But because measuring the relation between non-response and the accuracy of a survey statistic is complex and expensive, few rigorously designed studies provided empirical evidence to document the consequences of lower response rates, until recently.
Such studies have finally been conducted in recent years, and they are challenging the presumption that a lower response rate means lower survey accuracy.
One early example of a finding was reported by Visser, Krosnick, Marquette and Curtin (1996) who showed that surveys with lower response rates (near 20%) yielded more accurate measurements than did surveys with higher response rates (near 60 or 70%). In another study, Keeter et al. (2006) compared results of a 5-day survey employing the Pew Research Center’s usual methodology (with a 25% response rate) with results from a more rigorous survey conducted over a much longer field period and achieving a higher response rate of 50%. In 77 out of 84 comparisons, the two surveys yielded results that were statistically indistinguishable. Among the items that manifested significant differences across the two surveys, the differences in proportions of people giving a particular answer ranged from 4 percentage points to 8 percentage points.
A study by Curtin et al. (2000) tested the effect of lower response rates on estimates of the Index of Consumer Sentiment (ICS). They assessed the impact of excluding respondents who initially refused to cooperate (which reduces the response rate 5-10 percentage points), respondents who required more than five calls to complete the interview (reducing the response rate about 25 percentage points), and those who required more than two calls (a reduction of about 50 percentage points). They found no effect of excluding these respondent groups on estimates of the ICS using monthly samples of hundreds of respondents. For yearly estimates, based on thousands of respondents, the exclusion of people who required more calls (though not of initial refusers) had a very small one.
Holbrook et al. (2005) assessed whether lower response rates are associated with less unweighted demographic respresentativeness of a sample. By examining the results of 81 national surveys with response rates varying from 5 percent to 54 percent, they found that surveys with much lower response rates were only minimally less accurate.
As a result of these and other such recent findings, it now seems clear that a low response rate does not guarantee lower survey accuracy and instead simply indicates a risk of lower accuracy. Consumers of survey results are therefore cautioned to view response rates as informative but to recognize that these rates "do not necessarily differentiate reliably between accurate and inaccurate data."