Please find below relevant extracts taken from 8 internal documents.
‘eCensus Usability Testing – Round 2 – May 2014’
Author: 2016 Census Data and Data Infrastructure)
Subject of questions
In some interviews, the addition of the new setup questions appeared to cause some confusion
around who to list as "Person 1" when questions began. This did not seem to be due to the concepts
of "persons present" and "persons away", as respondents appeared to understand what these mean
when asked to explain their interpretation, as discussed above. Rather, the issue seemed to be
caused by the flow of the instrument, the switching between the concepts of persons present and
away, and the reliance on respondents to determine who each question loop is referring to.
The issue of respondents not knowing who the questions are about also appeared in other sections
of the interview. Many respondents became confused at some point in the interview about who
"the person" referred to in the question actually was. In some cases, this confusion lead to minor
usability issues that were easily resolved (e.g. as one respondent explained. "It's fine, I think it's just
when it says 'the person', I have to think "oh, that's me"). However in other cases, it was evident
that the lack of clarity in the questions could result in a significant reduction in data quality. For
example, some respondents began answering questions for the wrong person at some point in the
form, and took a substantial amount of time to identify this error. Other respondents doubted
whether they were answering the form correctly when they reached the question asking for their
own name, thinking they had already provided it earlier in the form.
To avoid these data quality issues, and to make the form easier for respondents to complete, it is
recommended that the relevant person's name is inserted into the questions, rather than referring
to the subject only as "the person". This will require establishing the names of all persons present
and away at the beginning of the form.
Recommendations:
28. Establish names of people present and people away at the beginning of the form.
29. Insert the relevant person's names into all questions
Respondent Engagement in Multi-Modal Household Surveys – June 2014
Author: Statistical Services Branch
Personalisation of contacts has been shown to be an effective method of increasing response rates
in mail and online surveys (Cook et al., 2000; Edwards et al., 2010).
Personalising by using the contact name provided by the respondent. The respondent is more likely
to respond if their personal contribution is being directly sought than if the contact appears to be
generic and mass-mailed (Dillman, et al., 2009).
‘Respondent Burden’ – Version current as at 15/10/2012
Author: ABS Methodology
a. Question Order
34.
One strategy to minimise respondents' perceived burden is through the order in which
questions are presented on a self administered questionnaire. Starting a form with an easy and
broadly relevant question will give respondents a sense of self-efficacy (the belief that they can
answer the questionnaire) and a feeling of topic salience.
35.
The importance of placing the most salient questions first is reiterated through a study
conducted by Mullner, Levy, Byre & Matthews (1982). These researchers conducted an experiment
which employed two different questionnaire structures:the first version placed questions of greatest
saliency at the beginning of the survey, whereas the the second version had these questions at the
end. The form with the salient questions first obtained a significantly higher response rate than the
other form, 71.72% vs 67.46%.
‘Response Enhancing Techniques’ - Version current as at 06/08/2012
Author: ABS Methodology
De Leeuw and Hox (1988) investigated the use of personalisation on the cover letter and tested
whether or not personalisation had any positive effect on the response rate. They also found that
the use of certified mail may have a positive effect. They found that personalisation combined with
the certified mailing of the final reminder resulted in a statistically significant increase in response
rate. Both of these experimental treatments were based on the Total Design Method developed by
Dillman.
Similarly, on the issue of personalisation, Linsky's (1975) meta analysis found that out of sixteen
studies dealing with personalisation, three reported higher response rates for the non-personalised
letters, nine reported higher rates for the personalised letters, and three reported more or less the
same responses for both letters. While this may suggest that personalisation leads to a higher
response rate, the types of surveys used for analysis would need to be looked at carefully. This is due
to the fact that while personalisation may lead the respondent to believe that more effort has gone
into the survey implementation, they may also feel threatened by the fact that their anonymity is
challenged (De Leeuw and Hox, 1988) which would have negative implications for surveys which are
sensitive in nature.
Bibliography: The four extracts above reference the following:
Cook, C., Heath, F., & Thompson, R. L. (2000). A Meta-Analysis of Response Rates in Web- or
Internet-Based Surveys.
Educational and Psychological Measurement,
60(6), 821–836.
doi:10.1177/00131640021970934.
Dillman, D. A., Smyth, J. D., Christian, L. M. (2009).
Internet, Mail and Mixed-Mode Surveys: The
Tailored Design Method (3rd ed.). New York: Wiley.
Edwards, P. J., Roberts, I., Clarke, M. J., DiGuiseppi, C., Wentz, R. & Kwan, I., et al. (2010).
Methods to
increase response to postal and electronic questionnaires. (P. J. Edwards, Ed.)
onlinelibrary.wiley.com. Chichester, UK: John Wiley & Sons, Ltd.
De Leeuw, E.D. and Hox, J.J. (1988) The Effects of Response-Stimulating Factors on Response Rates
and Data Quality in Mail Surveys; A Test of Dillman's Total Design Method,
Journal of Official
Statistics, Vol. 4, 241-249.
Linsky, A.S. (1975) Stimulating Responses to Mailed Questionnaires: A Review,
Public Opinion
Quarterly, 82-101.
Mullner, R.M., Levy, P.S., Byre, C.S. & Matthews, D. (1982). Effects of characteristics of the survey
instrument on response rates to a mail survey of community hospitals.
Public Health Reports, 97,
465-469.
1978 Privacy Paper
Document Outline
