|Why does mode matter?|
Choosing the right mode in which to collect you data can be fundamental to the success of your survey.
The mode that you choose can have serious implications for your survey such as whether a business chooses to participate in your survey, right through to whether the quality of your data is fit-for-purpose.
There is never going to be one data collection mode that is perfect for all scenarios.
Each mode has certain advantages and disadvantages that make them suitable for some contexts and not others. The choice of mode usually results from a process of weighing-up a number of issues relevant to the collection, in-light of the resources available.
Key issues to consider
Type of topic: Some topics are more suitable for some modes that others. For example, surveys that require respondents to check their own records, or compile data from their systems are more suited to self-administered modes.
Complexity of topic: The level of complexity of a survey, as well as the detail of response required will often dictate which mode is selected. For example, if a survey requires complex sequencing then an electronic format may be suitable, as it enables the sequencing to be automated. For brief surveys that require relatively straight-forward responses, a telephone interview may be a suitable option.
Respondent preferences and characteristics: Consider the types of respondents and any factors that may influence how the data collection mode fits with the type of respondents. Are you targeting a particular type of respondent for which one mode may be more suitable than others? Do your respondents have a particular preference for a mode? For example, electronic web forms may be the most convenient format for some businesses to provide data, but not for those without easy access to the internet.
Response rate: The choice of mode can have a significant impact on the achieved response rate. Face-to-face interviews tend to achieve higher response rates than other modes because respondents find it more difficult to refuse participation to a person's face, than over the phone or a mail out/mail back survey.
Collection resources: The mode that is chosen affects not only the resources required for the collection of data itself, but can also impact on the resources required for follow-up and the processing of data. For example, data taken by a person, whether it's face-to-face or over the phone can be edited as it comes in, whereas self-administered surveys will need more editing when they come back in. Some modes may typically achieve a higher level of quality than others, such as face-to-face interviews. However, they can also cost a lot more to run. Therefore, it's important to weigh up what is achievable with the resources available and will still produce data that is fit-for-purpose.
Face-to-face interviews: Involves an interviewer travelling to a respondent's business to ask the respondent questions in person.
- higher response rates
- interviewer can clarify questions
- facilitate good relationships with respondents
- costly and resource intensive
- can be subject to interviewer bias and social desirability bias
Telephone interviews: Involves an interviewer telephoning respondents and asking respondents questions over the phone.
- costs less than face-to-face
- can be more timely than face-to-face or paper forms
- calls-backs and follow-ups are relatively inexpensive
- can schedule calls to suit availability of respondents
- not suitable for long, complex surveys
- non-response and partial non-response high compared to face-to-face interviews
- respondents difficult to reach by phone
Paper forms: Forms are sent to respondents (e.g. by mail, fax, or dropped off) for respondents to self-complete and return.
- suit detailed surveys and surveys that require respondents to collate data
- costs can be much lower than other modes, e.g. no interviewer costs (except for follow-up)
- lower response rates
- reliance on postal services
Electronic / web forms: Self-administered forms that the respondent completes using a computer.
- electronic submission produces a much faster response than paper forms
- complex sequencing can be automated
- edit checks can be incorporated into the form
- technical problems with the form/system can lead to loss of data and/or respondent frustration
- high costs associate to develop the form, maintain the systems and ensure the security of data
Mixed modes: Multiple modes of data collection used for the same survey.
- exploit the advantages of the different modes selected
- allow respondents to choose the preferred mode of response
- can increase the response rate
- mode effects may be produced, whereby data is artificially different between modes, as a result of the different modes rather than any genuine difference between respondents
This refers to the effect that different modes of administration may have on the way respondents answer the survey questions (and hence the data that is collected).
For example, respondents may choose to answer sensitive topics in a manner that they see as more favourable if data is being collected by a person, than if the survey is self-administered (known as social desirability bias).
It is particularly important to consider mode effects if comparisons need to be made with the data, such as for mixed mode designs and when a repeating survey changes from one mode to another between cycles.
Data collection mode: The method that is chosen to administer the survey in.
Fit-for-purpose: The level of quality attained froma survey will meet the use for which data is collected.
Self-administered: Also referred to as self-enumeration. Forms that respondents complete themselves, without the help of an interviewer.
1. The ABS Mode Suitability Framework identifies the key issues that should be considered when selecting a particular mode. Click here to access the ABS Mode Suitability Framework.
2. The ABS Mode Suitability Model can be used to rate the suitability of particular modes (CATI, Mail, Fax, and Offline forms) against the key factors that influence the choice of mode. However, note that the framework does not yet contain web forms. Click here to access the ABS Mode Suitability Model.
3. Roberts, C. (2007). Mixing modes of data collection in surveys: A methodological review; Centre for ComparativeSocial Surveys, City University, London. Available at http://eprints.ncrm.ac.uk/418/1/MethodsReviewPaperNCRM-008.pdf