VACUUM CLEANER IN-HOME TEST – METHODOLOGY AND OUTCOMES
Design and approach
The overall purpose was to undertake a test of two vacuum cleaners, the Dyson DC40 Animal and the Shark NV680UKT Trupet, to establish which is preferred by consumers. This paper describes how the exercise was designed and conducted in order to provide a fair and objective comparative test of the two cleaners among a representative sample of British householders in their own homes.
A recruitment and recall exercise was recommended whereby a sample of Britons was recruited and invited to test the two cleaners over a 4-week period and speak again with an interviewer in order to provide feedback.
Interviewing was conducted by interviewers from GfK.
A pilot survey was conducted to ensure the methodology was robust.
The pilot survey comprised 20 recruits. Five interviewers recruited four respondents each in Kilmarnock, Nottingham, Runcorn, Deeside North Wales, and Oxfordshire.
Respondents were first contacted by traditional ‘door-knocking’ and recruitment took place in respondent’s homes. The key recruitment question was as follows:
“Do you personally use a vacuum cleaner to clean in your home at least once a week?”
Only people who answered ‘yes’ to this question continued the recruitment interview.
In addition, quotas were set to ensure that the sample provided broad representation in terms of age and social grade and a balance of men and women. As part of the recruitment process, consent was obtained for respondent details to be forwarded so that the cleaners could be delivered. A diary was left with respondents so they could record their experiences with the cleaners. They were informed that one purpose of the exercise was to “provide information and material for a TV advertisement”.
All respondents were offered a £50 cheque as an incentive to take part in the exercise.
Respondents were given periods over which they should use each of the cleaners; half the respondents were asked to use the Dyson for weeks 1 and 3 and the Shark for weeks 2 and 4 and half were asked to use the Shark for weeks 1 and 3 and the Dyson for weeks 2 and 4.
All cleaners were sent by Shark to Philip Malivoire who arranged the dispatches. This process was followed to avoid dispatching cleaners from Shark offices, which might have revealed the identity of the company conducting the research.
After about two weeks of the test interviewers re-contacted the respondents, by telephone, to make an appointment for the recall visit. For that contact, interviewers were instructed simply to check that everything was OK with the test and to fix a time/date for the recall. They were told not to discuss anything else about the test with the respondents.
The recall stage was conducted face-to-face in the respondent’s home and a single question was asked: “Overall, which one of the vacuums did you prefer?”
The recruitment interviews were conducted between 30th June – 4th July. All cleaners were due for delivery on 12th July although in one case in which this delivery was delayed by 2 days. The recall interviews were conducted between 7th – 9th August.
19 out of the 20 recruits successfully completed the full test.
Following the pilot survey, two changes were made to the methodology for the main stage. First, two changes were made to the recruitment questionnaire. First, during the pilot process, there was a request that the recruitment process establish that respondents be British. A ‘nationality’ question was added for this purpose. Second, a minor change was made to the consent obtained regarding the “TV advertisement” element of the exercise.
The second change was the implementation of further quotas. As well as questions relating to gender, age and social grade, the recruitment questionnaire asked questions about the presence of children and pets in the household as well as the ownership of different brands of vacuum cleaner in the household. For the main stage, we also added quotas to reflect the known proportions of households with children and pets in the sample.
The main stage comprised 88 recruits. This number was selected in order to obtain 100 interviews across the 2 stages, allowing for some issues/incomplete tests. 22 interviewers each recruited 4 respondents in: Norwich, Havant, Thame, Southampton, Swindon, Watton near Swaffham, Hitchin, Brinsley near Nottingham, Beeston, Northampton, Camelford, Shrewsbury, Newcastle-under-Lyme, Newtown, Kilmarnock, North Shields, Houghton Le Spring, Runcorn, Morecombe, Scarborough and Huddersfield.
Recruitment interviews for the main stage were conducted between 12th – 17th August and cleaners were delivered between 18th – 23rd August. The dates of the recall interviews were ‘staggered’ to ensure that all respondents had a full 4-week trial. Recall interviews were conducted between 15th – 27th September.
One respondent reported a problem with his Shark cleaner after the first week of use. In order to reflect a ‘real-life’ response to such an occurrence a replacement Shark was sent to him for the second week of use.
Summary of sample and responses
The combined figures for both stages are set out below:
8 participants did not complete the test correctly. Therefore, 100 respondents completed the full test and answered the “which vacuum do you prefer” question. Overall, just under 80% preferred the Shark cleaner.