My tip is:- think clearly about whether you are being asked to evaluate or discover. And then structure your sample and method accordingly. They are journeys through different landscapes and need different tools and tactics. Just as you wouldn’t pack the same bag for a business trip as a holiday – so you need to pack your methodology differently for these two journeys!
I would estimate that about 70% of qualitative work is evaluation (testing concepts, admatics, copy or similar) while 30% or less falls into the discovery category. So I’ll tackle evaluative work first and come to discovery a bit later in the Tips series. I’m sure many of us are grateful and excited to do the evaluative stuff, but it is not in my view the natural strength of focus group methods. Why do I say that? It’s to do with the nature of the discourse. In any evaluative project the discourse must be sourced in comparisons. Is this better than that? Is this the right direction for this brand or service? Is the logo clear? Is the message on target? Is this more relevant/up-to-date?Once you create a comparative frame, it’s hard to be sure in your groups who is comparing what with what. Everyone in your group or sample brings a different frame of reference to the topic and quite often the stimuli have at least 5 or 6 versions – they are complex and pictures, symbols & language are often mixed together.
Most importantly, the people in your group are more likely to be influenced by you, the moderator and the other group members than the stimulus material. It is almost always of more importance to each person what the other people think of h/her than to be accurate about the stimulus or their opinions. It is not an exaggeration to say that for most group members the most important filter they apply to anything they might think of saying is: ‘what will the others think of me if I say that?’ Just picture yourself speaking out on an unfamiliar subject before a group of strangers!
Additionally, if the group has dived straight into examining the stimuli and there has been no practice of working with difference or disagreement, the group will regress to the norm – and so choose a safe option – usually based in a mild form of agreement. Not because it’s better for the brand – but it is better for the group. They may be driven by the need for a sense of unity, comfort in agreement, sensibleness and that they have got their job done.
If you are worried about groupthink and conformity, make sure you collect individual viewpoints before you ask the whole group. I usually do this by asking everyone to write down anything that comes to mind in the presence of the stimulus on an A5 card. I ask them to write for 10 minutes, so that I am sure to collect a flow of associations that I can look at and review after the group session. That avoids the safe, single word response like: OK!
In my experience, the question lingering in the debrief of such projects is often:
“How many people actually said that?” If this question is bubbling about, I think its nearly always a sign that quantitative methods might have worked better to resolve the ‘which is best?’ issue.
That doesn’t mean that evaluative work is a waste of time; far from it. It is essential to hear customers or potential customers working with the vocabulary and personality of the brand, and this can reassure clients and/or stimulate creative. I think though, that to get the best out of this type of group/depth, you need to include the possibility of disagreement and argument since this is the very framework you are using to assess. If it only comes at the analysis stage you will be left with having to guess! I will give you a tip on how to work with difference and argument next time.
Here is the pdf: