What users tell you in interviews vs. what they do
The inspiration for the article was a question from Laura Fiona Holder in a Slack group for UX designers, of which we are both members. She raised the question around qualitative User interviews that got me thinking.
I guess we all know the most important points about how to conduct a proper User Interview. I mean there are plenty of websites/blogs or articles with key tips one should consider; more listening than talking, focusing on facts, not opinions, not talking about solutions too early, just to name a few. But there is one thing that interviewers are sometimes not aware of. Probably because it’s often not obvious at all during the interview.
We all sometimes tell stories or say things that we don’t exactly mean. We state things that we wouldn’t agree on. And often we all give answers that are not well thought through and that we would not give in other circumstances. Probably because we want to avoid uncomfortable, more detailed questions. We could also want to make a conversation easier, understandable for both, or just don’t feel comfortable with how we could be seen by the person we are talking to.
I would say that is normal, but in one particular situation, it makes things harder. You guessed right: In the context of User Interviews.
How valuable are research results if they are based on the User’s answers that do not represent what the User or customer would do? For example, think about pricing. You ask the interviewed: “Here is solution ABC, would you buy it for a price of XYZ?”. In an ideal situation, their answer would reflect what their future behaviour would be. But imagine it is the end of November and the User just had to pay their car insurance or buy Christmas presents. Then potentially the answer may not reflect their future behaviour against this situation or with other circumstances. If you are lucky, they could spend even more money than they told you in the interview. But if you aren’t, people just won’t buy at the price you offer it, even though you thought you did the pricing based on research results. The same applies to features or services research too, of course. Worst case the result would be the same as if you hadn’t done any research at all. You would just be guessing why people aren’t using or buying.
But can this situation be avoided?
In my opinion, some major factors contribute to minimizing the risk of getting falsified answers in this scenario. Here are some pointers that I find helpful. I’m open for extras or comments of course:
1. Making the interviewed feel comfortable
I think one of the most common situations when a person is pretending, is when they do not feel they are accepted as they are. Especially more introverted people who are not super confident and may need some time to warm up. And as soon as they do, the answers will most likely represent their future actions better.
2. Make sure you do enough qualitative interviews
This point doesn’t erase falsified answers, but it somehow evens them out with a greater number of participants. The more people you interview, the less weight a single “false” answer carries or skews the results. It certainly increases the time and effort required, but sometimes this is worthwhile.
3. Try to keep the situation objective
This one is well known but definitely earns its place here. A qualitative interview must be objective. It can sometimes be challenging to switch from a very personal warm-up (to make the interviewee feel comfortable) to a rather fact-based professional interview situation. But for the best results, objectivity is key.
4. Take your time
Sometimes a lack of time can result in a person trying to shorten the interview or give the “right” or more fitting answer. But an interview doesn’t have a right or wrong response as long as they reflect the User’s behaviour. Both sides should feel like they have the time to explain their answers properly. Trying to squeeze in some additional interviews, into an already packed day, is probably not the best way to go.
5. Make the interviewee “feel” the scenario (help them recall or predict accurately)
A helpful approach to getting a relatively accurate memory is by identifying the pain points first and then digging deeper into that specific memory. This way the memory tends to be more vivid. The tricky thing here is to find the right balance between a rather emotional memory (pain point) and a fact-based answer for the specific solution later on.
6. Leave binary answers to quantitative interviews/get the how’s and why’s
Doing a qualitative interview to answer binary yes/no questions may not be the perfect setup. Avoid at all costs! Apart from that try to re-evaluate what kind of interview you are conducting and which points or questions you aim to clarify with your questions.
7. Leave out your personal value
On the one hand, this is the most obvious one, but also the hardest to do. We all know we are not our clients and we are interviewing people to understand what they think.
BUT: As always in psychology, even though we know it, we are not safe from the effect of overvaluing opinions that confirm our own view in comparison to ones that don’t. I guess this point just really depends on the experience and empathy of the interviewer.
These are just some of the many things I like to consider when doing proper User interviews. In the end, it depends on a lot of differences and variables. I’d love to hear what you think about this particular issue? Have you already experienced it yourself? How do you deal with it? Let me know your thoughts by contacting me on LinkedIn.