In this chapter, we’ll show you how you can get better at running user research sessions. We’ll cover interviewing techniques, managing tasks, and how to deal with tricky situations that can throw you off balance. As we mentioned in Chapter 6, interviewing is only one of the activities that takes place while you’re running the session. But because it’s so important, it gets a whole chapter to itself.
Research interviews are a strange, artificial social interaction. If they’re in a purpose-made research lab with cameras and recording equipment, they’re even stranger.
Fundamentally, an interview is an exchange of information between two people. But think of how many variants that includes: job interviews, police interviews, dates, quizzes, journalistic interviews… the list goes on. In each case, there are subtle differences in the rules and dynamics. Through personal experience – or from watching television – most people have an understanding of how that social situation works.
Think now of research interviews. In many cases, participants have no idea what to expect. Is it a test? Is it a chat? Will I be expected to give away intimate information? Because they don’t know what the rules are, they’ll be nervous, and they’ll look for analogies in other interview-like situations that they’re more familiar with to make sense of it. They want to help you, but they don’t know how.
This is where you can show the way. By clearly communicating the rules, process and expectations of the session, you can show them how to help you. They’ll be relieved, and you’ll get a better, more open, session.
The rules of a research session:
By stating the rules at the beginning (and reminding the participant about them as necessary later on), you can build trust, and avoid misunderstandings. Sometimes, we find it helpful to post the rules on the wall of the interview room for further emphasis.
If you want a session to run smoothly, the first thing you can do is be clear about what you need:
Consider what you wear when you’re running interviews. Use your clothing to make your participants feel at ease, and to set the tone of the session. We tend to dress informally and comfortably to reflect the way we run our sessions. Also consider who you are testing with. There may be occasions when more formal clothes are a better option, for example when researching with businesspeople or on a more serious subject.
When we think of interviews, we tend to think of a series of questions and answers between two people. In reality, the words are just the beginning – you have many more tools at your disposal.
Because this kind of concentrated listening goes beyond just the words that the participant voices, we call it ‘paying deep attention’.
When we train researchers, they’re often most nervous about how to ask questions. Actually, the core skill is simple: stop worrying about your next question, and instead pay attention to the participant. You’ll find your next question comes naturally from what they say or do.
Obviously there’s a bit more to it than that. Firstly, paying deep attention is harder than it looks, especially when you need to sustain it over time and avoid the temptation to judge. Secondly, there are better and worse ways to ask a question (which we’ll cover below). Thirdly, as moderator you’ll normally be juggling several other tasks: making sure the prototype is working, keeping an eye on the clock, ensuring you’re covering off the topic areas in your guide.
There’s no shortcut to mastering deep attention. You may find it easier if you’ve practiced mindfulness, or if you’re gifted with strong powers of concentration or empathy. The best way to develop this skill is to practice it:
If we’re doing lab testing, we always introduce ourselves outside of the testing room and explain what is going to happen during the session. We’ve found that this makes people more comfortable than taking them straight into the test room, because it puts them at ease while they’re getting used to you and the rules of the session.
In our waiting area we have a comfortable sofa, some magazines, and we offer everyone a cup of tea or coffee. The interviewer gives the participant a few minutes to settle in and then goes to say hello. We introduce ourself and thank them for taking the time to come in and help us out. We explain what we’re doing and what they can expect from the session, and give them a chance to ask any questions.
You need to explain your consent form to your participants and be ready to answer their questions. You can read about writing a consent form in Chapter 3. They have to understand what is happening and provide their consent before you can start the session with them. At this stage, you should also make them aware of any codes of ethics or professional standards that apply. For example, at cxpartners we abide by the MRS Code of Conduct.
At the end of the session, you may also want to re-confirm the participant’s consent by asking if they’re still OK for you to keep a recording.
If you’re guerrilla testing, you’ll have less time with people to introduce yourself. Also, while you’re introducing yourself you’re also trying to explain what the research is about, and persuade them to help you. That’s tricky and needs some consideration.
What you shouldn’t do:
What you should do:
This gives people context, establishes their eligibility, and starts a more natural conversation.
In this section we’re going to look at how questions work:
Here are 10 pointers to making you a better interviewer. Try to focus on one or two of these each time you run a session, and ask your colleagues for feedback on how you did.
Think of a question as having two parts:
On their own, neither is much use:
Yet when we combine them, it’s clear what we’re talking about and what we want to know: “Let’s talk about ice cream. What‘s your favourite flavor”
Asking the subject first, then the query, makes it slightly easier to process the question. But you can introduce them in either order: “What’s your favourite flavor of ice cream?”
A question like, "What’s your favourite flavor of ice cream?" introduces a new issue. With this structure, we’ve assumed that the participant has a favourite ice cream flavor, and therefore the query is relevant to them. We’ve asked a leading question. This is bad news, because the participant may feel under pressure to pretend to like ice cream, and invent a favourite flavor. That’s worse than no answer at all.
To avoid this problem, we should use a qualifying question to establish the relevance of our query:
It’s easy to ramble on about the subject and let it turn into a kind of introductory statement, as in the example below. Apart from wasting time and patience, it’s possible that the participant will forget what you were asking about!
It’s best to avoid this kind of rambling, but if you do need use an introductory statement, always restate your subject immediately before or after the query:
Closed and open questions both have their role, and it’s best to use them together.
There’s a certain kind of word or phrase that you should be particularly attentive to when a participant is speaking. We call these ‘value words’: adjectives or descriptions that appear meaningful, but are actually open to wide interpretation. Some examples: “corporate”, “cluttered”, “gritty”, “feels like the real thing”, “my kind of place”.
When you hear value words, you should do one of two things:
Value words are your doorway to real insight in the interview. Don’t waste them!
Imagine you’re a detective. The participant will be wanting to present a certain version of him or herself to you: perhaps someone who exercises every day and eats plenty of fruit and vegetables. Whether or not this is a conscious effort on their part, it’s unlikely to be the whole picture.
If you can be attentive to contradictions in the research session, you’re more likely to get under the skin of the topic, and find out what’s really going on. This might be a contradiction between what someone said and what they did: “Earlier on, you said your preferred holiday destination was Mexico, but when you went onto the website, the first page you visited was South Africa. That’s interesting isn’t it? Tell me more about that.” Or it might be an inconsistency between two different statements that they made: “Earlier on, you said that your favourite ice cream flavor was rum and raisin, but just now you said it was coconut. Do you have more than one favourite flavor, or is there another factor at play here?”
When you’ve spotted the contradiction, follow it up. It’s absolutely essential that you’re respectful in your phrasing, but participants are generally curious about their own internal processes, and intrigued by what you’ve shown them.
Sometimes it can be as interesting to note what’s not happened. Perhaps the participant had mentioned a subject they were interested in, but flicked straight past it in the material you were asking them to look at. Or perhaps you were expecting them to comment on a particular design, but they said nothing.
It’s ok to draw attention to this. Your expectations (your rolling hypotheses, in other words) need checking, so ask:“A moment ago, I was wondering if you were going to click on to the information about Mexico, but you didn’t. Looks like I was wrong! Can you help me to understand your thought process?”
Now that you’ve been introduced to the main question types, you can learn to use them in sequence, as ‘question chains’. Here’s an example:
In that example, we’ve gone from using closed questions to establish relevance and initial facts, before using an open question to really kick off the conversation. We jumped on the value word ‘unusual’, and then explored the contradiction between adventure and home comforts. And in doing so, the participant has shared something that really matters to them: it’s important to them to be able to show off about their holiday afterwards. In other words, we’ve gone from some very neutral facts, to a statement about feelings and self-perception.
Once your chain runs out of steam, you can start another one. As you get more practiced, you’ll be able to run several question chains at once, spotting the contradictions between them as you go, and reviving them when the time is right.
When you want the participant to keep talking, but you don’t want to ask a new question, you can use prompts. It’s helpful to have a repertoire to use. These can include:
The timing of your question is as important as the phrasing. If you choose the right moment, you’ll barely have to give any explanation: the participant will understand what you’re interested in and give you a long, detailed answer. Choose the wrong moment and your question will fail, or get a radically different response.
Here’s an example. Imagine you wanted to get feedback about a new website design from your participant. Your plan is to ask them: “I’d like to know what changes you’d like the designers to make to this website. What are your top three suggestions?” If you asked this question after the participant had used the site for just 60 seconds, then you’d be likely to get comments that focused on visual design: “the colours are too garish”, “I don’t like the photography”, etc. If, on the other hand, you asked them at the end of the session, they might concentrate on difficulties with completing tasks, or missing functionality. Be flexible about the order in which you tackle subjects, and don’t feel obliged to follow the sequence in the discussion guide.
Most people who are learning to interview talk too much. They don’t give the participant enough space to talk or think, and as a result they miss valuable insights. Get used to silence – become comfortable with it. Some participants (particularly more introverted personalities) appreciate a moment to gather their thoughts before or after answering a question. What’s more, silence is a great prompt.
As you get more accustomed to interviewing, you’ll create your own methods and favourite questions. You’ll develop your own style. And because you’re comfortable with the technical side of interviewing, you’ll be able to pay deep attention at the same time as juggling your various other tasks. Once you’re up and running, interviewing participants can be fascinating, and great fun. Enjoy it!
The vast majority of problems that occur with participants during interviews relate to your setup. If they’ve been correctly recruited, you’ve piloted the session and prepared your material, warmed them up, and been clear about your expectations, you’ll find that sessions run smoothly most of the time.
Occasionally, though, you’ll find yourself in a difficult situation. See it as a challenge: there are plenty of techniques you can use to get yourself out of trouble. If you’re finding a participant uncooperative, select from the escalating scale of tactics below to help turn it around.
The main point to remember is: never panic. You’re in charge. Easier said than done, perhaps, but don’t forget, it’s your interview and you’re the one setting the rules. Even if you can’t control how the participant acts, you do get to choose how the session is run, and whether to terminate it. Just having that knowledge in mind can be enough to avert a drama.