Is This Right? Responding to Uncertain Participants
Abstract
This chapter covers situations where the participant is uncertain about his behavior or the feedback he is providing, and looks for reassurance from you. These include situations such as the participant directly asking you for your opinion, the participant being nervous or uncomfortable, and the participant blaming himself for not doing something correctly or being helpful enough. For each situation, you’ll learn what to do, what to say, what not to do or say, and how you might be able to avoid this in the future.
Keywords
user research nervous participant; usability participant nervous; usability blame; user research participant comfort
8.1 Participant looks for affirmation
8.2 Participant asks for your opinion
8.3 Participant looks or sounds uncomfortable and/or nervous
8.4 Participant is self-blaming
8.5 Participant asks, “Did other people have trouble with this?”
8.6 Participant is unwilling or unsure to put in effort or unsure how to articulate himself
The artificial situation created by user research may cause participants to be nervous or worried about not being a “good” participant. Even if you tell a participant that he’s not being tested, a usability lab setting or the presence of recording equipment and observers may be intimidating. You need a set of responses and comforting steps that can help reassure the participant and make him more comfortable with the session.
8.1 Participant looks for affirmation
What to do
This situation is similar to one where participants blame themselves and apologize (see section 8.4), except in this case they’re more explicitly looking for feedback from you to direct them.
Assure the participant that he is providing the kind of feedback you need (which is different than assuring him he is doing anything “right”), and reiterate that anything he does or says is helpful for the research team.
Check your body language and see if anything you’ve done during the session may have caused the participant to think that he is doing something wrong. For example, if you’re taking notes, make sure that you’re taking notes both when the participant is being successful and when he isn’t, instead of just when the participant encounters an issue.
Rather than letting the participant dwell on his desire for affirmation, follow your reassurance with an immediate redirect to another follow-up question or task related to the research goals. For example, you may say, “This is just the kind of feedback we want to hear. Now, earlier you mentioned.…”
What to say
“This is just the kind of feedback we want to hear.”
“All of this has been very helpful to hear and see. Thank you for your continual feedback.”
“Don’t worry about how you’re doing—we’re not looking at it that way. We’re just here to see/hear how you’d approach it, and get your feedback on the process.”
What not to do or say
Avoid agreeing (or disagreeing) with a participant about his feedback. For example, don’t say, “Yeah, this is obviously a big problem we never thought about.”
In a usability study, avoid letting the participant know whether he is doing a task correctly or not.
Avoid using leading phrases like “You’ve completed this task” or “This task is complete” because it implies correct and incorrect ways of approaching tasks. Similarly, avoid leading questions that imply or suggest an expected answer. For example, “So you do this every day, right?”
How to avoid
Keep your body language consistent throughout the session so participants, who may be extra sensitive due to being observed, don’t feel like you’re being affected by their performance. For example, avoid leaning forward when the participant is taking an unexpected path during a usability study if you’ve been leaning back throughout the rest of the session. Similarly, avoid frowning or shaking your head if you disagree with feedback provided during an interview.
Encourage the participant by periodically telling him that his feedback is helpful. If you take this approach, be sure to not just provide this encouragement after the participant provides certain types of feedback (for more on this technique, see Chapter 3 sidebar “Should You Tell a Participant That Her Feedback Is Helpful?”).
Watch Video 1 to see an example of a moderator interacting with a self-blaming participant during a usability study. The participant is very nervous about the session and is looking for affirmation.
Visit our website (http://www.modsurvivalguide.org/videos) or use your QR reader to scan this code. 
8.2 Participant asks for your opinion
What to do
The participant may be using these questions to engage you in a conversation like he would if you were talking in a more casual setting. A conversation may feel more natural to him than being asked to just respond to your questions.
Redirect the question by reminding the participant that you’re interested in what he thinks. Explain that you want to stay neutral.
If you find that the participant is being overly friendly or conversational with you, and his friendliness is leading to these types of questions, adapt your moderating style (as we discussed in sections 1.5 and 1.6) away from a more informal disposition. For example, if your style was similar to the Friendly Face, you could shift to the more formal Down to Business. In conjunction with this change, you may need to disengage slightly from the participant by subtly changing your tone and body language/positioning as discussed in section 3.6.
How to avoid
Use language at the start of the session to let the participant know that you’re trying to be a neutral observer, which means that you may not answer his questions directly. Even though this won’t prevent the question from ever coming up, it’s important to set the expectation at the beginning so you can refer back to what you said if the participant keeps pressing you for an answer.
8.3 Participant looks or sounds uncomfortable and/or nervous
What to do
Usability participants often are nervous and uncomfortable at the start of the session. They may be surprised that it’s a one-on-one situation rather than a focus group, or feel nervous about being watched. And, of course, as much as we tell participants they’re not being tested, sometimes they still feel like they are. Similarly, contextual inquiries are all about being watched, so a participant may feel awkward or nervous about your presence.
If a participant seems nervous at the beginning of the session, try to ease his discomfort so that he doesn’t get even more nervous. Spend a few minutes asking how he is, if he had any difficulty getting to your location (if applicable), and/or other niceties like the weather or (noncontroversial) news of the day. Ask if he has participated in any kind of research before to get a sense of any prior experience. Then, proceed to your normal participant briefing.
If the participant seems nervous about being observed, explain at a high level who the observers are and why they’re watching. You can also reassure him that while it may feel strange at first to be watched, you’ll be the only person interacting with him and after a while he’ll probably forget about the observers.
Remember to check if there is anything you’re doing to contribute to the situation, as we discussed in section 2.3. Check your body language and see if anything you’ve done during the session may be contributing to the participant’s nervousness. For example, ensure that you’re not sitting too close to the participant, or doing anything uncomfortable like staring intently at him.
Tone of voice can go a long way toward calming a participant’s nerves. Be genuinely interested in what he says, which will help keep your tone kind and gentle, while still neutral and nonpatronizing. This is a tricky balance to achieve but an effective one.
Provide occasional reassurance that the participant’s feedback is helpful. For more on how to provide this reassurance, see the Chapter 3 sidebar “Should You Tell a Participant That Her Feedback Is Helpful?”
If a participant seems extremely nervous or uncomfortable, and is not responding to the previous suggestions, take a short break. After the participant has had a few minutes to himself, you can then ask him, in private (e.g., outside of the research room), if he is okay to continue with the session.
What to say
Before the session:
“Please know that we’re not at all testing you or your abilities; you’re here to help us <evaluate the product/learn more about your process>.”
“Anything you do or say is useful for us to know; there are no wrong answers.”
If disclosing observers:
“Any observers who may be watching are all on the team for this project, and want a chance to see feedback firsthand from their <current/potential> users—both positive and negative.”
“Even though there may be observers, you’ll only be interacting with me. After a while, you may even forget that anyone else is watching.”
During the session:
What not to do or say
If your moderating style is similar to the Down to Business style described in section 1.5, you may find that such a hands-off approach makes a participant’s natural nervousness even worse. Instead, try adapting your style to be a bit more reassuring and gentle with your questions and instructions.
Don’t ignore a participant’s nervousness, as this may make him even more nervous or uncomfortable.
How to avoid
Nervousness can’t usually be avoided but it can be tempered by some of the techniques discussed—adapting to a more gentle style, assuring a participant upfront that he is not being tested, and reassuring him throughout the session that what he is doing is helpful. Also, follow the tips described in section 2.3 to ensure that you don’t contribute to a participant’s nervousness.
Watch Video 1 to see an example of a moderator interacting with a self-blaming participant during a usability study. The participant is very nervous about the session and is looking for affirmation.
Visit our website (http://www.modsurvivalguide.org/videos) or use your QR reader to scan this code. 
See also
6.1 Participant is reluctant to say anything negative
8.4 Participant is self-blaming
8.6 Participant is unwilling or unsure
Chapter 3 sidebar: “Should You Tell a Participant That Her Feedback Is Helpful?”
8.4 Participant is self-blaming
What to do
Refer back to what you told the participant during the briefing at the beginning of the session—that he is not being tested, and that his feedback is appreciated.
If the participant doesn’t think he has the experience level necessary or is otherwise unqualified, reassure him that you’re interested in hearing from users with all types of experience.
Rather than letting the participant dwell on his self-blaming, follow your reassurance with an immediate redirect to another follow-up question or task related to the research goals. For example, you may say, “Remember what I said earlier—if we don’t see what areas work well and not so well, we wouldn’t learn anything. Tell me more about what you were expecting.…”
What to say
“Please don’t feel badly. Remember that we’re evaluating the product, not you.”
“If we didn’t see what areas work well or not so well for you, we wouldn’t learn anything.”
“You’re here to partner with us to evaluate the design, so all of this is really helpful for us to see and hear.”
“There are no wrongs here; you’re here to help us.”
“It’s important for us to bring in users with all levels and types of experience because we want this product to be easy to use for everyone.”
What not to do or say
Do not dismiss the participant’s comments by saying something like, “It’s okay,” or “No problem,” as it may be perceived as confirming that he is not doing a good job.
Try to avoid phrases like “Everyone else hasn’t been able to figure that out,” or “You’re not the only person who’s had difficulty with this” because (a) you’re acknowledging that he is doing something wrong and (b) by stating that others have had problems, you’re empathizing too much and not remaining unbiased.
How to avoid
Making a participant feel as comfortable as possible in the beginning of the session will help avoid this. As part of your participant briefing, let the participant know that anything he says or does will be useful to the team, and that any problems he experiences are a reflection on the product, not on him; those problems are an opportunity for improvement. Say this as sincerely and empathetically as possible; we’ve found that just racing through this section of your study plan often results in information going in one ear and out the other.
Watch Video 1 to see an example of a moderator interacting with a self-blaming participant during a usability study. The participant is very nervous about the session and is looking for affirmation.
Visit our website (http://www.modsurvivalguide.org/videos) or use your QR reader to scan this code. 
8.5 Participant asks, “Did other people have trouble with this?”
What to do
Participants often want to know if they’re the only ones who can’t understand or do something. If a participant knows that others have had trouble too, he thinks that he’ll feel better about any problems he’s having. Because the question is asked so directly, you may feel on-the-spot to answer or address it.
Avoid answering the question, even if you want to say “yes” to make the participant feel better.
Try to turn the question into an affirmation that his individual approach and feedback is what really matters, and then redirect the conversation to close the subject.
If it feels too disingenuous and you feel the need to give some kind of answer, tell the participant that you can’t remember or are not sure, and quickly redirect to another question or task.
What not to do or say
Do not respond “yes” or “no.” Saying “yes” to a participant has an inherent bias because he may change his approach and reaction to the product given the new piece of information. Saying “yes” also may come across as agreeing with him, which breaks your neutrality. Saying “no” may make him feel inadequate or insecure about his abilities.
How to avoid
There is no good way to prevent this question from coming up. However, you can reduce the odds by telling the participant at the beginning of the session that you’re trying to be a neutral observer, which means that you may not answer his questions. Even though this won’t prevent the question from ever coming up, it’s important to set the expectation at the beginning so you can refer back to what you said if the participant keeps pressing you for an answer.
8.6 Participant is unwilling or unsure
What to do
While sometimes participants really just don’t know, if they give one- or two-word responses repeatedly it usually indicates a problem. Two common causes are that participants are either unwilling to put in effort or are unsure about how to articulate their thoughts. The former tend to be either people who were forced to participate (e.g., by managers) or are just participating for the compensation. The latter can be anyone, but you can sometimes tell that they’re struggling to find words. The signs of a nervous/uncomfortable participant described in section 2.4 might also accompany these responses.
Try to address this situation at the very beginning of the session or, if it appears later in the session, as soon as you notice it. Thank the participant for joining the session and tell him how much his feedback will help.
Keep encouraging the participant throughout the session. Without interrupting his workflow or train of thought, continually ask for the participant’s feedback so he knows that his opinion matters.
If the participant seems to have trouble articulating his thoughts, be sure not to rush him. Sometimes a little bit of extra time is all that is needed. If the time doesn’t seem to help, you can try asking a question a different way or try to narrow the range of possible answers. For example, instead of, “How was your experience using this vacuum?” ask “Was your experience with this vacuum the same, better, or worse than the experience with other vacuums you’ve used?”
If the participant’s reticence seems to stem from nervousness, use some of the techniques outlined in section 8.3 to alleviate his nerves.
If your session is being held remotely, the participant may be distracted and doing other things, which may make him seem unwilling to put in any effort. See section 11.1 for more discussion on this situation.
If despite your best efforts, the participant still seems distracted or frazzled and unwilling to put in a good effort, don’t lose hope. Instead of being annoyed at him, change your perspective and see the situation as a challenge. Try to get him engaged in the session by making your task/question more personal. One of the most satisfying things for a moderator is watching a participant go from moody or annoyed to calm and engaged during the course of a session.
What to say
Before the session:
“Thanks so much for participating. I know that we’re taking time out of your busy day, but I want you to know that your feedback is extremely important to us. You’re helping us make the product better.”
“Hopefully this session will be a fun break from your typical work.”
During the session:
How to avoid
Participants should never be forced to participate. Sometimes we leave recruiting to others who have access to the right people, like call center managers or representatives at a target organization. But they don’t always know the ins and outs of user research, and don’t know the trickle-down effect that their decisions can have on the quality of the research. If they’re not positioning the research as something valuable, or worse, are coercing participants to take part, you’ll end up with participants who may not be interested in putting forth the effort.
We suggest working closely with those recruiting for you to make sure they’re finding people to participate voluntarily and that the participants know ahead of time what to expect. Refer to section 15.1 for more discussion.
See also
6.3 Participant is not thinking aloud
6.5 Participant ignores or pretends to understand your question
8.3 Participant looks or sounds uncomfortable and/or nervous
SURVIVAL STORY: “SHE LOOKED AGITATED”
Carol Peterson
My colleagues and I were doing onsite testing of Windows-based search and retrieval software in the early 1990s. Our users were librarians, not all of whom had experience with Windows since a lot of catalogs were still on text-based mainframe systems. We had told our recruiter we wanted participants with at least some Windows experience.
Our participant arrived, and I welcomed her and explained she would be trying to do some search tasks on the computer we had brought. I told her she could start the first task so she turned to the computer … and just sat there, not saying or doing anything. After a bit, I asked her what she was thinking. She didn’t answer me and she looked agitated. After another silence that seemed interminable, she suddenly got up, said “excuse me,” and quickly left the room.
My colleagues (who were acting as observers) and I looked at each other in confusion. Several minutes passed, during which we wondered what had happened and if she was going to come back—we had no idea where she had gone or why.
Finally, after about 10 minutes she did return, still looking nervous. She told me she had been very anxious about the test to begin with, and when she saw the computer screen, she didn’t know what to do since she had never used a mouse—she had needed to leave to compose herself.
Of course I immediately reassured her, apologized for causing her angst, etc., while inside my head I was frantically thinking of ways I could salvage something from the study. Fortunately, at that time Windows was new enough it included mini-tutorials on how to use a mouse and work with the desktop. And even more fortunate, I actually remembered this. I asked her if she wanted to try one of the tutorials to learn the mouse, and she was willing. After completing the tutorial, she had calmed down and felt competent enough to try some of the tasks.
We didn’t get a lot feedback on the software, but I learned some valuable lessons:
Ensure your recruiter understands which criteria are “make or break”—they must understand not only what will qualify a participant, but what will disqualify one.
Get to know the participant. When the participant first arrives, before I talk about the tasks I want them to perform, I ask them to briefly describe their background and their current job. Talking about a known topic helps relax the participant, plus it gives me a chance to judge whether there are any potential gaps in their knowledge that will affect the test.
Stay flexible during the test. If I hadn’t been able to adapt, not only would we have missed a chance to get data, but we would have left the participant feeling awful, which in my mind is even worse.