This chapter covers situations where you encounter something that slows or halts your session’s momentum. These include situations such as participants who start going on tangents, participants who are very slow or more thorough than you expect, and participants who struggle excessively with tasks. For each situation, you’ll learn what to do, what to say, what not to do or say, and how you might be able to avoid this in the future.
usability participant tangents; usability slow participant; user research vague answers; user research task struggle
10.1 Participant starts going on a tangent
10.2 Participant consistently focuses on irrelevant details
10.3 Participant does something very unexpected
10.4 Participant is slow or thorough
10.5 Participant gives vague responses to questions
10.6 Participant is difficult to hear or understand
Momentum blockers slow the progress of the session, sometimes all the way to a full stop. Whether you’re having difficulty understanding the participant, or you’re running out of time to accomplish your tasks, this chapter will help you figure out the best way to move forward and bring the session back on track.
The participant’s tangent may be irrelevant to your research for a variety of reasons. Perhaps the functionality or processes he’s using or discussing are out of scope, you already have enough data about the subject matter, or it has nothing to do with what you’re researching. However, the participant doesn’t know this, and the information he’s providing may be important to him.
Ask the participant to tell you in his own words what he’s trying to do. His answer will help you figure out if he misunderstood you or is pursuing the tangent for other reasons. If he did misunderstand you, repeat or reword your question, or ask him to re-read the task.
If you have time in the session, consider letting the participant pursue his tangent. It helps him feel heard, especially if it’s on a topic he’s been eager to talk about. You may be surprised at the relevant feedback that can surface from letting him do this.
Set a time limit for how long you’ll let the participant stay on a tangent before steering him back. The time limit depends on the length of your session, what else you need to cover, and whether it seems like the tangent may connect back to something that’s relevant to your research. For example, let’s say you’re running a one-hour interview about how users think about retirement. If, just a few questions in to the session, the participant goes on a tangent about mobile applications he’s had negative experiences with, you may want to politely rein him in after just a couple of minutes. However, if he goes on the same tangent 45 minutes in to the session when you only have one or two questions left, you may let him go closer to five minutes. If you see a way to connect the tangent back to your study goals—say if the goal of your research is to understand how users approach their smartphones compared to their tablets—you may let him go even longer, and maybe probe for more specifics about why those applications were so frustrating.
If you’re running a usability study:
Keep in mind that if you keep steering a participant back from his tangents to a specific area (the area that is the goal of your research), you’re essentially telling him that the other ways are “wrong.” This both clues him in on the “right” path, and also can make him feel more like he is being tested, and is failing. Using an apologetic tone and acknowledging what he was trying to do can help negate this. For example, “It’s helpful to know that you would want to look in this area, and what you’d be trying to find. I would like to get your feedback on this other area as well….”
If the participant seems genuinely interested in exploring part of the product that’s not part of your study plan, you can let him know that if time permits, you’ll let him return to that area at the end of the session.
At the beginning of the session:
“We have a lot to get through, so I may at times gently push you along so that we can make sure to cover everything.”
“If we wind up spending too much time on any one topic, please don’t be offended if I ask us to move on, just to keep things on track.”
During the session:
“This is all very helpful to see and hear. Just for the sake of time, I’m going to ask you to go <to the next task/go back to….>”
“I’m interested in hearing more about this. I do want to make sure we cover everything we have planned. So if there’s time, let’s come back to this at the end of the session.”
If you need to interrupt and redirect:
Do not zone out while a participant is talking about topics you don’t care about. We’ve seen moderators stare into the distance, scratch their heads, look at their watches, and go on their mobile devices or computers. This behavior is disrespectful to participants. You should give the impression of your undivided attention at all times, no matter how irrelevant or tangential you consider their feedback. If you’re a bad actor and need something to help you focus, try taking notes on what the participant is saying to help him feel heard. However, keep your ears peeled because you never know when the participant will move into something that is relevant to your research—you don’t want to miss it!
Verify that the feedback provided by the participant really is irrelevant. The behaviors that participants keep coming back to during a session may represent their natural behavior and can be a valuable set of data for a research project. If there is a way to get value out of his behavior or answers, you can let him continue as long as time and your research agenda permits.
Ask the participant to tell you in his own words what he is trying to do. His answer will help you figure out if he misunderstood you or if something else is at play. If he did misunderstand you, repeat or reword your question, or ask him to re-read the task.
If you find yourself asking the participant to re-read the task or are repeating questions over and over again, acknowledge what he is trying to do and explain what you want to focus on instead. For example, if the participant keeps trying to use a search engine instead of the website you’re evaluating, let him know that you understand that he would typically use a search engine to attempt these tasks, but you need him to stay on the website so you can find ways to make that experience better.
If the participant seems to have an agenda regarding what he wants to provide feedback on, consider letting him vent for a few minutes and then redirecting him back to the task/question as described in section 11.7.
“You’ve mentioned that <feedback on irrelevant aspect of design>. What do you think about <relevant aspect of design>?”
“I’ve noticed that you have a lot of feedback on <irrelevant topic>. Due to our time constraints, we unfortunately can’t focus on that in today’s session. But, do you want to take a minute or two now to tell me your thoughts before we move to the other topics?”
In a usability study:
At the beginning of the session, set expectations about what the participant should try to focus on. However, avoid going into so much detail about the research goals that you bias him.
Warn the participant that, for the sake of time, you may sometimes move him along to another topic just to make sure you cover everything you need.
If your stakeholders are responsible for finding participants, remind them to avoid filling the schedule with their most vocal customers or ones who frequently volunteer their feedback. These types of participants often participate with a preset agenda and expect to talk about what they have on their mind. See section 15.1 for more on how to avoid this.
Let the participant continue down his path for a few minutes. If you’re running a usability study, remind the participant to think aloud. With a little more information, you should be able to tell whether the participant misunderstood you or is instead revealing something interesting about your users and their potential behavior. Using the example of asking the participant about his mobile application, he may have assumed that “mobile” meant the systems he interacts with in his car rather than on his smartphone.
If you’re still not sure why the participant is doing this unexpected thing, repeat the question or task to verify that he didn’t mishear you, but be careful with your tone so it doesn’t sound like you’re reprimanding him (see examples in Appendix A). You can also try rephrasing or adjusting the question or task slightly, and see if he does the same thing. For example, if you asked the participant to change his mailing address on an e-commerce website and he goes someplace that doesn’t seem to make sense, ask where he would go to change his billing address.
Similarly, ask the participant to tell you in his own words what question he is trying to answer, what he is trying to do, or what he is looking for.
Verify that you’re not using industry jargon or being overly verbose in your questions and tasks, and reword if necessary.
If the participant is watching your lips closely or tilting his head to be closer to yours, he may be having trouble hearing you. Move your chair a bit closer to his, speak a little louder, and speak in short, direct sentences.
If you’re conducting a usability study and think the participant did something unexpected because he missed the correct path, and you need to get him back on the correct path due to dependencies in future tasks, bring him back to where the divergence happened and ask a general question about the purpose of that screen or the options available. Then, if necessary, you can direct him to where you need him to go using the technique described in the Chapter 3 sidebar “The Diversionary Assist.”
For any method:
“Please tell me what you’re thinking.”
“I just want to make sure that I gave you the right <task/question>, so let me go over it again. Sorry if I’ve caused any confusion.”
“Let me know if you’re having trouble hearing me—sometimes I don’t speak loudly enough.”
In a usability study:
Before your first session, hold a pilot session with a representative participant so you can get initial feedback on any tasks or questions that might be confusing.
If your tasks are complex, consider printing out the tasks for participants to reference. This reduces potential misunderstandings and frees their working memory to focus on each task.
To see if the participant is just a naturally slow or thorough person, ask him whether what he’s doing is similar to what he’d do on his own. For example, “If you were doing this on your own, would you read through these Terms and Conditions like you are now, or spend more time on them, less time on them …?”
If the participant is being slow because that seems to be his natural approach, accept that there’s not much you can do to change that. Rushing him might confuse him or provoke anxiety. Instead, go into prioritization mode. Think across all your upcoming questions or tasks and consider what can be dropped and skimmed over.
If the participant is being slow because he feels like you want him to be thorough, reassure him that you want him to behave as he would be if he was doing this on his own without you watching. This cue should help speed him up. If it doesn’t, use the same prioritization technique mentioned in the previous point to direct him to the next task/question with the highest priority.
If the participant seems to feel bad that he didn’t get through all your tasks or questions, assure him that you hadn’t planned to get through everything in the allotted session time.
If you anticipate the participant being slow upfront, give background instructions:
“We have a lot of <questions/tasks>, but we probably won’t go through all of them. You may see me skipping around to make sure I make the most out of our time.”
If you want to see whether the participant is behaving naturally or is being slow and thorough for your sake:
“Please go at whatever pace you would if you were doing this on your own and I wasn’t sitting here with you. Don’t feel like you have to rush or be more thorough because I’m here.”
“You seem to be spending some time <doing this task>. Would you be doing that if you were on your own at <place/home/work>?”
If the participant needs some pushing along midsession:
“This is all very helpful to see and hear. Just for the sake of time, I’m going to ask you to <go to the next task/go back to.…>”
“I’m interested in hearing more about this. I do want to make sure we cover everything we have planned, so if there’s time, let’s come back to this at the end of the session.”
If the participant feels badly about not getting through everything:
Don’t rush participants through tasks or questions for the sake of getting everything done. Be respectful of the participant’s speed and level of detail, and adjust your questions/tasks accordingly.
Don’t panic. If you’re concerned about time, try not to let it show. This, of course, comes with practice but it’s good to be mindful of it.
There is no way to avoid participants who are slow and deliberate. It’s just a matter of being prepared! The best way to be prepared is to identify which questions/tasks are the highest priority as you put together your study plan. This will help you adapt the session if necessary as discussed in section 15.2.
Try clarifying or rewording your questions in case the participant didn’t understand what you’re asking. If necessary, try to back up to broader, easier questions and then, once you’ve built up that foundation, lead into the harder questions. If he seems frazzled or confused, take responsibility for “not being clear” about your question.
Ask “Why?” to help prompt the participant to dig deeper into his response. For example, if you ask a participant about his experience using a remote control and he says it was okay, you can follow up with, “Tell me more—why do you say it was ‘okay’?” You can ask “Why?” multiple times until you feel like you’ve reached the level of detail needed.
If the participant is being vague because he seems to be unfamiliar with the subject matter, move on to the next task/question that doesn’t cover the topic. If familiarity with that topic is a deal-breaker, see section 4.1.
If you sense that the participant is shy and reluctant to share feedback, try to adapt your moderating style as discussed in section 1.5 to a more informal and approachable one such as Friendly Face or Inquisitive Mind.
If the participant just doesn’t seem interested in the topic or seems eager to get through the session as quickly as possible, see section 8.6 for ideas on how to inspire his interest.
To clarify:
To build engagement:
Craft your questions to provoke a thoughtful, detailed response. A poorly worded question or a question that can be answered with a simple “yes/no” may generate a vague and uninteresting answer. For example, a participant may give a general response to “What do you think about your local bus service?” but will be more specific when asked “Describe the last time you used your local bus service.”
Make sure the session is recorded so you can go back and review the participant’s feedback more closely.
If you’re running a usability study, ask the participant to use his mouse (or finger if the product does not have a screen) to indicate the areas that he’s providing feedback on. This can also provide vital context that may help you better understand his feedback.
If you’re having difficulty hearing the participant:
Ask the participant to speak up. This situation is a good place to use a pretext of the recording equipment having difficulty capturing voices if the participant is too quiet or a poor phone connection that keeps cutting out.
If the participant is remote, ask him to move closer to his phone or microphone. Also check the volume on your phone’s speakers or your screen-sharing software to ensure that it’s turned up as high as possible.
If you’re having difficulty understanding the participant:
If the session is in person, look at the participant’s face and maintain eye contact while he’s talking, even if that means you’re unable to take detailed notes. Sometimes maintaining eye contact and being able to see a participant’s facial expressions can help you better understand what he is saying.
If the participant is remote, ask the participant to repeat what he said, using difficulties with the phone connection as a pretext. However, this tip may not be practical if the participant continues to be difficult to understand. In those situations, you can use this tip occasionally, but accept that you’ll need to go back to the recordings later.
Ask the participant to read tasks aloud. This may help you “calibrate” to the participant’s way of speaking as he’ll be reading something that you know.
“Can I have you repeat that just a little bit louder? I want to make sure that our recording equipment is able to pick you up.”
“Would you mind repeating that for me, a little bit slower so I can write it down?”
“Could you use the mouse to show me on the screen what area you’re talking about?”
If the participant is remote:
Ask whoever is handling the recruit to screen out potential participants who are difficult to understand. However, if you’re recruiting internally or interviewing very targeted users, you may not have enough flexibility to screen someone out for this reason.
Keep in mind that even if a participant is difficult to understand, he still has valuable feedback to offer!
Go into prioritization mode. Think across all your questions or tasks and consider what can be dropped and skimmed over. Or resolve to go partway through your tasks or questions and let the session end where it ends.
If your priority is going for breadth rather than depth (i.e., getting a small amount of feedback on each of the remaining tasks/questions instead of attempting fewer complete tasks), consider turning the rest of the session into a “gut-reaction” game where the participant shows or tells you his first inclination in response to a task or question. For example, instead of having the participant complete a task of finding a savings account that meets his needs on a banking website, bring him to the homepage and ask him where he would click first if he was going to attempt that task. This approach will quickly give you “first-click” information that, depending on your research goals, may be useful.
When you’ve gathered the information you need from a question/task, gently push the participant along to your next item. If you plan to do this repeatedly, warn participants that due to time constraints, you may keep moving them along.
Consider narrowing the focus of any remaining tasks/questions. For example, you may tell the participant to only focus on a certain part of the design or process rather than the full experience.
If a participant seems to feel bad at the end of the session because he didn’t get through all of your tasks/questions, assure him that you hadn’t planned to get through everything, and take the blame for trying to fit too much into one session.
To gently bring the participant to the next task/question:
“This is all very helpful to see and hear. Just for the sake of time, I’m going to ask you to <go to the next task/go back to.…>”
“I’m interested in hearing more about this. I do want to make sure we cover everything we have planned, so if there’s time, let’s come back to this at the end of the session.”
If you know that you need to move quickly:
“I just did a time check, and we’re running a bit low on time. We have a lot of <questions/tasks> that I’d like to get your feedback on, so I may just push you along at times so that we can cover as much as possible.”
“For the remaining tasks, we’re going to move faster than we have before. For each of the following tasks, show me where you would click first.”
If the participant feels bad about not getting through everything at the end of session:
Don’t rush participants through tasks or questions for the sake of getting everything done. Adjust and prioritize what you need participants to do based on the time available so they don’t leave the session feeling like they were rushed.
Don’t panic. If you’re concerned about time, try not to let it show. This, of course, comes with practice but it’s good to be mindful of it.
Run a pilot session with a participant to make sure that you have an accurate sense of timing. Of course, you’ll always get circumstances that result in a time crunch, but at least you can adjust the things that are under your control.
As you put together your study plan, identify which questions/tasks are the highest priority. This will help you adapt the session if necessary as discussed in section 15.2.
Watching where and why participants struggle is a main purpose of performing usability studies. Discovering those pain points will help you improve the product for future users (Dumas and Loring, 2008). However, excessive struggling can take an emotional toll on the participant’s well-being, or use up valuable session time that could be spent exploring other research questions.
If the participant is struggling excessively to the point where it’s taking too much time or no longer providing data that’s relevant to your research goals, but the participant is not showing signs of being emotionally distraught, have him stop what he’s doing and move onto the next question or task. If moving on requires that you give an assist (as discussed in section 6.4), go ahead and do so.
If the participant is displaying minor emotional discomfort, such as frustration or annoyance with the product, or seems to be less engaged in what he is doing, take a short break. After giving him a few minutes, check in with him privately to make sure that he’s comfortable continuing. When the break is over, reassure the participant that his feedback has been helpful and then move on to a different task or question. If he becomes more upset, see section 12.7 and end the session early if necessary.
To move on from the current task:
To take a break:
“This seems like a good stopping point. Why don’t we take a quick five-minute break?”
“I’m going to turn off our recording equipment, and give you a short break. I’m going to step out, but I’ll be back in a few.”
While talking in private:
“I don’t want you to continue the session if you’re feeling uncomfortable. You’ve already given us very helpful feedback. Is there anything I can do to help, or would you like to end here?”
If ending the session, and the participant feels bad:
This situation is not usually avoidable, but the tone you set at the beginning of the session can affect the emotional impact that struggling has on the participant. If he seems particularly sensitive or nervous at the start of the session, you may need to adapt your moderating style to be more approachable and reassuring, as discussed in section 1.6.