This chapter covers situations where there is a mismatch between the participant's expectations for the sessions and what you have planned. These include situations such as a participant who brings someone else to participate with her, a participant who thinks she is there for something other than a one-on-one user research study, and a participant who has concerns about logistics such as being recorded and compensation. For each situation you'll learn what to do, what to say, what not to do or say, and how you might be able to avoid it in the future.
user research recording; user research consent form; user research compensation; contextual inquiry
5.1 Participant thinks that she is participating in a focus group
5.2 Participant doesn’t want to be recorded or has other concerns about the consent form
5.3 Participant has different expectations for the compensation
5.4 Participant brings you to a conference room or other space instead of her office/workspace
5.5 Participant treats a contextual inquiry like an interview
5.6 Participant brings someone else to participate with her
When there is a mismatch between the participant’s expectations for the session and what you have planned, you need to clarify your expectations before continuing. If the participant brought someone or something else with her to the session, you also need to tactfully and quickly figure out if, and how, to proceed. These situations require you to immediately take responsibility for any misunderstanding and figure out the best way to move forward.
Most people are familiar with focus groups but not with usability studies, so this is a common misperception. Let the participant know that what she’ll be doing is similar to a focus group, but that she’ll be the only person in the room with you and you’ll be asking her to interact with a product directly rather than having a group discussion.
Ask if the participant has been in a focus group before. This approach provides a lightweight way to get her talking, and may also reveal if the participant has actually participated in any kind of research before or if she has just heard of focus groups.
If the participant assumed she signed up to participate in a focus group where she could hide behind a crowd, she may be shyer about performing tasks and thinking aloud. She may not be a personality type who would have signed up for a usability study. Because of this, you may need to adjust your style to be comforting and engaging until she feels more comfortable. You may also need to reiterate the kind of feedback you’re looking to receive from her throughout the session.
“What you’ll be helping us with today is similar to a focus group, but you’ll be the only person in the room with me and you’ll be working with a product. I’m going to ask you to try to use the product to accomplish some tasks while thinking aloud about your experience. This will give us feedback on how the product works—both what works well and what should be improved. Let me tell you a bit more about how our session will work.…”
“It sounds like you’re familiar with focus groups. Have you participated in any kind of research, like a focus group or usability study before?”
Don’t disparage focus groups. Many user experience practitioners see focus groups as a technique that produces misleading results, but someone who has been in a focus group and had a great time doesn’t need to hear your take on the methodology. Instead, focus (no pun intended) on the more practical aspects of how the research you’re doing today will be different than a focus group. For example, the session is one-on-one, and the participant will be interacting with a product.
In your recruiting screener and confirmation letter, be explicit about the type of research you’re doing and that the session will be one-on-one between the participant and a moderator (you). However, be aware that no matter how explicit you are during the recruiting process, you’ll still get participants who assume they’re there for a focus group.
See if there is a level of recording that the participant is comfortable with. For example, instead of recording picture-in-picture, offer to remove the capture of the participant’s face while still recording the screen and audio. However, don’t press the point. The participant is under no obligation to be recorded if she doesn’t want to be, and a recording is usually nice to have instead of a necessity for the session.
If the participant has issues with video/audio altogether, ask if it would be okay for you to just take detailed notes or have another person in the room to help take notes. If she agrees to having another person take notes, ask your colleague to join you and sit behind you quietly with a notepad or note-taking device.
If you’re providing session recordings to stakeholders, be explicit about their storage and deletion responsibilities. You can share those responsibilities with the participant if she is nervous about how the recordings will be used and reassure her that her video will not end up on YouTube!
If the participant is concerned about any other clause in the consent form, ask her to explain her concern. Do your best to address her concern, but if she is still uncomfortable, offer to cancel the session and let her leave with her full compensation.
Be sure the participant signs the consent form before the session begins. If she refuses to be recorded but is willing to continue, mark up the consent form to reflect the change. Barring legal advice otherwise (see “How to avoid”), use a permanent pen or marker of a different color to make the edits required for the participant to continue. Make sure the edits are obvious. If a participant doesn’t want to be recorded, for example, cross out the line regarding recording and write in big letters at the bottom, “NO RECORDING.” For recognition purposes, it may help if you place your initials next to the change, and have the participant add her initials next to yours.
“I know you’re concerned about the recording. Let me explain what would actually be recorded if you agree to it. The recording is a picture-in-picture showing the computer screen in the big window, and a small inset of your face in the corner. If you prefer, we can set it up so we do not capture your face, only your voice. The recording will only be used to make sure that we have an accurate record of your feedback. It will only be shared with the team working on this project, for this project—not for any other purpose. Does that make sense?”
“Would it be okay if we adjusted what is being recorded so we’re only capturing <what’s being recorded, e.g., just your voice and what you’re interacting with, without capturing your face>?”
“If you don’t mind, tell me what’s concerning you about <topic of concern>.”
Be explicit about the recording and how it will be used. Have whoever is handling the recruit share this information with the participant during the recruitment process and in her confirmation letter. Make sure that the participant knows the recording is optional, which it usually should be. Also share your consent form with the recruiter so she can answer participant questions ahead of time. If you work with the same recruiter multiple times, highlight anything unique or different that will be included in your study’s consent form that hasn’t been in previous forms.
When a participant arrives, explain the consent form rather than just handing it to her. If you explain the clauses in plain terms upfront, she is less likely to panic when trying to read and interpret the text on her own.
Ensure that your consent form is written in plain language that clearly explains what will be happening during the session and what will happen with the recording. This is often challenging in commercial environments, but is well worth the effort. Keep in mind that the consent form is a legal contract and work with your legal team to craft something that is both legal and clear. See Appendix C for user research method books that go into more detail about consent forms.
If you have access to legal representatives at your organization, discuss with them how much leverage you have in adjusting consent forms on-the-fly. For example, you may be allowed to use a pen to make a change to the form, put your initials next to the change, and have the participant put her initials next to yours.
Expect that some participants will prefer not to be recorded, and plan accordingly. Have a note-taker to support your session or the materials with you to take your own notes.
Address this issue as soon as it comes up. Ideally, a participant will notice the discrepancy at the beginning of the session when she signs the consent form (which should explicitly state the compensation she’ll receive for participating). But, if it comes up later in the session, address it right away instead of waiting until the very end.
If there is a way to provide the compensation in a different format, and the participant is insistent on receiving it that way, go ahead and make the adjustment if you can (and if there are no issues from a legal perspective for your organization—some organizations require that compensation be handled in very specific ways).
If the participant is insistent that the recruiter promised her a different amount, apologize for the misunderstanding and ask if she has something documenting that amount, such as a confirmation letter, email, or voicemail. If there was indeed a mistaken promise, find a way to give her the promised amount even if it’s different than what you compensated everyone else. Then, immediately follow up with the recruiter (if applicable) to see if this was a mistake that happened with other participants as well. If the participant can’t document the misunderstanding, follow the earlier recommendations for what to do.
If the participant is expecting a different amount because she knows other people who participated who received that amount, go ahead and make the adjustment if the inconsistency is an oversight on your part. If the other participants were doing something distinct from this session (e.g., had 1.5-hour sessions instead of 1-hour sessions), apologize and explain the discrepancy to the participant.
Be prepared to explain any compensation formats that a participant may be unfamiliar with. For example, when we’ve provided participants with American Express cash cards, we also spend a few minutes explaining how those cards work and can be used.
If you’re trying to understand where the misconception came from:
“Do you have a copy of your confirmation letter, or a voicemail from <recruiter>? I believe that she specified your compensation amount in there.”
If you’re able to accommodate the different amount or format:
“I’m sorry for the misunderstanding. Please give me just a couple of extra minutes so I can make this adjustment for you.”
If you’re unable to accommodate the different expectation:
“I apologize for any confusion about this. Your confirmation letter for the session specified the amount that we can provide and the format that we can offer. If you’re unwilling to continue, let me know.”
“I’m so sorry about this. We’re unable to offer compensation in a different format due to company restrictions.”
Don’t criticize or blame your recruiter in front of the participant, even if you see proof that she provided the participant with incorrect information. Be professional and apologetic about the mistake, keeping in mind that the participant may not see a distinction between your organization and the one that recruited her (if different).
Try to be consistent with compensation amounts among participants, as participants sometimes encounter and talk to each other and may be unhappy at discovering that someone else is getting paid a different amount for the same amount of time.
The participant may not understand why you want to see her in her workspace, and assumes that she is being helpful by providing a conference room or other space instead. Thank her for reserving the space, but explain that one of your goals is to understand how she works when you’re not there, and that seeing her environment and what she does within that environment will provide the information that you need.
The participant may feel like she doesn’t have space for someone to join her in her workspace, or that her space is too messy for visitors. Let her know that you don’t need a lot of space, and that it doesn’t matter how it looks—you’re interested in her day-to-day reality, not the tidied-up version of her work.
If the participant has a crowded or open workspace, offer to chat in private with the participant at the end of the session. This private conversation may be an opportunity for her to give any feedback that she doesn’t want her coworkers or manager to overhear in her work environment.
“I realize I may not have been clear enough <in the confirmation letter/when we talked earlier>. It’s really important for us to be in the environment that you’re normally working in. Could we move there instead? It’s okay if it’s noisy or cramped. We can come back to this conference room at the end of the session to wrap up.”
“I’d like to watch you work in your own environment, so that I can witness your day-to-day tasks. But after that, we can debrief back in a conference room. Does that sound okay to you?”
When setting up the session, specify that you want to be with the participant in her workspace, even if it’s noisy. Explain that because your focus is on seeing how she actually works, it’s important to be in her own environment. If the participant seems reluctant to have you in that space (e.g., because she sits right next to her manager), offer to split the session between her space and a conference room so you can get the best of both worlds.
This situation may happen because the participant was given vague details when recruited for the study, or made the wrong assumption when asked to participate. Most people are familiar with the idea of being interviewed, but a contextual inquiry is not a very common method outside of the user research community.
Clarify your expectations as soon as you realize what is happening, and apologize for not being clear enough ahead of time about what you were planning. Hopefully the participant will be able to adapt and you can continue the session as planned. If you can’t continue as planned, see if you can reschedule or shift the focus to some other way to get feedback.
If she seems embarrassed or feels bad about the mix-up, take responsibility for the miscommunication (even if was someone else’s fault).
Prepare a list of questions for times when you can’t watch the participant work. This may come up even without a misunderstanding. For example, the participant’s system may be down, or she may have completed a project early and have nothing to show you.
“What I was actually looking to do today is watch you work in the way that you typically would if I weren’t here. I may have some questions for you along the way, or things I ask you to show me, but in general, you should be working and my role is to mostly be a wallflower and watch quietly.”
“Are you able to get set up so that I can quietly just watch you while you work?”
“I’m so sorry for the mix-up—I must not have clearly communicated my intent for this session. If it isn’t too much trouble, could you go ahead and show me what you’re describing?”
Don’t end the session, if you can help it. Try to find a way to have the participant get set up to work. If that is not possible, shift the focus of the research if turning it into an interview would provide some useful information. As a last resort, find a time to reschedule if possible and end the session. Since site visits usually take a lot of effort and time to plan, you should do your best to get some feedback out of the session.
When you’re running a study within an organization, this misunderstanding may occur because participants (or the people recruiting the participants) don’t understand the purpose of your research and assume that it’s a training or sales session. Or, the other attendees may just be really interested in what you’re doing and want a chance to provide their own feedback. The best thing to do in this situation depends on who the other attendees are.
If the other attendees have similar roles/responsibilities (or fit the original recruit criteria) as the participant:
Schedule additional sessions to talk to the other attendees, and proceed one-on-one with the original participant. This technique may be appropriate if you have additional time slots available.
Turn the session into a codiscovery session. This technique may be appropriate if there is only one additional attendee who cannot be scheduled for a separate session, and the session is a usability study or interview.
Talk to each attendee separately, one-on-one, for a shorter period of time.
Make sure to give or send consent forms to the additional participants and have them review and sign the form before continuing the session.
If the other attendees have very different roles/responsibilities (and do not fit the original recruit criteria) or are the participant’s manager, family, or friends:
Explain to the other attendees that your goal is to talk one-on-one with the original participant and that, while you’re excited that they’re so interested in giving feedback, you need them to leave the room. If they have a misconception that they’ll be missing out on something, clarify that you’re not providing any kind of training.
If they insist that they’ll sit quietly during the session, politely decline. Their presence may change the behavior of the participant. Instead, explain that the session is really designed to be one-on-one with the participant with no one else present. If you ultimately decide to let them stay, give or send them consent forms to sign as soon as possible. This form may even be a deterrent to the additional attendees staying, since it may emphasize the session’s level of formality.
Take responsibility for not being clear enough about the purpose of the session and the need for it to be one-on-one.
“It’s great that you’re both so enthusiastic about participating. Can you tell me why you’re interested in participating?”
If the other attendees are trying to not miss out on a training opportunity:
“I’m running this study to <get early feedback from users on designs/observe the current experience of the product>. I won’t be providing any kind of training, just watching as <participant> uses the system.”
If dismissing other person/people:
Don’t let the participant continue the session if her manager is in the room or observing the session, as you do not want any power issues to get in the way of your feedback.
If you need to ask the additional attendees to leave, don’t be rude about it. Be firm, but kind. You’re representing your organization to anyone you come in contact with, even if they don’t wind up as a participant. You don’t want to create any grievances with a manager who has a lot of clout in your organization, or with an important customer.
This situation often occurs because the participant is unclear about what will be happening during the session. Explain the setup of the session to the participant before she arrives (e.g., during the recruit and in the confirmation letter) and make sure the one-on-one aspect is clear. However, even if you’re very explicit, you can expect to still run into this situation and should plan accordingly.
This misunderstanding may happen if you’re doing research for a well-known organization. If someone is unemployed and expecting calls for interviews, she may feasibly misinterpret a user research recruitment call for the other type of recruitment call. This can also happen accidentally if you have a main waiting area for lots of different people—site visitors, job interview candidates, customers, and user research participants. It’s easy to bring in the wrong person and, believe us, things can get quite confusing and interesting after that.
Be very clear and explicit in the briefing at the beginning of the session about why the participant is there and what she’ll be doing. Hopefully any misunderstanding will surface at this time.
If at any point you suspect that the participant is there for a job interview, try asking a couple of general questions. For example, ask what made her come in today, or if she has participated in a user research session before, and reconfirm the recruiting criteria. If it’s still not clear, you might need to directly ask her if she is there for the session.
If the miscommunication becomes clear at the start of the session (e.g., when asking background questions), spare the poor participant further embarrassment and explain the actual purpose of the session. Take responsibility for the miscommunication and offer to cancel the session while still providing her with the full compensation. If you realize that you brought in the wrong person from the waiting area, quickly find the participant!
If you’ve offered to end the session but the participant seems legitimately interested in participating, consider letting her continue (unless your actual participant is waiting for you). If she doesn’t fully meet the recruiting criteria, you could see if there is a way to let her stay and feel useful rather than that she wasted the travel time and a perfectly nice interview outfit. You may want to:
Run the session anyway if she is able to get through the tasks and questions without needing specialized knowledge or skills. Just make note in your findings how this participant deviated from the recruiting criteria.
If the participant can’t continue the tasks without key skills, consider shifting the focus of the session to something that better suits her. For instance, turn a usability study into an interview, or refocus interview questions on topics that you can still make use of and that she understands.
If the participant asks you to connect her with someone specific at the organization, or wants you to take her resume and “pass it along,” do what makes you feel comfortable. Don’t do anything out of obligation or to just placate her. If you don’t want to be involved, try to find a gentle way to let her down and point her in a more helpful direction. For example, you may say that you’re a consultant (if that is true), so you don’t really have any connection with the other parts of the organization but know that the corporate website has a section for job applicants.
If unclear about why the participant is there:
Start general: “What made you decide to participate today?”
More specific: “You’re <name>, correct? And you’re here for the research study?”
At the beginning of session, if you realize there is a misunderstanding:
“<Participant>, I’m really sorry but I think there was a misunderstanding. This is not a job interview but rather a research study. It’s completely our fault for not being clear. If you’d like, I can just give you the promised compensation and we’ll cancel the session.”
“I completely understand if you want to leave, but if you’re still interested in the study, I could tell you about it.”
If the participant asks for a connection or interview and you don’t want to be involved:
“I’m sorry. I wish I could help but I’m just here to run the session today. If you want, I can provide you the link to the careers web page.”
“I am really just involved in user research, so I can’t help you with that. I can walk you to the front desk if you’d like, and they may be able to put you in touch with someone in HR.”
Provide your recruiter with a script that is explicit about recruiting for a research session. If someone within an organization is doing the recruit for you, provide her with an explanation of the session that can be shared with potential participants to alleviate any confusion. We recommend talking with your recruiter about it as well so she is very clear that the session is for research, not an interview.
When you greet the participant, greet her by name and confirm that she is the right person before bringing her into your research space.
Confirm that the participant is aware of how long the session will last and what you’ll be asking her to do. You can also show her the space that you’ll be in, as that may change her mind about continuing (in which case, you can offer to reschedule her or just offer the compensation for coming in).
If the participant has brought a child, ideally she came prepared with items to keep her child occupied during the session. If not, take a deep breath and get creative! Depending on the child’s age, you can volunteer to bring in some paper and markers for her to use, or some other set of (safe!) materials for the child to play with, such as a deck of cards or a tablet.
While we haven’t seen participants bring pets other than dogs to usability sessions (unless they’re service animals), we’re sure it can happen. In that case, raise any potential concerns with the pet being there. For example, you need to limit future participants’ exposure to allergens, and make sure the pet’s behavior and “calls to nature” can be controlled without interfering with your session.
If the child or pet is being excessively and/or consistently disruptive after an initial period of time, try to be patient and understanding if the participant seems eager to continue anyway. But if there are constant disruptions, you might want to end the session early. Do so delicately, without blaming the child or pet. If you need to use a pretext, you can blame technical difficulties or explain that you made it through all your questions faster than anticipated. Offer again to reschedule for a time when the participant can come on her own. If she is unable to reschedule, we recommend still giving her the full compensation.
When the participant arrives:
“This is meant to be a focused one-on-one session. Will your <child/pet> be okay in the room with us? Our setup may not be comfortable for <child/pet>, so we’re happy to reschedule you for a time when you can come by yourself.”
If concerned about a pet, blame allergies:
“I’m sorry, we try to keep this room allergen-free since we have so many people coming through here. Are you able to reschedule for a time when you can come by yourself?”
If child/pet is being excessively disruptive and you need to end early:
Do not allow team members to babysit or petsit during the session, even from within the research room. Doing so would potentially leave your organization vulnerable if something happened to the child or pet while they were out of the participant’s view. Also, you don’t want to risk the possibility of your colleague being bitten (by a child or pet!). If the participant asks for this service, let her know that you’re not equipped to do so and offer to reschedule at a more convenient time when she can come in on her own.
Explain the setup of the session to the participant before she arrives (e.g., during the recruit and in the confirmation letter) and make sure the one-on-one aspect is clear.