Chapter 5

Participant Misconceptions

Not What the Participant was Expecting

Abstract

This chapter covers situations where there is a mismatch between the participant's expectations for the sessions and what you have planned. These include situations such as a participant who brings someone else to participate with her, a participant who thinks she is there for something other than a one-on-one user research study, and a participant who has concerns about logistics such as being recorded and compensation. For each situation you'll learn what to do, what to say, what not to do or say, and how you might be able to avoid it in the future.

Keywords

user research recording; user research consent form; user research compensation; contextual inquiry

5.1 Participant thinks that she is participating in a focus group

5.2 Participant doesn’t want to be recorded or has other concerns about the consent form

5.3 Participant has different expectations for the compensation

5.4 Participant brings you to a conference room or other space instead of her office/workspace

5.5 Participant treats a contextual inquiry like an interview

5.6 Participant brings someone else to participate with her

5.7 Participant thinks the session is a job interview

5.8 Participant brings a child or pet to the session

When there is a mismatch between the participant’s expectations for the session and what you have planned, you need to clarify your expectations before continuing. If the participant brought someone or something else with her to the session, you also need to tactfully and quickly figure out if, and how, to proceed. These situations require you to immediately take responsibility for any misunderstanding and figure out the best way to move forward.

5.1 Participant thinks that she is participating in a focus group

When the participant arrives, it becomes clear that she is expecting to be in a focus group with other people. She may ask where everyone else is, or talk about her previous experience participating in focus groups. Or, she may arrive late because she assumes that the group won’t be delayed or affected by one person not being there on time.

Method(s): In-person usability study

Frequency: Frequent

Pattern(s) to apply: Clarify the task/question; Reassure the participant; Build engagement

What to do

ent Most people are familiar with focus groups but not with usability studies, so this is a common misperception. Let the participant know that what she’ll be doing is similar to a focus group, but that she’ll be the only person in the room with you and you’ll be asking her to interact with a product directly rather than having a group discussion.

ent Ask if the participant has been in a focus group before. This approach provides a lightweight way to get her talking, and may also reveal if the participant has actually participated in any kind of research before or if she has just heard of focus groups.

ent If the participant assumed she signed up to participate in a focus group where she could hide behind a crowd, she may be shyer about performing tasks and thinking aloud. She may not be a personality type who would have signed up for a usability study. Because of this, you may need to adjust your style to be comforting and engaging until she feels more comfortable. You may also need to reiterate the kind of feedback you’re looking to receive from her throughout the session.

What to say

ent “What you’ll be helping us with today is similar to a focus group, but you’ll be the only person in the room with me and you’ll be working with a product. I’m going to ask you to try to use the product to accomplish some tasks while thinking aloud about your experience. This will give us feedback on how the product works—both what works well and what should be improved. Let me tell you a bit more about how our session will work.…”

ent “It sounds like you’re familiar with focus groups. Have you participated in any kind of research, like a focus group or usability study before?”

What not to do or say

ent Don’t disparage focus groups. Many user experience practitioners see focus groups as a technique that produces misleading results, but someone who has been in a focus group and had a great time doesn’t need to hear your take on the methodology. Instead, focus (no pun intended) on the more practical aspects of how the research you’re doing today will be different than a focus group. For example, the session is one-on-one, and the participant will be interacting with a product.

How to avoid

ent In your recruiting screener and confirmation letter, be explicit about the type of research you’re doing and that the session will be one-on-one between the participant and a moderator (you). However, be aware that no matter how explicit you are during the recruiting process, you’ll still get participants who assume they’re there for a focus group.

5.2 Participant doesn’t want to be recorded or has other concerns about the consent form

The participant was recruited with the understanding that the session will be recorded. However, once she arrives and reads through the consent form, she decides that she is uncomfortable with being recorded and will not proceed unless you refrain from recording the session. If she is remote, she may change her mind about being recorded once she is in the session, even if she already sent you back a signed copy of the consent form. Or, the participant may have other concerns about the consent form and refuses to sign it unless you make changes.

Method(s): Any

Frequency: Occasional

Pattern(s) to apply: Reassure the participant

What to do

ent See if there is a level of recording that the participant is comfortable with. For example, instead of recording picture-in-picture, offer to remove the capture of the participant’s face while still recording the screen and audio. However, don’t press the point. The participant is under no obligation to be recorded if she doesn’t want to be, and a recording is usually nice to have instead of a necessity for the session.

ent If the participant has issues with video/audio altogether, ask if it would be okay for you to just take detailed notes or have another person in the room to help take notes. If she agrees to having another person take notes, ask your colleague to join you and sit behind you quietly with a notepad or note-taking device.

ent If you’re providing session recordings to stakeholders, be explicit about their storage and deletion responsibilities. You can share those responsibilities with the participant if she is nervous about how the recordings will be used and reassure her that her video will not end up on YouTube!

ent If the participant is concerned about any other clause in the consent form, ask her to explain her concern. Do your best to address her concern, but if she is still uncomfortable, offer to cancel the session and let her leave with her full compensation.

ent Be sure the participant signs the consent form before the session begins. If she refuses to be recorded but is willing to continue, mark up the consent form to reflect the change. Barring legal advice otherwise (see “How to avoid”), use a permanent pen or marker of a different color to make the edits required for the participant to continue. Make sure the edits are obvious. If a participant doesn’t want to be recorded, for example, cross out the line regarding recording and write in big letters at the bottom, “NO RECORDING.” For recognition purposes, it may help if you place your initials next to the change, and have the participant add her initials next to yours.

What to say

ent “I know you’re concerned about the recording. Let me explain what would actually be recorded if you agree to it. The recording is a picture-in-picture showing the computer screen in the big window, and a small inset of your face in the corner. If you prefer, we can set it up so we do not capture your face, only your voice. The recording will only be used to make sure that we have an accurate record of your feedback. It will only be shared with the team working on this project, for this project—not for any other purpose. Does that make sense?”

ent “Would it be okay if we adjusted what is being recorded so we’re only capturing <what’s being recorded, e.g., just your voice and what you’re interacting with, without capturing your face>?”

ent “If you don’t mind, tell me what’s concerning you about <topic of concern>.”

What not to do or say

ent Don’t dismiss the participant’s concerns. The prevalence of platforms such as YouTube and Vimeo have shown how easy it is to share video, so a participant’s concerns regarding what will happen with her recording is warranted.

How to avoid

ent Be explicit about the recording and how it will be used. Have whoever is handling the recruit share this information with the participant during the recruitment process and in her confirmation letter. Make sure that the participant knows the recording is optional, which it usually should be. Also share your consent form with the recruiter so she can answer participant questions ahead of time. If you work with the same recruiter multiple times, highlight anything unique or different that will be included in your study’s consent form that hasn’t been in previous forms.

ent When a participant arrives, explain the consent form rather than just handing it to her. If you explain the clauses in plain terms upfront, she is less likely to panic when trying to read and interpret the text on her own.

ent Ensure that your consent form is written in plain language that clearly explains what will be happening during the session and what will happen with the recording. This is often challenging in commercial environments, but is well worth the effort. Keep in mind that the consent form is a legal contract and work with your legal team to craft something that is both legal and clear. See Appendix C for user research method books that go into more detail about consent forms.

ent If you have access to legal representatives at your organization, discuss with them how much leverage you have in adjusting consent forms on-the-fly. For example, you may be allowed to use a pen to make a change to the form, put your initials next to the change, and have the participant put her initials next to yours.

ent Expect that some participants will prefer not to be recorded, and plan accordingly. Have a note-taker to support your session or the materials with you to take your own notes.

5.3 Participant has different expectations for the compensation

The participant thought that she was receiving either a different amount for her compensation (e.g., $100 instead of $75) or a different format (e.g., cash instead of a gift card). She may get very upset about this misunderstanding and feel like she was lied to.

Method(s): Any

Frequency: Occasional

Pattern(s) to apply: Take responsibility

What to do

ent Address this issue as soon as it comes up. Ideally, a participant will notice the discrepancy at the beginning of the session when she signs the consent form (which should explicitly state the compensation she’ll receive for participating). But, if it comes up later in the session, address it right away instead of waiting until the very end.

ent If there is a way to provide the compensation in a different format, and the participant is insistent on receiving it that way, go ahead and make the adjustment if you can (and if there are no issues from a legal perspective for your organization—some organizations require that compensation be handled in very specific ways).

ent If the participant is insistent that the recruiter promised her a different amount, apologize for the misunderstanding and ask if she has something documenting that amount, such as a confirmation letter, email, or voicemail. If there was indeed a mistaken promise, find a way to give her the promised amount even if it’s different than what you compensated everyone else. Then, immediately follow up with the recruiter (if applicable) to see if this was a mistake that happened with other participants as well. If the participant can’t document the misunderstanding, follow the earlier recommendations for what to do.

ent If the participant is expecting a different amount because she knows other people who participated who received that amount, go ahead and make the adjustment if the inconsistency is an oversight on your part. If the other participants were doing something distinct from this session (e.g., had 1.5-hour sessions instead of 1-hour sessions), apologize and explain the discrepancy to the participant.

ent Be prepared to explain any compensation formats that a participant may be unfamiliar with. For example, when we’ve provided participants with American Express cash cards, we also spend a few minutes explaining how those cards work and can be used.

What to say

If you’re trying to understand where the misconception came from:

ent “Do you have a copy of your confirmation letter, or a voicemail from <recruiter>? I believe that she specified your compensation amount in there.”

If you’re able to accommodate the different amount or format:

ent “I’m sorry for the misunderstanding. Please give me just a couple of extra minutes so I can make this adjustment for you.”

If you’re unable to accommodate the different expectation:

ent “I apologize for any confusion about this. Your confirmation letter for the session specified the amount that we can provide and the format that we can offer. If you’re unwilling to continue, let me know.”

ent “I’m so sorry about this. We’re unable to offer compensation in a different format due to company restrictions.”

What not to do or say

ent Don’t criticize or blame your recruiter in front of the participant, even if you see proof that she provided the participant with incorrect information. Be professional and apologetic about the mistake, keeping in mind that the participant may not see a distinction between your organization and the one that recruited her (if different).

ent Try to be consistent with compensation amounts among participants, as participants sometimes encounter and talk to each other and may be unhappy at discovering that someone else is getting paid a different amount for the same amount of time.

How to avoid

ent Be explicit in the confirmation letter about the promised compensation, its delivery mechanism, and any other reimbursement being offered (e.g., if you’ll reimburse up to a certain amount for parking/transportation expenses).

5.4 Participant brings you to a conference room or other space instead of her office/workspace

You’ve scheduled a contextual inquiry at a participant’s work location with the goal of watching her in her own environment. However, when you arrive, the participant brings you to a conference room instead, explaining that it will be quieter and easier to talk there.

Method(s): Contextual inquiry

Frequency: Occasional

Pattern(s) to apply: Take responsibility

What to do

ent The participant may not understand why you want to see her in her workspace, and assumes that she is being helpful by providing a conference room or other space instead. Thank her for reserving the space, but explain that one of your goals is to understand how she works when you’re not there, and that seeing her environment and what she does within that environment will provide the information that you need.

ent The participant may feel like she doesn’t have space for someone to join her in her workspace, or that her space is too messy for visitors. Let her know that you don’t need a lot of space, and that it doesn’t matter how it looks—you’re interested in her day-to-day reality, not the tidied-up version of her work.

ent If the participant has a crowded or open workspace, offer to chat in private with the participant at the end of the session. This private conversation may be an opportunity for her to give any feedback that she doesn’t want her coworkers or manager to overhear in her work environment.

What to say

ent “I realize I may not have been clear enough <in the confirmation letter/when we talked earlier>. It’s really important for us to be in the environment that you’re normally working in. Could we move there instead? It’s okay if it’s noisy or cramped. We can come back to this conference room at the end of the session to wrap up.”

ent “I’d like to watch you work in your own environment, so that I can witness your day-to-day tasks. But after that, we can debrief back in a conference room. Does that sound okay to you?”

What not to do or say

ent Don’t blame the participant for misunderstanding—take responsibility for not being clear enough about what you’re doing or what you need.

How to avoid

ent When setting up the session, specify that you want to be with the participant in her workspace, even if it’s noisy. Explain that because your focus is on seeing how she actually works, it’s important to be in her own environment. If the participant seems reluctant to have you in that space (e.g., because she sits right next to her manager), offer to split the session between her space and a conference room so you can get the best of both worlds.

Watch Video 6 to see an example of a participant who is expecting an interview instead of a contextual inquiry. The moderator clarifies her expectations, but has to repeat them a couple of times as the session progresses.

Visit our website (http://www.modsurvivalguide.org/videos) or use your QR reader to scan this code. image

5.5 Participant treats a contextual inquiry like an interview

You arrive onsite to perform a contextual inquiry but the participant expects that she just needs to talk to you about what she does with a product and what she thinks of it. For example, a call center representative anticipates an interview and schedules special time off from her phone queue to chat with you. Your goal was actually to watch her take calls and ask her questions in between calls. If you’re doing the session remotely, the participant may just talk about her workflow instead of showing it to you through the screen-sharing software.

Method(s): Contextual inquiry

Frequency: Occasional

Pattern(s) to apply: Take responsibility; Clarify the task/question; Shift the focus

What to do

This situation may happen because the participant was given vague details when recruited for the study, or made the wrong assumption when asked to participate. Most people are familiar with the idea of being interviewed, but a contextual inquiry is not a very common method outside of the user research community.

ent Clarify your expectations as soon as you realize what is happening, and apologize for not being clear enough ahead of time about what you were planning. Hopefully the participant will be able to adapt and you can continue the session as planned. If you can’t continue as planned, see if you can reschedule or shift the focus to some other way to get feedback.

ent If she seems embarrassed or feels bad about the mix-up, take responsibility for the miscommunication (even if was someone else’s fault).

ent Prepare a list of questions for times when you can’t watch the participant work. This may come up even without a misunderstanding. For example, the participant’s system may be down, or she may have completed a project early and have nothing to show you.

What to say

ent “What I was actually looking to do today is watch you work in the way that you typically would if I weren’t here. I may have some questions for you along the way, or things I ask you to show me, but in general, you should be working and my role is to mostly be a wallflower and watch quietly.”

ent “Are you able to get set up so that I can quietly just watch you while you work?”

ent “I’m so sorry for the mix-up—I must not have clearly communicated my intent for this session. If it isn’t too much trouble, could you go ahead and show me what you’re describing?”

What not to do or say

ent Don’t end the session, if you can help it. Try to find a way to have the participant get set up to work. If that is not possible, shift the focus of the research if turning it into an interview would provide some useful information. As a last resort, find a time to reschedule if possible and end the session. Since site visits usually take a lot of effort and time to plan, you should do your best to get some feedback out of the session.

How to avoid

ent During the recruit, specify the exact setup of the session and what it involves. For example, include in your recruiting script and confirmation letter that you need to watch the participant at her work computer using a specific application.

Watch Video 6 to see an example of a participant who is expecting an interview instead of a contextual inquiry. The moderator clarifies her expectations, but has to repeat them a couple of times as the session progresses.

Visit our website (http://www.modsurvivalguide.org/videos) or use your QR reader to scan this code. image

5.6 Participant brings someone else to participate with her

You have scheduled a participant for a one-on-one session. However, when she arrives (or when you call her if she is remote), she has brought one or more additional people with her to participate as well. This other person may be a coworker, her manager, or a friend who she thinks would be interested as well.

Method(s): Any

Frequency: Rare

Pattern(s) to apply: Take responsibility

What to do

When you’re running a study within an organization, this misunderstanding may occur because participants (or the people recruiting the participants) don’t understand the purpose of your research and assume that it’s a training or sales session. Or, the other attendees may just be really interested in what you’re doing and want a chance to provide their own feedback. The best thing to do in this situation depends on who the other attendees are.

If the other attendees have similar roles/responsibilities (or fit the original recruit criteria) as the participant:

ent Schedule additional sessions to talk to the other attendees, and proceed one-on-one with the original participant. This technique may be appropriate if you have additional time slots available.

ent Turn the session into a codiscovery session. This technique may be appropriate if there is only one additional attendee who cannot be scheduled for a separate session, and the session is a usability study or interview.

ent Talk to each attendee separately, one-on-one, for a shorter period of time.

ent Make sure to give or send consent forms to the additional participants and have them review and sign the form before continuing the session.

If the other attendees have very different roles/responsibilities (and do not fit the original recruit criteria) or are the participant’s manager, family, or friends:

ent Explain to the other attendees that your goal is to talk one-on-one with the original participant and that, while you’re excited that they’re so interested in giving feedback, you need them to leave the room. If they have a misconception that they’ll be missing out on something, clarify that you’re not providing any kind of training.

ent If they insist that they’ll sit quietly during the session, politely decline. Their presence may change the behavior of the participant. Instead, explain that the session is really designed to be one-on-one with the participant with no one else present. If you ultimately decide to let them stay, give or send them consent forms to sign as soon as possible. This form may even be a deterrent to the additional attendees staying, since it may emphasize the session’s level of formality.

ent Take responsibility for not being clear enough about the purpose of the session and the need for it to be one-on-one.

What to say

ent “It’s great that you’re both so enthusiastic about participating. Can you tell me why you’re interested in participating?”

If the other attendees are trying to not miss out on a training opportunity:

ent “I’m running this study to <get early feedback from users on designs/observe the current experience of the product>. I won’t be providing any kind of training, just watching as <participant> uses the system.”

If dismissing other person/people:

ent “Because this session is designed for one-on-one feedback, unfortunately I can’t have you stay in the room with us. I apologize for any misunderstanding about that. But we can provide you with a beverage if you want to stick around in our reception area and wait for <participant>.”

What not to do or say

ent Don’t let the participant continue the session if her manager is in the room or observing the session, as you do not want any power issues to get in the way of your feedback.

ent If you need to ask the additional attendees to leave, don’t be rude about it. Be firm, but kind. You’re representing your organization to anyone you come in contact with, even if they don’t wind up as a participant. You don’t want to create any grievances with a manager who has a lot of clout in your organization, or with an important customer.

How to avoid

ent This situation often occurs because the participant is unclear about what will be happening during the session. Explain the setup of the session to the participant before she arrives (e.g., during the recruit and in the confirmation letter) and make sure the one-on-one aspect is clear. However, even if you’re very explicit, you can expect to still run into this situation and should plan accordingly.

5.7 Participant thinks the session is a job interview

The participant shows up in a polished suit and gives only positive feedback. At the end of the session, she hands you her resume and asks when she’ll hear back about the job!

Method(s): In-person usability study; In-person interview

Frequency: Rare

Pattern(s) to apply: Take responsibility; Shift the focus; End the session early

What to do

This misunderstanding may happen if you’re doing research for a well-known organization. If someone is unemployed and expecting calls for interviews, she may feasibly misinterpret a user research recruitment call for the other type of recruitment call. This can also happen accidentally if you have a main waiting area for lots of different people—site visitors, job interview candidates, customers, and user research participants. It’s easy to bring in the wrong person and, believe us, things can get quite confusing and interesting after that.

ent Be very clear and explicit in the briefing at the beginning of the session about why the participant is there and what she’ll be doing. Hopefully any misunderstanding will surface at this time.

ent If at any point you suspect that the participant is there for a job interview, try asking a couple of general questions. For example, ask what made her come in today, or if she has participated in a user research session before, and reconfirm the recruiting criteria. If it’s still not clear, you might need to directly ask her if she is there for the session.

ent If the miscommunication becomes clear at the start of the session (e.g., when asking background questions), spare the poor participant further embarrassment and explain the actual purpose of the session. Take responsibility for the miscommunication and offer to cancel the session while still providing her with the full compensation. If you realize that you brought in the wrong person from the waiting area, quickly find the participant!

ent If you’ve offered to end the session but the participant seems legitimately interested in participating, consider letting her continue (unless your actual participant is waiting for you). If she doesn’t fully meet the recruiting criteria, you could see if there is a way to let her stay and feel useful rather than that she wasted the travel time and a perfectly nice interview outfit. You may want to:

ent Run the session anyway if she is able to get through the tasks and questions without needing specialized knowledge or skills. Just make note in your findings how this participant deviated from the recruiting criteria.

ent If the participant can’t continue the tasks without key skills, consider shifting the focus of the session to something that better suits her. For instance, turn a usability study into an interview, or refocus interview questions on topics that you can still make use of and that she understands.

ent If the participant asks you to connect her with someone specific at the organization, or wants you to take her resume and “pass it along,” do what makes you feel comfortable. Don’t do anything out of obligation or to just placate her. If you don’t want to be involved, try to find a gentle way to let her down and point her in a more helpful direction. For example, you may say that you’re a consultant (if that is true), so you don’t really have any connection with the other parts of the organization but know that the corporate website has a section for job applicants.

What to say

If unclear about why the participant is there:

ent Start general: “What made you decide to participate today?”

ent More specific: “You’re <name>, correct? And you’re here for the research study?”

At the beginning of session, if you realize there is a misunderstanding:

ent “<Participant>, I’m really sorry but I think there was a misunderstanding. This is not a job interview but rather a research study. It’s completely our fault for not being clear. If you’d like, I can just give you the promised compensation and we’ll cancel the session.”

ent “I completely understand if you want to leave, but if you’re still interested in the study, I could tell you about it.”

If the participant asks for a connection or interview and you don’t want to be involved:

ent “I’m sorry. I wish I could help but I’m just here to run the session today. If you want, I can provide you the link to the careers web page.”

ent “I am really just involved in user research, so I can’t help you with that. I can walk you to the front desk if you’d like, and they may be able to put you in touch with someone in HR.”

What not to do or say

ent Don’t let the participant leave thinking that the session was a job interview.

How to avoid

ent Provide your recruiter with a script that is explicit about recruiting for a research session. If someone within an organization is doing the recruit for you, provide her with an explanation of the session that can be shared with potential participants to alleviate any confusion. We recommend talking with your recruiter about it as well so she is very clear that the session is for research, not an interview.

ent When you greet the participant, greet her by name and confirm that she is the right person before bringing her into your research space.

5.8 Participant brings a child or pet to the session

The participant arrives with a child (or children!) or animal in tow. The animal isn’t a service animal, but is just a pet. The participant may expect you or your team to provide childcare, or may indicate that the child or pet can “quietly” stay in the room with you.

Method(s): Any in-person method

Frequency: Rare

Pattern(s) to apply: Take responsibility; End the session early

What to do

ent Confirm that the participant is aware of how long the session will last and what you’ll be asking her to do. You can also show her the space that you’ll be in, as that may change her mind about continuing (in which case, you can offer to reschedule her or just offer the compensation for coming in).

ent If the participant has brought a child, ideally she came prepared with items to keep her child occupied during the session. If not, take a deep breath and get creative! Depending on the child’s age, you can volunteer to bring in some paper and markers for her to use, or some other set of (safe!) materials for the child to play with, such as a deck of cards or a tablet.

ent While we haven’t seen participants bring pets other than dogs to usability sessions (unless they’re service animals), we’re sure it can happen. In that case, raise any potential concerns with the pet being there. For example, you need to limit future participants’ exposure to allergens, and make sure the pet’s behavior and “calls to nature” can be controlled without interfering with your session.

ent If the child or pet is being excessively and/or consistently disruptive after an initial period of time, try to be patient and understanding if the participant seems eager to continue anyway. But if there are constant disruptions, you might want to end the session early. Do so delicately, without blaming the child or pet. If you need to use a pretext, you can blame technical difficulties or explain that you made it through all your questions faster than anticipated. Offer again to reschedule for a time when the participant can come on her own. If she is unable to reschedule, we recommend still giving her the full compensation.

What to say

When the participant arrives:

ent “This is meant to be a focused one-on-one session. Will your <child/pet> be okay in the room with us? Our setup may not be comfortable for <child/pet>, so we’re happy to reschedule you for a time when you can come by yourself.”

If concerned about a pet, blame allergies:

ent “I’m sorry, we try to keep this room allergen-free since we have so many people coming through here. Are you able to reschedule for a time when you can come by yourself?”

If child/pet is being excessively disruptive and you need to end early:

ent “It looks like we made it through everything faster than I expected, so you’ll have a bit of extra time in your day. Thank you so much for your feedback, and here is your compensation.”

What not to do or say

ent Do not allow team members to babysit or petsit during the session, even from within the research room. Doing so would potentially leave your organization vulnerable if something happened to the child or pet while they were out of the participant’s view. Also, you don’t want to risk the possibility of your colleague being bitten (by a child or pet!). If the participant asks for this service, let her know that you’re not equipped to do so and offer to reschedule at a more convenient time when she can come in on her own.

How to avoid

Explain the setup of the session to the participant before she arrives (e.g., during the recruit and in the confirmation letter) and make sure the one-on-one aspect is clear.

See also

SURVIVAL STORY: “SHE WAS DESPERATE FOR WORK”

Adrian Howard

I was moderating in-person usability sessions that compared a number of websites designed for people seeking work.

The sessions were run at the client’s site and we had a lot of participants. I helped set up the screening parameters, but the recruitment of the participants was managed by the client.

We ended up organizing a testing production line. While I was moderating one session, the next participant was being prepared in a different room by a coworker. The people who handled the recruitment managed the meet-and-greet as participants arrived.

Toward the end of a long day, a participant walked into the test room under a major misapprehension. A few minutes into the session it became very clear she thought this was an actual job interview.

Oh dear.

Obviously I stopped the session immediately. It would have been unethical to continue. Once I explained, the participant became upset almost to the point of tears. She was desperate for work and had used time and money to attend that she could ill afford to waste. All because she thought it could lead to full-time employment.

We went to a private office to talk in more comfort and I emphasized the situation wasn’t her fault but ours. I immediately reassured her that she would not be out of pocket for the day and gave her time to compose herself while I fetched coffee.

We ended up providing a taxi home along with some additional compensation to cover her costs in attending. She would never have participated if she had understood it was just a usability session.

When we looked to the underlying causes of this failure we discovered three main factors: poor communication, poor screening, and time pressure.

The lack of communication was the biggest issue. The recruiters didn’t raise problems they encountered when people arrived. The person briefing the participants assumed that they had had the process explained to them. I assumed that the participant was ready for the test. Our innocent victim fell between the gaps.

Better communication would have helped us spot the poor screening questions. The language used could be misinterpreted by somebody in a job-seeking context (e.g., the word “interview”). It became obvious our participant had verbally okayed things in the screening and briefing that she hadn’t fully understood in the vain hope that this rather strange job interview process would soon make sense! I should have been more involved with the recruiters and ensured that the screening questions were working well.

Finally, at the end of a long day, everybody involved was tired and not paying as much attention as they should have been. Tight time constraints cause problems. We need to give ourselves the time to pay full attention to every participant we work with. If we hadn’t been running a production line to try and jam as many sessions as possible into a tight schedule, we would have caught this participant much earlier.

SURVIVAL STORY: “HE REFUSES TO LEAVE”

Cliff Anderson

I was at my desk when I got a call from the lab (another usability engineer was running a study that week). It was from our lab assistant, who seemed more than a little excited. “Cliff, there is a man here, and he refuses to leave, and he is threatening us, and we don’t know what to do.” After calming her down, I learned that he had expected to get his $100 compensation in cash, but was balking because we were offering Visa gift cards—something that was new for us and for our regular recruiting agency.

I told her “I’ll be right down,” then hit the stairs to get to the lab. When I got there, I found our assistant, the usability engineer who was running our test, and the user. Things seemed pretty under control, but I did notice that the user seemed a little perturbed.

I went right up to him, introduced myself, shook his hand, and asked him what was the matter. He went into his story, and I did my best job at active and empathetic listening, using all the tricks and tropes that I had learned over the years working with users and doing research.

I could tell he was calming down, but I still hadn’t come up with a way to address his problem. So, I put my hand on my chin, knotted my brows, and it came to me! “Why don’t you give me that gift card, and we can go to the ATM in the lobby downstairs, and I can withdraw your $100?”

He agreed, we withdrew the $100, shook hands, and parted amicably. I went back upstairs shaking my head, but giving myself credit for keeping cool under pressure (in general, one of the best skills a usability engineer can have) and also making sure that our screener (and recruiting agency) made sure to mention this new form of compensation ahead of time.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset