2Managing the Test Team

Keywords: Myers-Briggs Type Indicator (MBTI), RACI matrix, SMART goal methodology, test architect

2.1Introduction

In the previous chapter, we talked about defining what you are trying to accomplish (test objectives), how you will go about it (test strategy), and how you will measure success (test metrics and goals). In this chapter, we turn to an essential ingredient in successful testing: managing the people doing the testing work.

To be successful as a test manager, you must be able to build a test group, develop the skills of the people in that group, and lead it toward the objectives. You may have a single group of individual testers or a collection of two or more test teams, each run by a test lead or test manager who reports to you. The group may consist of people working at a single office, people working at multiple locations, or people working many miles and time zones away. There are many variations in the way test groups are organized, and an expert test manager should be able to manage any of those approaches.

In some organizations—especially larger ones—you might have a human resources (HR) department or personnel office dedicated to supporting staffing issues. Such a department would typically handle employment agreements and benefits packages. It might also provide support for employees and managers in terms of interviewing, conflict mediation, and when necessary, employee layoffs and termination. In some cases, there is not an actual department but just one or two people who handle these tasks.

In this chapter, we’ll assume that such a department does exist and that you will work with them on many of the activities covered here. If there is no department or person responsible for this role in your organization, then in some cases, you’ll need to work with your management colleagues and company legal staff (if applicable). In other cases, you’ll be on your own, though we hope you have some senior managers and some precedents to guide you.

2.2Building the Test Team

It is a management aphorism that smart managers surround themselves with a team that consists of people smarter than they are. The “smarter than they are” part might be an exaggeration, but it is certainly true that only bad managers deliberately hire sycophants and underachievers so that they can enjoy flattery and look good by comparison. Smart managers know that hiring mistakes such as these—hiring the toady, the incompetent, the hostile, the lazy, and the downright evil—will damage the productivity and morale of your group. In some cases, such a mistake can cost the organization a lot of money, as companies have discovered when employees abuse their position to steal, avenge themselves on coworkers, or throw a monkey wrench into the organization itself.¹

So, we can state an ironclad rule for managers here: Hire the right people. While the hiring process should involve multiple people, if you are a test manager hiring a tester or another test manager, the decision ultimately comes down to you. The decision whether to hire someone is among the most important decisions you will make as a manager. Make every effort to do it right.

Hiring the right people involves the following steps:

• Have a good process for evaluating and hiring candidates.
• Clearly define the position for which you’re hiring.
• Understand the skills a person will need to have to fill that position.
• Together with other people, carry out effective interviews that definitively sort candidates from least qualified to best qualified.

If your human resources department offers training or support for hiring and interviewing, you should take advantage of that.²

In many organizations, strict rules exist about how the hiring process must work. These rules are usually there for a good reason—e.g., obeying the law, avoiding a lawsuit for discrimination, etc.—and so you should study them carefully and adhere scrupulously, if they apply. Check with your human resources staff. Ask them to explain the rules and process thoroughly, including whatever documentation is required throughout the process. These rules are perhaps most commonly violated in the résumé selection and interview processes, so be careful.

Building the right test group doesn’t end with hiring the right people. The new people must be assimilated into the group and melded together with the existing staff into an effective team. People also leave teams, in some cases because of contractual reasons. In other unfortunate cases, some of the staff—hopefully people you inherited rather than hired—must be terminated. An expert test manager must be able to handle all of these activities, and we’ll discuss these activities further as this chapter continues.

2.2.1Job Descriptions

A job description, obviously enough, is a document (or web page or wiki page or some other human-readable item) that describes the job and, by implication, the ideal person to do it. It should address the following questions:

• What tasks and responsibilities will the successful candidate have?
• What kind of experience, and how much experience, is required?
• What specific skills will the successful candidate need to have?
• What additional skills would be preferable for a candidate to have?
• What is the salary range for the position, including base salary and bonuses if applicable?
• Are there specific training, education, certification, security clearances, or licenses required?

The best job descriptions often include other relevant details, such as the work hours; the dress code; a polite notice that background checks or drug tests apply (if they do); what perks like telecommuting, a company gym, or other benefits the job brings; whether travel is required and, if so, how frequently; what the career path is for the position; when should the candidate plan to start and, if applicable, whether the position has a predefined end date (e.g., when program funding will end). If you have an HR department, there is probably a template and guidelines for job descriptions, and you should comply with those. You can find examples of good (and bad) job descriptions on Internet recruiting sites such as Dice.com and Monster.com.

If you’re writing your first job description, spend some time to study the art form before slapping together a bad one. You’ll miss out on good applicants and probably have to sort through a lot of bad applicants if your job description is poorly done.

The job description should be concise, clear, and free from jargon because the people soliciting candidates and screening their résumés may be unfamiliar with testing. That said, be sure to be sufficiently specific and precise so that you and the candidates can determine whether a fit exists. The job description should not dissuade the qualified from applying, nor should it encourage the unqualified to apply. This also means that you should be clear about which qualifications are required (a candidate without these qualifications will not even be considered) and which qualifications are desirable (a candidate with these qualifications has an edge in the hiring decision).

Depending on the size of the test group and the way in which it’s organized, you might have specialized test positions, each with its own job description. We’ll return to this topic of team specialization a bit later in this chapter.

Some job descriptions fit into a sequence, as someone moves up their career path. For example, someone might join as an entry-level tester and, after some time, apply for a promotion when a senior position opens up in the testing team. You should consider this as you write your job descriptions. You don’t want to create a situation in which promotion from the inside is difficult or impossible because the senior positions have requirements that junior people would be unlikely to gain during their work in your group. One example would be requiring only a high school degree for entry-level positions but then requiring a college degree in computer science to advance to any testing position beyond the entry level. You might be better off simply requiring the college degree from everyone, even the junior testers, to avoid situations in which people get stuck.

All too many companies take the job description for another position and use it as the basis for the tester job description, but this is a worst practice we recommend you avoid. For example, it’s common for software sales technology companies to use the entry-level programmer job description and glue on some language about testing. IT organizations (which build software used inside their company or agency) often make a similar mistake, which is taking a technical support, business analyst, or senior user job description and gluing language about testing on it. While technology and business domain knowledge is important for testers, a tester is more than a modified programmer, analyst, or user.

Please note that when we’re talking about job descriptions here, we are not referring to the typical job postings you’d find in a newspaper or in a technology magazine. Most of these are too terse, due to the limitations of space, to serve as a good model for a job description. Frequently, those job postings are distilled from the large job description. If someone other than you does that distilling, make sure you review the job posting before it gets posted. If key points are missed (or mangled), you will again face a situation in which you don’t attract candidates that you want and, possibly, do attract candidates that you don’t want.

Once the job description is written, the job posting should be properly distilled and submitted to the appropriate locations. This would include job bulletin boards and Internet recruiting sites, corporate employment sites, newspapers or industry magazines, and other appropriate media. The idea is to attract qualified candidates to start the hiring process.

2.2.2Résumés

Given that you wrote the job description and posting properly—and depending on the local job market—the result will be an incoming stream of résumés or curricula vitae (CVs), most of which will be from qualified candidates. In some organizations, the human resources staff filters these incoming résumés, eliminating people who are clearly unqualified. Of course, there are limits to how well they can do this, given that they are not subject-matter experts. You should expect to receive some number of unqualified candidate résumés because otherwise the human resources staff would be filtering out qualified candidates. If there is no human resources assistance available, you will need to filter the résumés yourself or delegate that task to someone on your staff.

So, how should you review résumés? What do you look for? Here are some ideas:

• Does the candidate have the required skills and experience? You should match these against the job description and eliminate candidates that don’t match. People tend to embellish their skills and experience, so be sure to note, on any résumés you retain, which skills and experience areas you want to check in the interview.
• If applicable, does the candidate have the required or preferred education and certifications, such as a college degree or the ISTQB Certified Tester Foundation Level certification? If they do claim such educational degrees or certifications, ask the candidate to provide proof prior to the interview process. For some reason, lying about these types of qualifications is amazingly common.
• Is the candidate’s salary history and requirements consistent with what you can offer? Asking someone to take a huge pay cut, even if they have been out of work for a while, is a source of eventual disgruntlement.
• Is the résumé organized and easy to read? Remember, testing typically involves a lot of written communication, and the candidate’s written communication skills are on display in their résumé.
• Are there spelling or grammar errors on the résumé? Testing also involves attention to detail. Not rereading the résumé, forgetting to use built-in spell and grammar checkers, or worse yet, ignoring unsolicited warnings from the word processor indicates a level of sloppiness that you might want to avoid associating with your test team. It’s not unusual for the test manager to eliminate candidates with such mistakes on their résumés for exactly this reason.

Remember to retain résumés for those who meet the required credentials. If you have desirable but not required credentials included in the job posting, those should not be used to eliminate candidates. The additional credentials should be used to create an “A-list” of people you want to interview first.

2.2.3Interviewing

A good interview process is much the same whether you are hiring a tester, business analyst, or developer. It has the following attributes:

• Multiple rounds of interviews. A candidate should not be interviewed by a single person, who then decides without any input from others. Instead, the candidate should go through a multistage interview process.
• A mixed interview team. A candidate should be interviewed by people at different levels in the organization and by people outside of testing who are testing stakeholders.
• An understood interview process. The process progresses clearly through a set of steps, each step designed to efficiently remove those candidates who would not be suitable hires, until the final candidates are the best fit for the position.

During the interview, the interviewer should explore whether the candidate displays the appropriate strengths for a testing position:

• Good problem-solving skills. Given a typical problem that the candidate would face daily, can they solve it?
• Critical thinking skills. Given a realistic work scenario, can the candidate recognize the essential elements, analyze them, select useful paths forward, reject dead ends, and be able to explain the entire path of thinking that lead to their conclusions?
• Good written and verbal communication skills. As mentioned, the ability to explain facts, observations, and directions in written and verbal form is essential for testers. Can the candidate be easily understood in the interview? Can they write acceptable defect reports and test cases?
• A strong ability to work within a team. Hiring someone who is otherwise perfectly qualified but disrupts morale and cohesion within your test group is a disastrous mistake. Does the candidate seem like a person who will enjoy working with your existing team, and will your existing team enjoy working with the candidate?
• A curious nature. Does the candidate show signs that they would investigate anomalous situations and use good judgment in how far to take those investigations?
• Technical and business domain knowledge. Does the candidate know enough about the underlying technology behind the system under test, given the type of testing they will do? Does the candidate know enough about the business problem solved by the system under test, given the type of testing they will do?
• The appropriate testing skills. Does the candidate have appropriate test design skills, test automation skills, test management skills, and so on, given the position they are interviewing for?
• The experience and background needed for the position. Beyond these specific qualifications, are there other qualifications identified in the job description that the candidate must have, and does the candidate have them?

The exploration of the candidate’s strengths and weakness through the interview consists, naturally enough, of asking the candidate questions, considering their responses, and perhaps pursuing a line of questioning related to their responses before moving on to the next topic. Also, remember that the candidate is also interviewing you and your organization, so make certain to show your best self and be cordial, respectful, and prepared.

The ISTQB Expert Test Manager syllabus includes a number of very useful questions to keep in mind as you explore these strengths and weaknesses. The questions will help you assess the interviewees’ statements during the interview. They fall into various categories, and various interview questions can address these categories. In Table 2-1, we have taken this material, categorized and reformatted it, and provided sample interview questions.

Table 2-1Considerations for Interviewing Testers

Table 2-1 is not an exhaustive list, but it should help you start creating interview questions as you plan your interview process.³

Your planning should also include the use of different types of interviews. In Table 2-2, we’ve organized the types listed in the syllabus and the pros, cons, and our own opinions for each type.

Table 2-2Interview Types

While interviews are useful, it’s always good to see someone in action, doing what they will actually do in their daily work. This is the idea behind audition interviews, which are called assessments in the Expert Test Manager syllabus. These could take a number of forms, depending on the position for which the candidate has applied:

• Junior or senior tester: Provide the candidate with a typical, existing test, written at the same level of detail as the tests the candidate will execute if hired. Have the candidate run the test against the actual system, record their results, and log any anomalies noted. You can make this more challenging by picking a test with subtle anomalies.
• Junior or senior tester: Provide the candidate with one or more test conditions. Have the candidate create tests that adequately cover the test conditions.
• Senior tester: Provide the candidate with an actual requirements specification or user story from a current or previous project. Have them identify test conditions. Depending on the requirement, have them create a flow chart, state transition diagram, or decision table.
• Senior tester, test lead, or test manager: Have the candidate present a topic to the test team (and perhaps to non-testers as well) or discuss some topic with that group. For example, provide the candidate with a typical or challenging test status report, and ask them to role-play presenting that information as the test lead to the project team.

These are just some examples; you can apply this technique to any pertinent and demonstrable skills. After the audition is complete, the work product produced by the candidate, and the way the work product was produced, should be assessed by one or more members of the test team, perhaps in consultation with other staff members who are qualified to judge such work. This should often include a discussion with the candidate to ask them why they decided to approach the work the way they did. Just as we have not encountered any fairer way to measure certain types of knowledge than exams, audition interviews are the fairest way to evaluate certain types of skills.

2.2.4Selecting the People to Hire

Once you have finished interviewing the candidates, you can make your decision on whom to hire (and whom not to hire). This decision involves evaluating the results of the interviews, of course, as well as any other organizational hiring processes:

• Reference checks: This can involve speaking with (or at least emailing) past employers, managers, and colleagues. You have to be careful about these reference checks because you can get into trouble by asking the wrong things. Your HR department should have some tips on how to check references and might even require you to allow them to carry out all reference checks.
• Background checks: In cases where past criminal history matters, you might need to do a background check. Again, be careful; there could be legal consequences if background checks are conducted improperly. As with reference checks, the HR department might be able or required to help.
• Credit checks: These are similar to background checks and applicable in certain situations. Consult with HR before carrying these out.
• Verification of legal work status: This can take the form of getting standard identification information from the candidate and possibly making copies of these documents. Consult with HR.
• Verification of, or request for, security clearance: Where applicable, you or the HR department might need to verify someone’s security clearance or start the process of applying for a security clearance. Obviously, this latter action would apply only to candidates that you were confident would qualify and had already decided to hire (pending their obtaining a security clearance).

Of course, you might decide to hire none of the candidates. If none of the candidates met the minimum qualifications, then this can be a smarter decision than hiring someone who is unqualified. You can consider using a contractor to fill the need or, if applicable, a testing service provider (see Chapter 3). However, you should remember—during the interviewing process and when selecting the new hires—that your goal is to build and maintain a cohesive and effective team. Are you setting the minimum qualifications too high? Are you looking for the perfect person when what you need are people who will fit into your team in terms of personality and who have the potential to grow into strong contributors? Even if individuals are not perfect, you might have the perfect test team for the job. Different types and levels of experience and knowledge, along with varied viewpoints, can create a team whose whole is greater than the sum of its parts.

2.2.5Bringing People Into the Team

Even when a new hire is amazingly well qualified, there are sure to be parts of the job he/she doesn’t already know. Processes, procedures, tools, and methodologies used to create and manage tests, defect reports, and other work products tend to vary from one organization to another. While immediate immersion is one approach to ramping up new hires (also called “sink or swim”), it’s certainly not the best practice.

The best practice is for the test manager to create and deliver some sort of standard training for new hires. At a minimum, this training should cover the following topics:

• Processes and procedures for the test group as well as the larger organization (with the latter elements often provided by the human resources department)
• Tools that the new hire will use within the test department and on the project
• Examples of and standards for work products the new hire will create, such as tests and defect reports

In addition to explanation of these topics, the training should include hands-on demonstrations, discussions with existing test team members, and exercises on selected topics. If multiple people are hired at the same time, they can attend this training together as team-building as well as skills-building exercises.

Consider assigning the test team the task of creating and delivering the new-hire training program. This will promote uniformity in practice and approach across the test team, allow the test team members to share their knowledge with each other, reduce the possibility of deliberate or inadvertent information hoarding by test team members, and motivate the test team by making them key players in defining and promulgating the “way things are done around here.” If your organization tends to hire a lot of testers, it might make sense to create (or to have created) an e-learning course that can be used (often supplemented by some live training elements).

The new-hire training need not—indeed, should not—occur all at once. A two-, three-, or even four-day training course that starts the day the new hires arrive is likely to leave them feeling overwhelmed. Not only can this be demotivating (e.g., the confused new hires start to doubt their ability to master all the necessary skills), it is generally ineffective because people start to have minimal retention once they have hit a limit. Instead, break the training into consistent modules, with each module, say, three or fewer hours long. Not only will this make it easier for experienced test team members to deliver the training, it will also result in new hires acquiring new skills and immediately applying those skills after the training. Immediate application of skills aids retention and training effectiveness.

For example, you might use a schedule such as the following, where each training session is a half-day or less:

• First week: HR trains the new hires on organization policies and procedures on the first day. That same day, the new hires are given their workstations and set up for access. The second day they are trained on the basic processes and procedures for the test group, such as the organization of the project information repositories. They are assigned some tasks, such as reviewing project documentation, that are within their current capabilities.
• Second week: The new hires are trained on more advanced processes, such as defect reporting, defect management, test creation, and test management, possibly over a sequence of days. Again, the new hires are assigned tasks that are within their current capabilities.
• Third week: The new hires are trained in any additional tools they’ll use within the test department and on the project. For example, if the new hires will be involved in test automation, the existing test automation team might give the new hires a walk-through of the existing test scripts, the standards about how the scripts are created and maintained, and so forth.

While the test manager might not deliver all of these training sessions, they should make sure new hires do indeed attend all appropriate training events. For particularly critical training elements, it is a best practice to develop an exam to check the new hires’ knowledge after the training, or alternatively, after returning to work for a few days.

It’s also a good practice to assign a mentor to each new employee, someone who can answer the employee’s questions as needed for the first few months. For example, a tester might ask a mentor about the right way to handle a defect report when the confirmation test fails. Without a mentor, new employees must either ask their manager (who might not be available when needed), figure out the answer on their own (which might result in the wrong answer), or simply be stuck (which is both demotivating for the individual and inefficient for the team).

So far, we’ve talked about forging the team from people you’ve hired. Of course, you might not always have the luxury of hiring your entire team. You might take over a test management position where the team already exists. Perhaps you were promoted from within the team to replace a departing or promoted test manager. Perhaps you are integrating two or more existing test teams, such as, for example, after a merger or acquisition. Any number of situations can result in your managing people who were hired by someone else. Regardless of how the people on your team came to be on your team, you must make sure it’s effective. This topic will be addressed further in later sections of this chapter.

2.2.6Termination of Employment

Sooner or later in your tenure as a manager, you’ll have to terminate someone’s employment. Even when the person being terminated deserves the termination, you are unlikely—unless you have no empathy—to find doing so a pleasant experience. For this reason, some managers go to great lengths to avoid the task. This can result in inappropriate people remaining on the team or inappropriate approaches to removal being taken.

What are the situations in which trimming the test team is necessary? There are three typical triggers for terminating an employee:

• Poor individual performance. A person might display inappropriate attitudes or behaviors at work. Alternatively, the person might be unable to carry out the essential roles of their job. Unless the behavior is so egregious as to require immediate termination (e.g., sexual harassment, threatening coworkers, actual violence in the workplace, etc.), you should avoid a knee-jerk reaction in these situations. There was some reason the person was hired, and you should try to help the person get back on track. If, after attempts at remediation on your part, the problems cannot be resolved, then termination is the best course of action. You are doing yourself, the rest of the test team, and probably even the poor performer a disservice by avoiding termination. Of course, there is a possibility in this situation that a bad hiring decision occurred; you should see if there are lessons you can learn.
• A change in business focus or process. In some cases, businesses find that they must redefine themselves to succeed or even just survive. For example, the adoption of cloud computing proved to be such an event for one of our enterprise software clients, and it was quite a successful move for the organization. However, this did change the skills needed in its test teams. Similarly, it is often the case that teams adopting Agile methodologies require a higher level of technical skill in their testers. If it’s not possible to retrain the testers affected by these changes, you might find yourself forced to pick winners and losers and terminate some of your team members.
• Reduction in force (or layoff). Businesses that lose their way or that fail to keep up with evolving consumer and business tastes—and these trends can move lightning fast in high tech—might impose across-the-board reductions in force or layoffs. These layoffs can fall disproportionately on test teams. I’ve seen and heard of situations where 20 percent reductions in programmers were coupled with 50 percent or more reductions in test staff. Unlike the “change in focus” situation mentioned earlier, there will be no way to soften the blow by trying to find other homes in the organization for displaced testers. You must pick winners and losers, and there might even be more losers than winners.

In all of these situations, HR may have defined policies and procedures. You should certainly comply with these strictly. Failure to do so could trigger a wrongful termination lawsuit, with you as the star witness. It could also constitute a violation of applicable union regulations, state or national labor laws, or employment agreements.

In the “change in focus” or “reduction in force” situations, you must pick the testers you will keep in a clear-eyed fashion. You should consider the kinds of skills you’ll need for the team in the future rather than favoring your personal friends on the test team. Yes, we realize that this advice is much easier to give than to take. However, in both of these situations, the entire organization—and especially the test group—is in a perilous, high-risk position. You will need the strongest possible test group to increase your own likelihood, and your organization’s likelihood, of success.

In the case of termination for poor performance, company policies and plain old fairness often require that you document the problems before starting a termination procedure. If you, as a manager, find someone’s performance unacceptable, you should talk to the human resources department. You need to understand the proper steps to take to try to remediate the problems (sometimes called performance improvement plans) and the plan for terminating someone should they not improve.

When someone must be terminated, you must consider skills transfer, knowledge transfer, and intellectual property retention. Outgoing testers might know undocumented configuration steps, passwords, data locations, and more. This knowledge will be needed by the person taking over for the terminated employee. If you discover that you must bring back a terminated employee as a consultant because they have unique knowledge and skills, they might not be gentle in their rate negotiations if they feel ill-used by the termination. Of course, the about-to-be-terminated employee might not feel motivated to complete the knowledge transfer, especially given that a failure of the knowledge transfer may enable the very consulting engagement you would prefer to avoid. Therefore, the smart thing is to use cross-training, throughout the employment period, for everyone. If no single person knows something that no one else knows, you have minimized the risk of someone leaving with critical, unique, irreplaceable knowledge.

2.2.7Ending Contractual Relationships

Not everyone who works on a project or team is an employee. A number of organizations use individual contractors and outsource testing service providers to provide temporary help on projects. This can be driven by a need to fill skills gaps or to quickly increase the pool of available, qualified resources. The contractual options exist for testing as well as for other parties in the software development lifecycle, as will be discussed in the next chapter.

The best practice in the use of individual or outsource contractors is to have a contract with a predefined end date. This allows clear visibility into when resources and perhaps specialized skills will leave the organization. The test manager should make sure that they have planned for that moment, with resources identified to take over recurring tasks (if any) once owned by the contractors and a knowledge transfer plan in place for essential skills (if any) currently held by the contractors.

As with individual employees, it’s also possible for contractors or outsource test firms to fail to meet expectations. In this case, you might need to terminate the contract early. You’ll want to be sure, when the contracts are initially signed, that you can do so. Some organizations are good at winning contracts, not so good at fulfilling those contracts, and yet very good at avoiding being held accountable for their failures. As is the case with terminating an employee, you should plan to work with the human resources department when terminating a contractor or outsource services group. You might also have to involve the legal and procurement departments. If someone other than you made the engagement decision with the contractor or service provider, you’ll have to work especially hard to document the reasons for the termination because this termination might reflect badly on the business acumen of the individual who engaged the failed contractor or service provider.

2.3Developing the Test Team

An old management cliché says that managers are only as good as their teams. Like many clichés, it’s founded on truth. We may seem to be contradicting something we said in a previous section, that hiring managers should not look for a perfect candidate but rather a capable one who can grow. The key to resolving that apparent contradiction is in the last three words, “who can grow.” You might not be able to hire the perfect test team, and you probably won’t inherit a perfect test team either, but you can take a group of capable people with a thirst for new knowledge and turn those people into one remarkably good test team.

So, how do you accomplish this amazing transformation? There are three essential elements in developing a great team:

• Giving each member of the team an opportunity to acquire new skills in a way that complements the strengths and offsets the weakness of the existing skill set (e.g., cross-training)
• Arranging work assignments so that each member of the team can use those new skills and their existing skills to carry out their duties
• Ensuring that you, as the manager, support the development of each person on the team and of the team as a whole, inspire their best work, and never get in the way

While each of these three elements is important, perhaps the most challenging one for managers is the last. Because it’s so challenging, we’ll spend an entire section on effective leadership later in this chapter. In the following sections, we will focus on the first two elements.⁵ Let’s see what we need to do to develop a strong test team.

2.3.1Developing Individuals

Just as the strength of a manager derives from his/her team, the strength of the team derives from the individuals on it. Other than in silly movies, collections of misfits and losers typically do not accomplish amazing things. So, to the extent that your team is not perfect to begin with—and it won’t be—you need to develop the people on the team. This is not only good for your organization and test team, it’s also good for their individual careers.

However, you need to ensure that the individual development complements the needs of the test team as a whole. This complementary nature of skills development typically means that you as a manager must identify and then fill skills gaps within the team. This allows you to do well by your team and the organization while at the same time doing good for the individuals on the team.

Just as throwing a box of toothpicks on the floor is unlikely to result in a miniature model log cabin, you should not expect that ad hoc and reactionary approaches to skills development will result in a balanced team with strong individuals. As a manager, you should work with your team to create a development plan. (Indeed, your company might require you to do so.) This plan should identify current skills gaps and a road map for resolving them.

In terms of identification of skills gaps, the best practice in this area is to use a task analysis to identify the important tasks done by your team and the skills required for those tasks. The result of that analysis is then used to perform a skills inventory for your existing team. This process will clearly identify where your team is strong and where skills gaps exist. You can then use an awareness of those gaps to put together training plans for the people on your team. These training plans collectively make up the road map for filling the gaps.

What types of activities should you include in this road map? Here are some essential tasks:

• Send people to training courses that are aligned with the skills gaps in your organization (more on this topic in the section “Skills Development, Training Opportunities, and Mentoring” later in this chapter).
• Have individuals or groups use books, webinars, and other resources to self-study specific, relevant topics.
• As part of training or self-study, recommend that people work toward a testing certification or other certification that relates to the skills gaps in your organization.
• Encourage people to start or continue work toward a college degree (at least to the extent that this degree makes sense given the skills gaps in your team).
• As individuals gain new skills, assign them to take on new roles or tasks that require those skills.

As mentioned, individual development is good for the individual but also essential for the test team. Therefore, as a test manager, you should not just support individuals in their skills growth but actually require it. A common way to do that is to tie successful completion of skills developmental tasks to each person’s periodic objectives or goals (see the next section).

Of course, many of these training activities require money. Part of your development plan should include a budget for the training your team needs. Being able to show a clear connection between the work your team does (via the task analysis) and the skills needed and currently lacking (via the skills inventory) will help you justify this budget to management. We will address the process for performing a task analysis and creating a skills inventory later in this chapter in the section “Defining Clear Roles and Responsibilities.”

2.3.2Setting Goals and Objectives

One key role in managing people is guiding them to achieve their best possible work. One element of this role involves defining goals and objectives for each person on the team and then periodically evaluating their performance toward those goals and objectives. This is typically done as a part of regular performance reviews, which often occur annually.

In some organizations, companies tie employee achievement of their goals and objectives to their salary increases, bonuses, or other incentives. This is sometimes referred to as management by objectives (MBOs). In any case, but particularly when financial motivations are involved, it’s critically important that the goals and objectives be defined in a way that is fair to each employee and to the company as a whole. While this approach can be a rational and impartial way to evaluate employees, it can prove quite problematic when done poorly (or with ill intent).⁶

One approach to defining these goals and objectives is to use the SMART goal methodology. This approach says that each goal and objective should have five properties:

• Specific. Exactly what is expected, what will be true or accomplished if the goal is met? A general goal for a tester might be “Increase testing skills,” while a specific goal might be, “Take an ISTQB Foundation training course and apply techniques learned in the course to regular project work.”
• Measurable. How will you measure progress toward the goals? To make our specific goal measurable, we might rephrase it to say, “Take an ISTQB Foundation training course, pass the Foundation exam with a score of 95 percent or better, and apply at least ten techniques learned in the course to regular project work.”
• Attainable. Is the goal something that can be achieved and also something that goes beyond simply typical performance? To make our goal attainable, the manager must assure that there is budget available for the training and the certification exam and that the tester will be able to attend the training (through adjustments to their workload, for example).
• Realistic. Given all of the constraints and realities extant in the projects (both underway and coming in the future) and in the organization as a whole, would achieving this goal be something that would make sense for the individual, the test group, and the organization? To make our goal more realistic, suppose a review of the syllabus and a discussion with people previously taking the exam leads us to conclude that few people score 95 percent or better and that there are really six techniques that apply for most testers. We might rephrase the goal to read, “Take an ISTQB Foundation training course, pass the Foundation exam with a score of 85 percent or better, and apply six core techniques learned in the course (risk-based testing, equivalence partitioning, boundary value analysis, decision tables, state-based testing, and use case testing) to regular project work.”
• Timely. When should the goal be completed, and if milestones along the way exist, when should those milestones be reached? To make our specific goal timely, we might rephrase it to say, “Take an ISTQB Foundation training course and pass the Foundation exam with a score of 85 percent or better in the next three months. Apply six core techniques learned in the course (risk-based testing, equivalence partitioning, boundary value analysis, decision tables, state-based testing, and use case testing) to regular project work starting with the project immediately following the training.”

By using this approach, the managers can establish goals and objectives that not only provide fair standards for performance reviews but also provide employees with a meaningful framework for improving themselves.

Unfortunately, managers (and not just test managers) tend to make a number of mistakes when setting goals. One common mistake is to set goals that are unachievable, either because the goal is beyond the current capabilities of the employee or because organizational and project realities will pose insurmountable obstacles. For example, a manager sets a goal of ISTQB Foundation certification for all testers on their team, but does not have a sufficient training budget or the possibility of getting people away from work long enough to prepare for the exam. Another common mistake is to set goals that are so simple that achieving them really is no achievement at all and should rather be seen as a basic condition of remaining employed. For example, a manager sets a goal of attending all mandatory meetings. Yet another common mistake is to set goals that are so vague as to be meaningless. For example, a manager sets a goal like the “Increase testing skills” example given earlier.

It is not the case that managers make mistakes in setting goals because they are stupid or lazy or uncaring. These mistakes occur because setting good goals is hard. Sometimes tasks are hard to measure, or using easier tasks is not actually a fair way to measure employees. For example, setting a goal for testers of completing all their assigned test cases within the allotted time might not be a realistic goal if the time assigned to the tests is too short or poor-quality software is delivered for testing. Neither of these factors is within the control of the testers. When a factor that influences the achievement of a goal is not within the control of the person being evaluated, that goal is unfair and the person being evaluated will perceive it as unfair.⁷

If you look closely at many of the metrics we use in testing—metrics such as the number of defects found, the percentage of test cases passed, the number of defects found in production, and so forth—you will see that they are not controlled by the individual tester. That’s because most of those metrics are concerned with measuring the project, product, or process, as discussed in Chapter 1, Chapter 6, and Chapter 8. What we are talking about here are people metrics, and that’s a different matter. If you use process, project, and product metrics as people metrics, you must take care that doing so does not encourage bad behavior or penalize good behavior.

For example, incentivizing testers based on the number of defects they report—a common enough test manager mistake—generally leads to a higher number of low-quality defect reports as well as testers being rewarded for disproportionately reporting inconsequential, low-priority failures, which are often easier to isolate and report than complex, high-priority failures. This incentive will result in overall process metrics trending downward for the test team and in behaviors that are disconnected with desirable project and product outcomes. Thus we can say that goals and metrics must be properly aligned in terms of process, project, product, and people.

The best practice is for the manager to work with each employee, as part of their regular performance review, to set their goals and objectives for the upcoming evaluation period. These should be individual goals, with measurable deliverables produced by the employee. Note that individual goals are distinct from group goals, such as an initiative to improve defect detection effectiveness for the test team.

This is not to say that you shouldn’t establish group goals. These goals could apply to the entire test team or to specific people within the test team. In addition to setting individual goals for test team members, you can establish group goals for the test team at large or for specific roles within the test team. Suppose that three people support test automation within a larger test group. You might set the following group goal for them: “Using best practices in test tool selection, choose an open-source tool that supports GUI test automation for browser-based applications within the next three months. Acquire that tool and install it in the test environment within one month after selection. Identify a pilot project, train the test staff on that project in the use of the tool, and complete the pilot project within six months. Deliver a cost-benefit analysis on the pilot within one month of the completion of the pilot.”

One last point about defining goals. In addition to using the SMART methodology, and avoiding goals that lead to incorrect behaviors (through misalignment of process, project, product, and people goals), you want alignment of the people goals vertically. To continue the example given earlier, if we have set a goal for a team to introduce automation of testing, that should support a goal for the test director; for example, the test director might have as a goal reducing the cost of regression testing by 20 percent over the next year without increasing regression risk.

With the goals and objectives defined, the manager should not forget about an employee’s progress until the next performance review. Good managers will often meet regularly, in some cases weekly, with their direct reports. In addition to discussing the employee’s current workload and status in those meetings, the wise manager will spend time discussing progress toward the goals and objectives. That way, the manager can praise and encourage progress made and, if necessary, make adjustments to ensure that progress is made.

For example, suppose the director of a testing group has defined a goal of attaining ISTQB Advanced Test Manager certification for the four test managers who report to them. Due to heavy project loads, sending all four managers to external training is impractical, as is having in-house training in any given week. So, the director has decided to use a blended e-learning approach to training the test managers, with two months of e-learning access and four two-hour facilitation sessions with a trainer via a webinar-style interface.

In this example, the test director would be prudent to ask the test managers, in their weekly one-on-one meetings, how things are going with the training. Suppose that three of the test managers report making great progress. The director would encourage the managers to continue their hard work and focus not only on passing the exam, but also on learning skills they can apply to their projects. Suppose that one of the test managers says, “You know, this current program has some real challenges, and I find I have to spend a lot of time with the testers on each project team to keep them going. That’s preventing me from spending time on the e-learning course or attending the facilitation sessions.” This would be the time for the test director to brainstorm ways to redirect some of the manager’s time and energy toward the course, perhaps by identifying and resolving some of the problems facing the program.

2.3.3Defining Clear Roles and Responsibilities

This discussion of goals and objectives for individuals is strongly related to and parallels the discussion of mission, objectives, and goals for the test group as a whole, covered in the first part of Chapter 1. The goals and objectives for individuals must be aligned with the mission, objectives, and goals for the test group as a whole, as mentioned earlier. The individuals must understand the mission, objectives, and goals for the test group so that they consistently act in a way that supports them.

Furthermore, the objectives and goals for individuals must be aligned with their individual roles and responsibilities. As with the derivation of metrics and goals for the test group from the mission and objectives, the roles and responsibilities of teams and individuals within the test group must clearly relate to the team and individual goals and objectives. You must clearly define the roles and responsibilities of teams and individuals within the test group and ensure that these roles and responsibilities are clearly understood by everyone in the test group. Otherwise, the manager cannot properly or fairly evaluate individual or team performance.

There are two models for organizing the roles and responsibilities for individual contributors within a test group. One model is the generalized test group, where all testers have a consistent skill set. Any testing task that comes along can be done by almost any of the testers. The opposite model is the specialized test group, where small teams of two or more testers are specialized in certain skills. As testing tasks come along that require a certain skill, testers are assigned based on their particular specialty.

The generalized model favors efficiency. At least in the idealized case, because any tester can do any task that needs to be done, test managers can maintain almost perfectly even utilization of each human resource. However, effectiveness can be reduced because the testers will not tend to be experts in any particular area.⁸ In addition, there will be certain types of tasks—e.g., test automation—that cannot be handled by a purely generalized team because these tasks require a high degree of specialization and experience.

The specialized model favors effectiveness. The tester assigned a task is exactly the right person for that task, based on their skills and experience. However, efficiency can be reduced because different skill sets might be required at different points of time, resulting in some people being overallocated while at the same time other people are underallocated. If you take the specialized model to extremes, where a single tester has unique skills, this creates the problem mentioned earlier in the context of unavailability or termination and also can create bottlenecks with tasks that require unique skills.

Specialization is also a function of the size of the test group. It’s very difficult for a small test group to be highly specialized because there are too few people to dedicate to a small set of capabilities. In these situations, the test group may need to use outsourcing or insourcing—which we’ll discuss in the next chapter—to fill roles that require highly developed skills in particular areas.

Assuming that you do want to have some degree of specialization in your test group, what options exist? This depends on the skills you need, but here are some examples:

• Black box tester: Someone with business domain expertise as well as expertise in sophisticated black box test design techniques. This person follows the test analyst career path in the ISTQB program.
• White box tester: Someone focused on the technical details of system implementation as well as having expertise in sophisticated white box test design techniques. This person follows the technical test analyst career path in the ISTQB program.
• Performance test engineer: Someone focused on technical details of the system, but especially the design of the system. Not only are sophisticated white box and black box test design techniques required, but some understanding of modeling of system behaviors is necessary. This person also follows the technical test analyst career path in the ISTQB program, but with additional expertise in performance.
• Test environment and data administrator: Someone with the skills required to implement and maintain production environments and databases but applying those skills to the test environment. This person’s testing skills might be limited, or they might have the ability to run certain tests when demand requires.
• Tools developer: Someone with a strong background in programming and in test automation. Depending on the sophistication of the tools being developed, this person might follow the technical test analyst career path in the ISTQB program.
• Test lead and test manager: Someone focused on organizing and leading the individual testers. This person follows the test manager career path in the ISTQB program.
• Test architect: Someone with skills similar to those of a tools developer but with a strong understanding of testing on both the test analyst and technical test analyst side.
• Test automation specialist: Someone with a strong background in programming and in test automation. This person differs from the tools developer in that their focus is on using existing tools rather creating new tools. This person also follows the technical test analyst career path in the ISTQB program and should have at least five years of test automation experience.
• Security test engineer: Someone with a broad technical background who understands programming, system administration, and database administration as well as sophisticated black box and white box test design techniques. This person also follows the technical test analyst career path in the ISTQB program.

Now, as with any models, these are simplifications of reality. Furthermore, these models exist on the extreme ends of a spectrum. Most test groups tend to use a mixture of these models, with some elements of generalization (perhaps limited to some members of the test group) and some elements of specialization (again, perhaps limited to some members of the test group). For example, a test group may choose to have specialized business domain experts that create tests and execute experience-based tests, specialized technical experts that create test automation frameworks, specialized test leads and managers that organize and lead the testing efforts, and generalized testers who carry out all the other tasks that do not require specialized expertise. In addition, there may be some mobility in the sense that the generalized testers over time advance into specialized roles.

Whether you follow a specialized or generalized skills model, you will need a way to track and manage the skills of your team. One way to do so is via what is called a skills inventory. To create a skills inventory, you first perform a task analysis. A task analysis involves reviewing the tasks undertaken by your testers as part of their projects and their other regular operations. In this review, you analyze each task to identify the skills required to perform it competently.

These skills can typically be grouped into four categories: general professional qualifications, technical skills, business domain skills, and testing skills. If you haven’t identified skills in one or more of these categories, you probably haven’t analyzed the tasks thoroughly enough.

For example, suppose your test team is responsible for testing a web-based application that allows people to find, contact, and review businesses and attractions in their current location. It runs on all browsers and operating systems and supports mobile devices as well. Here are some of the skills you might need for testers, grouped by category:

• General professional qualifications: Oral and written communication, education (e.g., college degree), and testing certification
• Technical skills: Browsers, operating systems, mobile devices, programming, and Internet
• Business domain skills: Cartography and guidebook creation
• Testing skills: Black box, white box, and experience-based testing; test planning and estimation; functional test automation; performance and reliability test automation; and test status reporting

Once the skills are identified, you can organize your skills inventory. One method is to list the skills, grouped by category, in the left side of a spreadsheet. Then list the test team members across the top of the spreadsheet. At this point, evaluate each test team member’s level of skill for each identified skill. You can use a simple scale for this evaluation:

1. 0 (No Knowledge): The person cannot perform any tasks that require this skill.
2. 1 (Some Knowledge): Given someone available to answer questions, this person can perform simple to moderate tasks that require this skill.
3. 2 (Knowledgeable): This person can perform simple to moderate tasks that require this skill, can answer simple to moderate questions about this skill, and can perform complex tasks that require this skill, given someone available to answer questions.
4. 3 (Expert Knowledge): This person can perform all tasks that require this skill and can answer simple to complex questions about this skill.

It’s important to ensure objective and meaningful evaluations of these skill levels across the team.

With the skills inventory in place, you can identify areas of individual and team weakness based on these ratings. These areas of weakness should be addressed through training, mentoring, and other forms of skills development.

You can also use this skills inventory to create job descriptions. The skills required for a position should be consistent with the skills identified for that position in your inventory. You should also use recruitment as an opportunity to fill skills gaps in your team, by hiring people who are strongest where your current team is weakest.

Finally, you should also consider the skills inventory when making task assignments. Keep in mind that sometimes you will want to assign people to tasks for which they are not currently quite qualified. When you do, assign them a mentor. They can consult the mentor if they have questions, and the mentor can review their results and deliverables. This way, cross-training can be accomplished as part of people’s regular work.⁹

Another useful tool for task assignments and to identify roles and responsibilities is a RACI matrix. RACI matrices define four types of participation for any given task:

• Responsible: Doing the work involved in the task, and creating any work products the task should deliver. Typically, only one role is assigned responsibility, though it’s important to note that a role is not the same as a single individual, as explained later. A variant of the RACI matrix, the RASCI matrix, adds a supporting role, which involves assisting the responsible individual(s).
• Accountable: Being answerable (to the various project, product, or program stakeholders) for the proper, thorough completion of the task and its associated work products. There should be a single individual accountable for a task, but it’s not unusual to see organizations breaking this rule. In some cases, the accountable person approves the work done by the responsible person(s). In other cases, the accountable and responsible roles are the same.
• Consulted: This role involves advising, discussing, and reviewing solution options with the people in responsible and accountable roles. The communication is inherently two-way. Multiple roles can be assigned consultative participation.
• Informed: This role involves receiving information about the progress of the task from the accountable and responsible roles. This information may come periodically or perhaps only at the completion of the task. The communication is primarily one-way, though typically a person in an informed role could ask questions about information or even initiate a flow of information on request, such as, for example, by requesting a status update.

Other than the possibility of the same role being responsible and accountable, the best practice is for each role to have a single participation type for a given task. If a role has multiple participation types, that can indicate some confusion about who is doing what on the task.

A typical RACI (or RASCI) matrix has a list of tasks or deliverables on the left-hand side, with roles shown across the top of the columns in the matrix. The assigned participation type for any given task/role deliverable—i.e., R, A, C, I, and possibly S—is shown in the appropriate cell of the table.

As alluded to earlier, it’s important to distinguish between roles and individuals. A role arises from some group of activities, and multiple people can play a given role. For example, you can have a role of automated test engineer, which involves creating automated testing frameworks using open source and commercial off-the-shelf (COTS) testing tools. The role could be played by a developer who is building an automated unit testing and continuous integration harness. The role could also be played by a tester who is building a framework for functional regression tests.

Not only can multiple people play one role, but one person can play multiple roles. For example, the developer mentioned in the preceding paragraph is playing a programmer role when writing code and is playing a system analyst role when designing a system’s architecture. The tester mentioned is playing a test lead role when guiding a group of automated test engineers in the creation of a new, large, and complex framework.

Not only does the RACI matrix help test managers assign tasks, but for testers, it should clarify their specific roles and responsibilities. The RACI matrix may also be circulated outside of the test team, to help test stakeholders understand who does what within the team.

2.3.4Individual Personalities and Roles Within Teams

As a test manager, you must understand how each tester can best contribute to the test team. In part this is a matter of skills and qualifications and can be determined from the skills inventory discussed earlier. Even if the goal is to have a generalized team organization, you will find that there is no factory in the world that produces fully substitutable, equivalent testers; variations in skills and experience will create differences in testers that will affect what roles they can play.

The variations are not only in skills and experience. As you have noticed already, people have different personalities. In personal interactions, including those between managers and their testers, those personality traits have a strong influence. If managers do not skillfully handle their interactions with their testers, that can lead to a number of negative outcomes. One such outcome is a personal misunderstanding on one or both sides, where a difference of opinion or interpretation becomes a personal or relationship problem. Another such outcome is a loss of trust by the tester, resulting in the tester withholding information or providing information that is incorrect or false (perhaps because the tester fears the consequences of telling the truth). There are a number of psychological aspects of management, especially test management, that a good test manager should understand.¹⁰

One way to gain insight to these personality differences is personality profiling. Personality profiling is not something that an amateur should undertake. The Myers-Briggs Type Indicator (MBTI) is one of the most popular profiling tests used in business. MBTI tests can place people into 16 compound personality types, based on four classifications:

Extrovert (E) or Introvert (I): A person who focuses on the world around them, and especially the people in it, is extroverted. A person who is more inward focused is introverted.

Sensing (S) or Intuition (N): A person who focuses on facts and data is sensing. A person who is likely to interpret and draw conclusions from their observations is intuitive.

Thinking (T) or Feeling (F): A person who makes decisions on facts, logic, consistency, and coherence is thinking. A person who makes decisions based on people and is more comfortable to making exceptions to rules based on special circumstances is feeling.

Judging (J) or Perceiving (P): A person who wants to have clear plans and decisions, which are then followed closely with variation occurring only when clearly necessary, is judging. A person who prefers to take in new information and evaluate new options, perhaps in a continuous fashion, is perceiving.

These four classifications are combined to give the compound personality type. For example, someone on your team might be an ENFP, and that person would tend to be gregarious, sensitive to people’s feelings, and happy to proceed in situations where plans and roles are not clearly defined. Such a person would be a good fit to work as a tester on an Agile team. Another person might be an ISTJ, and that person would tend to be more thoughtful and reflective, fact-focused and logical, and decisive. Such a person would be a great fit to lead a team charged with selecting a new automated testing tool and creating and implementing an automated testing framework and strategy.

Another use of these personality types is to be aware of, and insightfully responsive to, potential personality conflicts within your team. For example, suppose you have a senior test engineer who has a judging personality type and is working on a testing project and reporting to a test lead; the test lead is more perceiving. On the one hand, the test engineer might become frustrated with the test lead because the senior test engineer might see the test lead as wishy-washy, unreliable, and providing weak leadership. On the other hand, the test lead might become irritated by what they perceive as rigidity and perhaps resistance from the test engineer. (Rex notes that he has personally been in exactly this situation, being more comfortable with setting and following plans in most cases.) The use of the MBTI can help you predict and ameliorate (sometimes in advance) such conflicts.

Classifying into personality types is a form of modeling, as was discussed in the Foundation and Advanced levels. This particular model is based on work done by psychologist Carl Jung in the 1920s and thus reflects his own philosophies and beliefs about human psychology. As with any model, to paraphrase W. E. Deming, the model is wrong (in that it is a simplification that captures only certain elements of the whole), but potentially useful (in that it aids understanding of the relevant elements of the whole). These MBTI types simplify the infinite nuances of human personality. Be informed, but not dominated, by the model.¹¹

Not only must the test manager understand the testers, but each member of the test team must understand their own position and relate effectively with the others. Do people understand their own roles and capabilities? Do people understand their colleagues’ roles and capabilities? Do people understand and respect their colleagues’ skills? Do people understand that colleagues with different skills are complementary to them, not incompetent and unqualified?

If the answer to any of these questions is ‘no,’ then you as a manager have some work to do. At the very least, you should take steps to help people understand these things. In addition, you might want to consider having a professional evaluation of your team, such as the MBTI test mentioned earlier. Such a test can help you better understand the dynamics of the test team. It can also help the individuals understand their own behavior and their colleagues’ behavior.

In addition to skills and personality types, there are other ways in which people are unique. People learn differently, which we’ll discuss in the following section.

Another important difference is how people deal with stress and where their stress thresholds are. The most extreme example comes from the military, in the form of combat stress reaction. (This is not to be confused with post-traumatic stress disorder, or PTSD, which is a permanent problem that can be caused by both physical and psychological factors but for which combat stress can be an initiating or aggravating factor.) Under stress, whether it is the stress of combat or the less potentially lethal but often quite stressful project gone wrong, people tend to become chronically fatigued and demotivated (often referred to as burnout). They don’t react as quickly as they once did because they have difficulty reaching a conclusion about the right thing to do next. They can lose track of important details or lose focus (often referred to as spacing out or in Internet parlance, 404). They can become angry, aggressive, bullying, or even violent (often referred to as snapping). Other factors may contribute to these behaviors, such as underlying psychological problems or chronic substance abuse issues, which are exacerbated by the stress.

A stress reaction can happen to anyone, but many important differences exist between people in terms of stress reactions. Some people find certain factors are particularly stressful, such as excessive overtime or concern about job stability. Different people have different levels of stress they can handle before a stress reaction occurs. Different people find that different activities can help them cope with and relieve stress. And different people have different tendencies to try to relieve stress in a functional rather than dysfunctional way.

Rex once worked with a project manager who treated his stress reaction with truly astounding rations of alcohol, including during the workday. He was moderate in his use of alcohol, and a very inspiring leader, until he suffered this stress reaction as the large, complex, and expensive project they were working on started plummeting toward destruction. As the day went on, his decision-making become more clouded by the alcohol, and his mood more erratic, irascible, and unpredictable. Rex started avoiding him, because as the test manager, he frequently had only bad news to tell him, and he did not react well when intoxicated.

It can help to keep stress in perspective and help your staff do so too. Rex remembers complaining on one project, after what he found to be a stressful meeting with the project management team, to one of his test leads. The test lead had been a noncommissioned officer in Iraq. He responded respectfully but with a Socratic smile, “So, did someone shoot at you down there?” Rex laughed, but he also learned something about himself and his need to keep things in perspective. (And he’s thankful to Amos for that insight.) While you need to manage your team, remember that you will learn a lot about management, and about yourself, from doing so. Ultimately, every test team is nothing but a collection of human beings: disappointingly fallible, yes; infinitely variable, yes; and yet often capable of amazing things.

2.3.5Skills Development, Training Opportunities, and Mentoring

In a previous section, we talked about skills inventories and using them to identify skills gaps in individuals and the team as a whole. Okay, let’s assume you have gaps in your team. We all do. As managers, we want our teams to be better and the people on our teams to be better. What can we do about it?

First, a manager should encourage their team members to improve their skills. In fact, we would suggest that continuing education—relevant education, of course—be part of the performance reviews mentioned earlier and discussed further later in this chapter. Relevant education would include the areas of tools, general areas of testing such as the ISTQB certification career paths, technology, methodologies and their effect on testing (e.g., Scrum), technical testing areas such as test automation or white box testing, business domain knowledge, and soft skills such as communication and leadership.

So, how can we improve people’s skills? As mentioned earlier, one approach is training. Training can take many forms:

• Training can be informal, where the materials and the training are delivered by other test team members. The advantage of such training is the direct relevance to your immediate needs. The disadvantages are the lack of training professionalism and the potential loss of exposure to externally recognized best practices.
• Training can be on the job, where the testers learn from mentors and colleagues while doing the actual work. The advantages of such training are not only the direct relevance of the training but also the fact that the knowledge transfer, from conceptual to applied, occurs immediately during the training. The disadvantages are the same as those for informal training but in addition, the fact that time pressures can lead the mentor or colleague to reduce the amount or scope of the training or to simply take over the job.
• Training can be professionally delivered by an instructor from outside the test team. The advantages of such training are a higher quality of materials and teaching (if you select the right training company) and the importation of externally recognized best practices. The disadvantages are the need for the attendees to figure out how to translate the lessons learned in training into actions in their daily work, though good, realistic exercises in the professional training can help with this problem. Cost is sometimes seen as a disadvantage for professional training, but careful selection of the training course and training company will result in a positive return on investment for the training.

Professional training can be delivered in three main modes. The first mode is instructor-led training, where an instructor comes to the client’s site, or members of the test team travel to a public training event, or a webinar-style interface is used to deliver the training. The second mode is pure asynchronous training, where trainees use an e-learning system to access training at their leisure. The third mode blends the first and second modes, where instructor facilitation and question-and-answer sessions accompany e-learning.

What if your training budget is limited and your organization small? You can always use cross-training by colleagues and mentors, as mentioned, to help fill skills gaps and increase the skill levels of your test team. What do people within your team know? You can use the answer to this question to try to generalize the skills within your test team. And don’t forget about reaching out to other groups such as development or marketing. The cross-training can involve technical skills, testing skills, or business domain skills.

There are various models for how people learn, wherein the models explore these different types of learning. For example, Neil Fleming’s VAK model says that people can learn through visual, auditory, or kinesthetic experiences. While most people will be able to learn through all three types of experiences, Fleming suggested that people tend to have one mode of learning that allows them to learn most quickly. A visual learner is someone who likes to see pictures or text; such a person would do particularly well in an interactive, live instructor-led course. An auditory learner is someone who prefers listening; while such a person can do well in a live training course, they are also well suited to take purely asynchronous e-learning courses where they listen to lectures. A kinesthetic (or tactile) learner is someone who prefers to apply new concepts immediately while learning them; such a person will do best when their training includes a lot of immediate sample questions, discussion points, and realistic exercises. When setting up less-structured training opportunities for people, such as brown-bag lunches and cross-training, you should take care to address people’s learning mode preferences.

Even if you are pursuing a specialized model of test team organization, there are likely some skills that you feel are required across the entire team. And, in specialized models, it’s a best practice to have more than one person with expertise in each critical skill so that you can handle situations in which one person is unavailable or leaves the company. Of course, in generalized teams, the need to ensure that all team members have an appropriate level of expertise is critical, so you should use cross-training aggressively to spread organization-specific skills.

Another variant of cross-training within the team is the use of mentors. Are there people within the test team who are particularly suited to help another within the test team move to the next level of skill? Is there someone on the cusp of becoming an expert in some critical skills area who could cross that boundary with the help of a current expert? Are there people entering the testing profession who need one or more experts to set an example for them and get them started in the right direction?

Mentors do not need to be from within the test team. You should consider what people outside the test team but within the organization know. These people can include developers, database administrators, and business analysts, among other people within the organization. Cross-training by such fellow professionals can be a good approach in any situation but especially in specialized teams where people need to learn particular skills that relate to the business domain or technology of system implementation.

Another excellent way to spread business domain or technical knowledge is through formal or informal job rotation. If a programmer or business analyst works in the test team for a few weeks, or maybe even months, imagine the knowledge transfer that can occur. Imagine the relationships that can be built on all sides and the informal problem solving that can occur through such relationships. This exchange of personnel can go both ways, of course, and testers can work within technical and business stakeholder groups as part of job rotation. You could assign a technical test analyst to work with programmers to help build an automated unit testing framework. You could assign a test lead to work with a programming or business analysis team to learn more technical, business domain, or management skills.¹²

2.3.6Performance Reviews and Feedback

Earlier in this chapter, we mentioned setting goals and objectives for employees and evaluating performance against those goals and objectives. It is important that this happen periodically, at least annually, and many organizations hold managers accountable for such reviews. Your human resources (HR) department might have specific requirements, forms, and so forth that relate to this process, so be sure to contact them before you start.

If you dread this periodic responsibility, you might be making mistakes in your management style. For one thing, if the periodic performance evaluations are a carnival of pain, perhaps your employees are not getting regular feedback on their work in a way that allows them to course-correct prior to the review? For another thing, don’t people deserve the attention and care inherent in their manager’s thoughtful consideration of their employees’ work products and deliverables, and the quality of that work, at least once in a while, away from the daily torrent of tasks and urgent issues? For yet another thing, part of your job as a manager is to inspire people to do their best possible work, so what could be a better avenue for doing so than such a discussion?

Of course, doing a good job does not just mean “doing a good job in my manager’s opinion.” Just as the test team has a number of stakeholders, each member of the test team has a number of stakeholders. Three-hundred-sixty-degree reviews allow the test manager to incorporate the opinions of those stakeholders in the review process. A first step, naturally, is identifying those people who have worked with or received work products from the person under review. (For those unfamiliar with the term, a “three-hundred-sixty-degree review” refers to a review that involves input from a person’s manager, their peer managers, and their direct reports.)

The next step of such a review is coming up with a questionnaire about the employee. This should include specific and open-ended questions about the employee. What work products have they delivered to the stakeholders in question? What is the quality of their work products? These questions should be both objective and subjective. Whether subjective or objective, the answers should be supported by facts and examples.

Of course, some managers dread performance reviews for good reason. Dreadful performance evaluations indeed can occur when employees have unrealistic expectations. These expectations can have to do with what their job is or how quickly they can advance in that job. So, when giving someone a performance evaluation you expect them to be unhappy with, be sure to explain the requirements of the job (which should already be clear from the job description), the employee’s demonstrated capabilities (backed up with data and facts), and the difference between those two. It’s great to have employees with aspirations, but when aspirations and reality diverge, your job as a manager is to bring those back into alignment, clearly and factually explaining the difference. Of course, it would be best not to wait until once a year to do this.

2.4Leading the Test Team

In many ways, leading a test team is like leading any other team. After all, people are people, no matter what work they do or where they live. Of course, there are some aspects that are specific to managing technical teams. And there are a few aspects that are specific to managing a test team. This will be the focus here in these final sections of Chapter 2. Given a good team, which you’ve developed carefully, how can you get the best work from them and keep them engaged and motivated?¹³

2.4.1Information Sharing and Communication

In Chapter 1, we discussed the importance of clearly defining the mission, objectives, and goals of the test group as well as how those goals are to be achieved and how to ensure that they are aligned with the needs of the organization. We also discussed the importance of defining the skills, roles, and responsibilities of people in the test group. Those are all necessary activities, but they are not sufficient. The test manager must ensure that the test group understands their role in the organization so that they can fulfill that role.

In addition to this general information about the role of the test group and each person within the group, the manager of a test group must ensure that each tester, test lead, and test manager understands how they fit into the project or program they are working on and how they fit with the stakeholders they are working with. Part of leadership is making sure that you obtain, and share with your staff, correct, current, and relevant information. So, not only do you need to be in the loop, you need to keep your people in the loop. Without obtaining and sharing the necessary information, your team cannot effectively and efficiently fulfill the goals discussed in Chapter 1.

While this may sound simple, there is a balance you must achieve. As a manager, you have to decide what information people need versus what information is irrelevant or distracting. We know of a company that had a stated policy of overcommunication. Everyone in the company received hundreds of emails and instant messages a day. Even the most effective and efficient employees spent inordinate amounts of time sifting through irrelevant and often inscrutable dross, trying to figure out whether any useful or actionable information existed therein. The less fortunate among them essentially drowned in a sea of trivia and irrelevancies, losing so much time and becoming so distracted that their effective productivity fell to zero. This was not a problem specific to the testing group because everyone suffered from this disease, but certainly the test director did not do anything to help the team rise above it.

We have also seen the opposite problem occur. We have known test managers and directors who adopted such strict need-to-know communication styles that people within their teams often found themselves lacking essential information. The most effective and efficient employees would often realize their dilemma and seek the information they needed from their managers and colleagues. This resulted in delays, of course, and sometimes incorrect actions when they received wrong information in spite of their best efforts. The less fortunate employees would simply become stuck, not knowing where to go for information or what to do in the absence of information.¹⁴

So, how to resolve this bind? One of the first things to accept is that you will not do a perfect job of resolving it. All managers—indeed, all employees—do an imperfect job of communicating within the organization. Every manager at some point or another communicates some irrelevancy to their staff, and every manager at some point fails to communicate some relevant fact or information to their staff. Be aware that you will have communication breakdowns, and deliberately choose which type of breakdown you want to have more frequently.

We prefer to lean toward liberally sharing information with our staff. This is because we have, as a manager, what Tom Peters referred to as bias toward action, and we want our teams to share that bias toward action. This means that we try to ensure that the pool of information available to each person is as complete as possible. It also means that we cannot punish or berate people for taking actions that were well-intentioned and well-informed but that did not turn out well. If someone on our team makes a mistake, our first assumption is that they did the best they could, acting on the information they had, and we work with them to find a way to avoid that mistake in the future.¹⁵

We have arrived at this preference for reasons we consider good. However, for various reasons, including the issue of organizational alignment discussed in Chapter 1, you might arrive at a different conclusion about how to strike the balance. We’re not trying to be prescriptive about what balance you should strike but rather trying to call your attention to the fact that you will strike a balance in this regard. That balance needs to be consistent with the way you choose to manage your team and your personal management style.

You should also remember that your staff will tend to mirror your own communication style. In other words, information-sharing and communication is a two-way street. If you are open in your communication style, then your staff will probably be open with you about information they have—provided you don’t create disincentives for them to do so. If you adopt a communication style that is too strict in holding information back, don’t be surprised when someday you are surprised by some pertinent fact that someone on your team already knew but didn’t tell you.

Within the parameters of your communication style, you will communicate with your team. How should that communication occur? There is a plethora of options, but typically one (or more) of the following methods is used:

• One-on-one meetings. These are private meetings between the manager and a direct report. Ideally, each person who works directly for the manager would have a chance to have such a meeting. The meetings should happen regularly rather than only in response to a crisis or mistake on the employee’s part. We think this is an excellent practice, especially when the manager adopts a policy of egalitarianism within the team because that encourages open communication.
• Weekly status reports. These reports should include discussions of current tasks, the progress on those tasks, and any obstacles or risks confronting the team member. Status reports can be individual and can then be aggregated into a group status report. We have used these, especially when information needs could be clearly defined. We have also used daily team status reports, especially during test execution when status evolved rapidly and information needed to be shared widely across the team.
• Informal communications. These can include emails, casual conversations, and social events with the team. Many managers adopt an open-door policy to encourage informal communication. Another practice is called “management by walking around,” where managers mix with their staff, often at all levels, to gather information. Of course, if you expect this to work, you must never punish or otherwise disincentivize people for sharing information with you. People who expect or fear negative consequences for sharing information will withhold information and probably look for ways to plausibly deny they knew about it or make it look like someone else had the responsibility of communicating it to you.

These communication channels need to go beyond simply the day-to-day tactical issues that arise in moving projects and programs forward. For example, on one of Rex’s teams, a personal dispute arose between two testers that did not—at least at first—directly affect project progress. The test leads became aware of the dispute but did not escalate it to Rex because they felt they did not need to distract him with it. (Rex accepted some responsibility for this error because he admits he was unconsciously sending a message of unavailability by being too absorbed in the moment-by-moment tactical realities of a project that was, indeed, barely staying within control.) By the time this problem reached the point where one person was accusing the other of harassment and accusing the test leads and Rex of unintentionally allowing a hostile work environment to arise, it posed a crisis to the progress of the project. Alcohol and gender were factors, which further complicated the situation. Your job as a manager is not just to manage the project and program work but to manage your people too.

Effective communication, especially verbal communication, is a matter of receiving information as much as transmitting it. If you send the message that you are not paying attention, or that you are exasperated, bored, or uninterested, that will come through to your staff, and they will stop communicating. Some managers are very good at communicating to their teams but not very good a communicating with their teams, which results in those managers not having important information when they need it.

When you are talking to someone, be an active listener. Look the person in the eyes. Reflect back to them what you understand from their communications, and give them an opportunity to confirm or clarify. Don’t allow yourself to be distracted by text messages, instant messages, Twitter or Facebook posts, incoming emails, and so forth. If your phone rings, use caller ID to decide whether to interrupt the discussion, and if you must, apologize before so doing and ask your communicant to hold their thought because you want to get back to the discussion as soon as you can complete the urgent call. You owe it to your staff to be open, present, and responsive during communications with them.

In addition to these communication channels, you should have test management tools that track project status information. That information should be available to test team members. Another approach to make information readily available is to use what are called “information radiators”—walls or whiteboards of reports and status information visible to all who walk past. Of course, this information must not only be available but also understandable. We’ll discuss test results reporting further in Chapter 6.

2.4.2Fostering Loyalty and Trust

According to Rex, the highest compliment he received as a manager occurred when a test lead on his test team told him, “You know, this project is really badly managed, and I can’t see how it can succeed. I’ve thought many times about quitting because there are so many things going on that bother me, but I know that quitting would leave you in a bad position. I guess I’d walk across fire to avoid creating a problem for you.”

A lot of the good luck Rex has had promoting loyalty and trust within his test teams arises from following the golden rule in terms of not doing to your employees what you wouldn’t want done to you. If you do have to ask people to do something you don’t want to do—e.g., working overtime or weekend hours—then make sure you share the pain with them. Stand by your staff members, and protect them from outside interference, innuendo, or intrigue. Supporting your team is a sign of respect, and you should, of course, respect your team. After all, if you’ve followed the steps outlined in this chapter, you built the team.

People also tend to respect managers who are open and honest with them. This goes back to the point made earlier on sharing of information. If you hide information from your team and then that creates a problem, people will be less motivated to help you resolve that problem than they would if you allowed them to see the problem coming. Of course, you can be put into a real bind here if you are told something by your managers and told not to disclose it to your employees, which we have seen happen during decisions about layoffs and outsourcing. When the information does come out, be sure that you explain to the team your reasons for withholding it, and do so in a way that does not come across as a weasel-worded attempt to evade your responsibility for keeping your team informed.

In addition, your staff should feel safe and valued. When people feel safe, they use their best judgment and efforts to advance the mission, objectives, and goals of the test team. When they feel valued, they trust their judgments and insights. They feel free, and indeed compelled, to tell their colleagues and managers what they need to know—in a respectful way—even when that is bad news. As a manager, if you want loyalty and trust, you should reward and never punish team members who do what is right, respectfully tell the truth, and act with good intentions and prudence, even when the outcomes are not always desirable. Feelings of safety and value promote excellence from your team.

Some people, when feeling threatened, will focus on doing exactly as they are told, nothing more. They will be sure to gather evidence, a paper trail, to prove that they are beyond reproach in their actions. They will waste time on petty intrigues against each other, and possibly against you. It’s very difficult to get excellence from a team that operates under conditions of fear, especially if it is you that they fear.

2.4.3Team Building

It is not enough that the team trust and be loyal to you. Another essential element of obtaining excellence from your test team is team cohesion, trust, and loyalty between each member. Even when people worship their manager, if the team is riven with disputes and enmity, it will prove a distraction at best. Even the most competent team, with the best manager, can be rendered ineffectual by internal disputes. If those arise, you must as a manager quickly extinguish them, but it’s far preferable to avoid discord in the first place.

Creating and fostering a sense of shared mission, of “teamliness,” if you will, is a complex thing. However, there are at least three important factors to consider. The first is open communication: We discussed this earlier in the section “Information Sharing and Communication.” A team feels more like a team when people feel that communication channels are open between you, as a manager, and them, as testers and test leads. A team feels more like a team when people feel safe communicating openly with each other within the team. You should avoid a situation in which all information has to flow up to you as a manager before flowing out across the team. We know of an organization in which such strict hierarchical information channels significantly stifle teamwork, and it is particularly frustrating to people that the manager of this organization, who had created this dynamic, keeps stressing collaboration and blaming individuals for not communicating openly with each other. You should also avoid a situation in which people on the team feel free to communicate with each other but not with you as a manager or with other members of the management team. When you create open communication channels within a team, people will tend to feel included. Closed or hierarchical communication channels make people feel isolated at best and at worst, embattled.

The second important factor in creating a team is respectful behavior between team members. While you want people to have open communication, we believe it is best to ensure that communications are respectful. Being encouraged to communicate openly does not mean being free to say whatever you want to your colleagues. However, this belief is not universally shared. Steve Jobs, the late head of Apple Computer, had a very blunt communication style and would openly and sharply criticize people when he felt disappointed or unhappy with their work.¹⁶

It’s also true that courtesy of communication lies on a spectrum, and each manager must establish, within the broader corporate culture, what the proper balance is between politeness (which can interfere with openness if it is excessive) and directness (which can feel disrespectful if it is excessive). That said, disrespectful behavior goes beyond communication, and teamwork is always degraded when people feel devalued or, worse yet, fearful and threatened. Such disrespectful behavior is especially unacceptable if it has cultural, gender, or sexual overtones. You lose credibility as a manager, and expose yourself and your company to legal risk, if you tolerate such a hostile work environment.

The third important factor is creating a shared understanding of the roles and responsibilities of the group, which we touched on in the section “Defining Clear Roles and Responsibilities.” Simply defining those clear roles and responsibilities is not enough. As a manager, you need to make sure people understand their own roles and responsibilities and those of their team members.

So, given these factors, how can you build a team and create positive team dynamics? As the cliché goes, you must talk the talk and walk the walk. When you say that open communication is important, you must be as open as you can with your team. When you say that respectful behavior is important, you must treat everyone on your team with respect. When you say that a shared sense of roles and responsibilities is important, you must share with people the entire team’s set of roles and responsibilities.

There are a number of team-building activities that can help you encourage a sense of team as well:

• Group meals. You can have lunches brought in, especially during hectic periods or at major project milestones. You can organize an event where an individual member brings in a special dish, which can be especially fun when you have a multicultural team. You can have group dinners from time to time too, but be sensitive to the situation of people with families.
• Celebratory parties. When a project is completed or other major milestones occur, you might consider marking the occasion. The party can be as simple as a quick break for cake or something more elaborate such as an off-site party. Party favors such as special shirts or coffee mugs (with a project logo or slogan, for example) can help people remember the event later. In certain company and local cultures, having alcoholic drinks available can be appropriate too, but you should remain sensitive to the problems and risks that excessive alcohol consumption can create.
• Sharing special events. When a team member has a major life event, such as a birthday, wedding, or new baby, it’s appropriate to notice that and in many cases organize a simple team celebration. This can be overdone, so take care that you do not mark these events in a way that makes people feel uncomfortable. When the major event is negative—such as a death in the family or a major illness—then sincere, thoughtful, and supportive actions are in order.
• Team newsletters or intranets. Having an internal newsletter, web page, or wiki page that provides opportunities for further communication can also help the team feel more connected.
• Educational sessions. We referred to “brown-bag” sessions earlier, where, over lunch, test team members can present new concepts to each other. These new concepts can include new test tools, interesting articles on testing, and a test technique that others can benefit from, among other ideas.
• Professional help. There are a number of companies that offer services to help build teams. For example, some companies arrange “ropes courses” where people and teams are challenged to do various rope-walking activities or simulated rock climbing. These can be appropriate for adventurous teams and can result in a feeling of energy, trust, and unleashed potential when people find that individually and together they can overcome self-imposed limitations and irrational fears. Other companies arrange more introspective types of activities, including human potential and group encounter events. You must use your judgment on how to implement these activities though; the situations used to build a team must be physically, emotionally, and psychologically safe for the participants if they are to result in actual team building rather than injury to the team spirit if not physical injury to the team members.

When you set up team-building activities, try to be as inclusive as possible. It can be hard to work around everyone’s schedule, but simply scheduling events without considering individuals’ personal situations, holidays, and other constraints tends to defeat the purpose. In the current distributed approach to working, scheduling inclusive team-building activities is made more complicated by the fact that teams are often not colocated. If you can’t get everyone into a single activity, then consider multiple activities.

Team-building activities need not be limited to building the test team only. Testing groups work within larger software organizations, which include other groups such as business analysts, programmers, technical publications, and so forth. Organizations of any kind work more effectively and efficiently when people understand each other’s roles and responsibilities, as mentioned earlier. Consider organizing a cross-functional event or series of events (such as an on-site lunch) where all the major groups within the organization give quick presentations about what their group does, what major inputs they need to receive, what major outputs they deliver, and how their work benefits the organization. Not only will such events make the organization more effective and efficient, they will build a stronger sense of teamwork across the organization.

2.4.4Motivating and Challenging the Test Team

Most adults are not inherently lazy and do not require constant pressure and micromanagement from their superiors to stay on task. Most adults are relatively self-motivated. They want to work and enjoy the feeling of accomplishment that comes from a job well done. They like to overcome challenges and learn new things in their jobs. They look forward to the next project or assignment. People are motivated by a salary that they consider fair. So, much of motivating a team consists of simply letting these natural human motivations work.¹⁷

That said, motivation is a matter of degree, and part of your job as a manager is to create a team that is highly motivated to succeed. Major elements of motivation have to do with skills growth and team building, which we’ve already discussed in this chapter. What are some other ways we can produce highly motivated teams?

Open communication is one part of motivation. When people understand the status of their projects, the organization, and the company as a whole, that tends to improve motivation. As a consultant, Rex sometimes hears line managers and individual contributors say, “This company practices the mushroom management style: We’re all kept in the dark and fed lots of steer manure [to use a polite term].” The people saying this are generally not motivated employees.

This also includes evaluating your team-members’ work. When their work is good, you can give the feedback publicly, and many times you should. It’s important that public praise be given in a way that is sincere and specific rather than phony and vacuous. Simply walking up to someone’s desk, apropos of nothing, and saying, “Great job, Janet; I really like your work lately,” only makes you look like a clueless management dork—someone straight out of a television parody of office life—while leaving the praised employee entirely nonplussed and possibly a little embarrassed.

When their work is not acceptable, the feedback should be one-on-one. Criticizing people’s work in public, especially if it’s accompanied with personal put-downs, is never acceptable and is demotivating to the individual and to everyone who witnesses it. It’s important that, in private sessions where you criticize someone’s performance, you start with a clear statement of the problem, including an objective presentation of the facts, and then honestly and supportively work with the person to guide them toward doing a better job. This includes listening to and considering their side of the story, because it’s possible that significant external factors contributed to the problem. Simply calling someone into your office, yelling at them about their lousy performance lately, and sending them along their way is less likely to produce the desired result than an honest discussion that includes specific guidance toward better work products and behaviors.

Most people are perfectly willing to work hard but find it difficult to do work that might be easy but is tedious or boring. The human brain is wired to seek out and enjoy activities that result in acquiring new information, new abilities, new challenges. You can take advantage of this evolutionary reality by giving people opportunities to do new and different work. Assign work in a way that challenges and stimulates people, considering their abilities and experience. Job rotations, training, and other ways of making people’s work life diverse and novel can all help maintain motivation.

We discussed team development and your need to build a team that is well aligned with the organization’s needs earlier in this chapter. There’s a motivational opportunity in this area too, which you can and should use. When you identify areas where the team needs to improve, ask the test team members which of those areas are of interest to them. You can then simultaneously promote and positively take advantage of an individual’s motivation by having them grow their own skills in that area.

One important benefit of leading an engaged and motivated team is that of retention. People who are happy in their work are generally less likely to “jump ship.” In fact, the positive feedback loop of a motivated team may entice others to try to join your team because working there is such a positive experience.

2.4.5Managing Distributed Teams

In the next chapter, we’ll take a look at some of the important “hard” factors associated with managing distributed and outsourced testing work, but there are also “soft” factors—human issues—to consider. The colocated project team, for all its manifold benefits and advantages, is a rare luxury these days. For many, indeed most, test managers, managing testing teams that are distributed throughout the building, around the city, across the country or even the world is a fact of life now. What are some of the challenges to consider, and what are the management questions raised thereby?

One of the challenges can be time-zone differences. Obviously, when members of a group work in different time zones, they are not all working at the same time. In extreme cases, such as overseas outsourcing, the number of common work hours—the window of time when everyone on the team can be assumed to be at work—may be limited or even nonexistent. This can create numerous communication challenges that you should be aware of and actively manage. This lack of overlapping hours can also affect the team-building activities discussed earlier, and you should look for creative ways to overcome this challenge to ensure that local and remote teams feel equally engaged.

Another challenge relates to cultural differences. People are people, it is true, but cultural differences can obscure that. For example, cultural differences can lead to problems when people misunderstand each other’s verbal and nonverbal communication. What we have found helpful is encouraging people on each side of the cultural divide to instead see cultural differences as a paid opportunity to learn about different cultures without having to spend a single vacation day or fly around the world packed into the economy end of an airplane.

Often concurrent with cultural differences are language differences. Language differences may seem to be the ultimate barrier to communication, but in fact, because these differences are so obvious and so clearly require direct action, they have the advantage of being managed in just about every instance where we’ve seen them arise. In fact, it’s the situations in which people spoke different dialects of the same language—e.g., US English versus UK English versus Indian English—where we’ve seen the most frequent language-related communication and other problems arise.

Language differences, cultural differences, and time-zone differences all directly affect communication. You should plan, in advance, how you intend to manage communication between the distributed elements of your team. Regularly scheduled conference calls or video meetings are useful. Internet telephony and webcams have made it possible for people to have sophisticated communications from around the world, and perhaps even from their homes, which can reduce the need for people to be in their offices at odd hours when you’re trying to coordinate a conference call across three far-flung time zones. For example, try to find a common work hour between New Zealand, United States, and Europe.

While no country has a monopoly on intelligence, it is true that skill levels tend to vary depending on whether a country’s IT economy is fully developed (e.g., North America, Europe, and Japan), just emerging (e.g., Vietnam and the Philippines), or somewhere in between (e.g., India, Brazil, and China). As a test manager, you must manage the impact of the relative skill levels of the testers available in a given locale. For example, outsourcing highly complex and interactive activities such as performance testing to a team of smart but utterly inexperienced testers in an emerging economy will not turn out well.

Finally, we have discussed in this chapter and the previous chapter the importance of alignment of expectations. That is a particular issue for distributed teams. When people are working with inconsistent expectations or objectives in a colocated situation, those misalignments become obvious in a relatively short order. The manager need not be especially observant to see them. However, in distributed situations, such issues can lurk, undiscovered, for quite some time. You will need to actively work to ensure that expectations and objectives are consistent and that there is an understanding and acknowledgement of these expectations across the team.

A common theme throughout this discussion of distributed teams is that of communication. Your goal, as a test manager in charge of a distributed test group, is to attain the same level of clarity, timeliness, and effectiveness of communication that you would have with a colocated team. Similarly, your goal is to attain the same level of alignment of expectations and objectives that you would have with a colocated team. While these goals may be a challenge to realize, you should aspire to this end, and you should remain aware that, to the extent that your distributed team falls short, the benefits of distributed work are reduced while the risks associated with distributed work increase.

2.5Sample Exam Questions

In the following section, you will find sample questions that cover the learning objectives for this chapter. All K5 and K6 learning objectives are covered with one or more essay questions, while each K2, K3, and K4 learning objective is covered with a single multiple-choice question. This mirrors the organization of the actual ISTQB exam. The number of the covered learning objective(s) is provided for each question, to aid in traceability. The learning objective number will not be provided on the actual exam.

The content of all of your responses to essay questions will be marked in terms of the accuracy, completeness, and relevance of the ideas expressed. The form of your answer will be evaluated in terms of clarity, organization, correct mechanics (spelling, punctuation, grammar, capitalization), and legibility.

Question 1

LO 3.2.3

Consider Scenario 2. Evaluate the following options:

1. Training in the application business domain for all staff.
2. Training in the application business domain for some staff.
3. Training in the application technology for all staff.
4. Training in the application technology for some staff.
5. Training and certification in the ISTQB program for all staff.
6. Training and certification in the ISTQB program for some staff.

Which of the following training programs best meets your needs and puts the options in the proper sequence?

First IV, then VI.

First V, then III.

First II, then VI.

First VI, then IV.

Question 2

LO 3.2.1 and LO 3.2.2

Assume that you are going to hire a new tester. You have résumés from three candidates, which can be summarized as follows:

• Candidate A’s résumé indicates a strong background in testing financial applications and nongame consumer electronics, strong technical skills, excellent testing skills, and ISTQB certifications.
• Candidate B’s résumé indicates some experience in testing games, limited technical skills, good testing skills, and ISTQB certifications.
• Candidate C’s résumé indicates a strong background in game testing, limited technical skills, and no structured testing skills or certifications.
• Candidate D’s résumé indicates some background in game testing, strong technical skills, five years of testing experience, and a desire to avoid ISTQB certification.

The question consists of two parts:

1. Explain which of the four candidates you consider the best candidate(s) and why.
2. Describe, using the interviewing concepts listed in the syllabus, how you will interview the candidate(s).

Answer each part succinctly but fully.

Question 3

LO 3.2.4

Consider the following options:

1. Frequent personal conflicts involving a particular employee.
2. Personal relationships between you and individual testers.
3. Cross-training the people who will replace the terminated employee.
4. Test execution effort exceeding planned effort on a project.
5. Ways to internally reassign the terminated testers.
6. A mismatch between an employee’s skills and the new product line.
7. Percentage of defects found by each tester.

Select three considerations that are always important when terminating an employee.

Question 4

LO 3.3.1

As a test manager, you have a development plan that covers all the individual testers on your team and addresses the existing skills gaps. This plan is aligned with the needs of individual testers, the test team, and the wider organization. The tester’s annual performance goals include making progress toward skills development planned for them in that year, and these goals conform to the SMART guidelines. You regularly review progress toward these goals with each of the individual testers.

Based only on the information given, which of the following statements is true?

1. This plan will gradually increase the capabilities of your team.
2. Using the goals is not a fair way to measure employees.
3. You now should do a task analysis and then a skills inventory.
4. This plan replaces any need for formal training.

Question 5

LO 3.3.2 and LO 3.3.5

Assume that you are working with an individual tester to set skills-development goals for that person in the coming year, working within the situation described in Scenario 2. Assume that risk-based testing is the primary test strategy for your group. The tester’s résumé indicates some experience in testing games, limited technical skills, good testing skills, and an ISTQB Foundation certification.

Which of the following goals is consistent with the team development plan outlined in Scenario 2 and fits the guidelines for a SMART goal?

1. Take a course on technologies used in developing social media games, identify ways to apply those technical skills on your next project, and develop and execute appropriate technical tests assigned by your test manager.
2. Take the in-house e-learning Advanced Test Analyst course, participate in quality risk analysis on your next project, and develop and execute tests consistent with the quality risk items assigned by your test manager.
3. Attend a lunchtime brown-bag session on risk-based testing, participate in quality risk analysis on your next project, and develop and execute tests consistent with the quality risk items assigned by your test manager.
4. Take the in-house e-learning Advanced Test Manager course, participate in quality risk analysis on your next project, and develop and execute tests consistent with the quality risk items assigned by your test manager.

Question 6

LO 3.3.3

Assuming that the test manager has aligned each tester’s roles and responsibilities with the goals and objectives for testing, which of the following answers best captures the reasons individuals should understand their roles and responsibilities in the test team?

1. To define a generalized model for organizing the test team.
2. To ensure that the test manager derives proper metrics for each tester.
3. To help them align their actions with the mission of the test group.
4. To support termination of testers that do not meet performance expectations.

Question 7

LO 3.3.4

Assume you are working on a project and most of the test development and execution will be done by an offshore team. The offshore team has clearly defined service-level agreements (SLAs) for their work and templates for their work products. If you need to assign a senior tester to review this team’s work products and advise them on how to improve those work products, which of the following statements is correct?

1. A thinking individual is not suited to review the work products from the offshore team.
2. A feeling individual is best suited to review the work products from the offshore team.
3. A judging individual is not suited to advise the offshore team.
4. An extroverted individual will tend to engage well with the people on the offshore team.

Question 8

LO 3.4.3

Assume you are managing a test team and significant regression testing is required and most testing is currently done manually. Which of the following approaches best manages the challenges of motivating this test team?

1. Build a regression-testing framework and train testers to build automated regression tests.
2. Publicly praise testers for finding regression defects with manual tests.
3. Publicly criticize testers when they fail to find regression defects with manual tests.
4. Outsource the manual regression testing to an offshore team.

Question 9

LO 3.3.6 and LO 3.3.7

Consider Scenario 2. Assume that you are doing annual performance reviews for two people on your team:

• The first person is a high performer. She has high skills in terms of testing generally, games testing specifically, and games technology. She has ISTQB Foundation and Advanced Test Analyst certifications.
• The second person is a poor performer. He is perfectly pleasant to work with and fits well with the rest of the team in terms of personality. However, he lacks general testing skills and is not ISTQB certified.

Describe the main elements of each of the two performance reviews you will write.

Question 10

LO 3.4.1

Assume that you are managing a team of testers that performs maintenance testing on monthly releases of bug fixes and patches for a commercially successful mass-market application. The testing process for these releases is routine, including regression testing using an automated functional and performance regression test set along with manual tests of new functionality. Work is proceeding smoothly, and your testers are able to work independently without regular course correction from you. Each tester has their own area of responsibility, but they also need to be aware of each other’s work. Your manager expects you deliver a regular report on the status of your testing.

Which of the following is the most effective communication method in this situation?

1. Have informal communications with your team toward the end of each week, and then aggregate the information from that meeting into a group status report that you deliver to your manager but not the test team.
2. Obtain weekly status reports from your testers, and aggregate that information into a group status report that is delivered to your manager but not the test team.
3. Hold one-on-one meetings with your team each day throughout the week, and aggregate the information from that meeting into a group status report that you deliver to your manager and the test team.
4. Obtain weekly status reports from your testers, and aggregate that information into a group status report that you deliver to your manager and the test team.

Question 11

LO 3.4.2 and LO 3.4.4

Consider scenario 2. Assume further that an analysis shows that miscommunication and failure to communicate within and between test teams was responsible for the largest number of incidents that reduced overall effectiveness or efficiency and that these communication breakdowns arose mostly from a lack of trust between individuals.

You have decided to address this problem in an urgent fashion because some of the incidents that have occurred are quite significant in their impact and embarrassing to the test team. Which of the following is the best teambuilding activity to help resolve the problem?

1. Team newsletter.
2. Professional help.
3. Group meal.
4. Celebratory party.

1We could give references to a few news stories or articles here, but you can certainly find ample recent references by scanning the news. This kind of behavior occurs regularly.

2Other good resources are Johanna Rothman and Gerald M. Weinberg’s book Hiring The Best Knowledge Workers, Techies & Nerds and Judy McKay’s book Managing the Test People: A Guide to Practical Technical Management.

3Other good sources for interview questions include Judy McKay’s book Managing the Testing People: A Guide to Practical Technical Management and Rex Black’s book Managing the Testing Process: Practical Tools and Techniques for Managing Hardware and Software Testing.

4The question of exam validity is a complex one. Do an Internet search on “psychometrics” to see all the statistical and other issues involved.

5In addition to the material discussed in this section and the next, we recommend Tom DeMarco and Tim Lister’s book, Peopleware: Productive Projects and Teams, and Steve McConnell’s Software Project Survival Guide (Developer Best Practices).

6Rex actually saw a situation where the CEO of a company manipulated the personnel evaluation process to reduce people’s merit increases to cover up some financial mistakes made by the CFO (who happened to be his girlfriend). It was a particularly ugly situation made even uglier when someone in management decided to forward the memo where the CEO made his plan clear. The company was out of business within 12 months.

7The classic demonstration of this problem is Deming’s red-bead experiment. In that experiment, management provides the tools to be used to separate a mixed container of different-colored beads into two containers of same-colored beads. Management also defines the process and the timeline for separating the beads. Individuals are then rewarded and punished based on their speed and error rate, even though the differences are due entirely to natural variation in the process as defined. Video clips and explanations of the experiment are available on the Internet, and Amazon even sells a red-bead experiment kit, should you want to try it yourself.

8This variance in effectiveness can be significant. As DeMarco and Lister cite in Peopleware, for any given task, the difference in effectiveness between the most qualified and least qualified person can be an order of magnitude—e.g., a ten-times capability difference.

9Further information on skills inventories and their use in creating job descriptions can be found in Rex’s book Managing the Testing Process, 3e. You can find examples of skills inventories in the Basic and Advanced Libraries on www.rbcs-us.com.

10For general psychological issues of management, it’s hard to top the gold standard set by DeMarco and Lister in Peopleware, 2e. Some of the psychological issues specific to test management are covered in Rex’s book Managing the Testing Process, 3e, and Judy McKay’s book Managing the Testing People. You can also listen to Rex’s webinar The Psychopolitics of Test Management, found at www.rbcs-us.com/software-testing-resources/library/digital-library.

11You can find more information on the MBTI test at www.myersbriggs.org.

12Rex had a number of clients tell him that they use job rotation in one form or another for cross-training. And don’t underestimate the power and value of informal problem-solving either. In many business cultures, including North American cultures, the ability to work with someone you know outside of your immediate working sphere to solve a problem will, as you move up, have a greater and greater influence on your ability to move up the organizational ladder. Tom DeMarco makes a number of related points in his book Slack.

13More on management of technical workers can be found in Tom DeMarco et al.’s book Adrenaline Junkies and Template Zombies: Understanding Patterns of Project Behavior. For discussions that are specific to managing testers, see Rex Black’s book Managing the Testing Process: Practical Tools and Techniques for Managing Hardware and Software Testing/Edition 3, Chapter 8, and Rick Craig’s book, Systematic Software Testing, Chapter 10.

14For readers not familiar with the term, need to know is an intelligence term that refers to classification of information in such a way that only those with a demonstrable and preapproved need to know some piece of information—i.e., as part of their current duties—receive that information. While this may sound reasonable, it presumes that everyone’s superior and those driving the classification system have a perfect understanding of what each person in the organization needs to know. That’s impossible. The most significant failure of this sort of policy of information hiding occurred in the events leading up to and upon September 11, 2001, when various intelligence and law enforcement agencies in the United States did not “connect the dots” and thus missed numerous indicators of the impending attack. The few individuals who did connect some of the dots were unable to overcome the information-siloing that was built in to the overall system. You can read more about this failure to use intelligence intelligently in The 9/11 Commission Report.

15See Tom Peter and Robert H. Waterman Jr.’s classic management book In Search of Excellence: Lessons from America’s Best Run Companies.

16See, among numerous other sources and Internet mythology, Walter Isaacson’s book Steve Jobs.

17See, for example, Daniel Pink’s book Drive: The Surprising Truth About What Motivates Us.