Chapter Five. Preparing for a Retrospective
Owl noticed Singed-Tail, the wolf formerly known as Big Bad Wolf, reading a cookbook entitled Swine in Wine. “Are you thinking about making another run at those pigs?” Owl asked, as he gently landed on a branch above him.
Singed-Tail growled, “Yep, and this’ll get me prepared.”
Owl probed, “Have you read anything else?”
“Nope,” Wolf responded.
“Nothing at all?” Owl asked disbelievingly.
“No, not recently. . . . What are ya gettin’ at, Owl?”
“Well, I’ve seen some of the other wolves reading The Sage Wolf’s Complete Guide to Hunting. I was just wondering if you might want to paw through the Getting Prepared section for a few good ideas. Maybe it will alert you to something you’ve missed.”
Singed-Tail yawned. “I read that book—a long time ago. It was just a bunch of common sense. Nothin’ to howl about. I’d rather be chewin’ on a weasel bone than readin’ that stuff again. Now scram before I get hungry and reconsider my no-chicken policy!”
Having been rudely made aware of Singed-Tail’s disinterest in his advice, Owl quietly flew off. Upon reflection, Owl realized that Singed-Tail didn’t understand why the wolf elders wrote their guide—to share their wisdom and to list all the things they had overlooked when they were young and inexperienced.
But did he have to call him chicken?
Human nature (and, as the fable shows, wolf nature, too!) makes it hard for people to appreciate the value of certain kinds of information unless it can be used immediately. Back when Singed-Tail had read the Getting Prepared section in the hunting guidebook, he wasn’t getting ready for a hunt. So, at the time, his reaction was that the advice contained nothing useful to him.
I suspect that this chapter on preparation may be best appreciated when you are actively preparing to lead a retrospective. The information may seem like common sense to the first-time reader, but the message I want to convey is that preparing fully for a retrospective is essential to your success.
If this is your first reading of this book, you may want to skim this chapter, looking at the cartoons and reading the True Story section before moving on. This approach is fine, so long as you come back to study the contents when you prepare for a retrospective. For now, I ask you to remember how foolish Singed-Tail was to conclude that information on preparation isn’t worth studying! For those wanting specifics, this chapter provides key information on how a facilitator can get ready for a retrospective, including how to prepare the participants (both developers and managers) and how to secure their involvement.
Connect with the Managers
The first job in getting prepared involves learning ways to connect with management. Even though retrospectives are a powerful way for managers to improve control of projects—that is, to change what didn’t work into a strategy for future success—not all managers like the idea. Not all see retrospectives as a way to learn how to manage better. Some managers fear that the retrospective will become a public review of their ability to manage. Others believe that they already know what needs to be changed, and only want to use the retrospective as a way to get their staff members to buy into the obvious. Still other managers see a retrospective as a waste of time, but think that it is politically advantageous to put on a show of having one. Even those managers who do support the retrospective and see it as an opportunity for improvement may find it difficult to give up being a leader to become a listener during the retrospective process. For these reasons, facilitators must start by establishing a sound working relationship with project management. The steps I use to connect with management follow.
The first activity I undertake is to meet with the managers to give them information about myself, to get a sense of why a retrospective is important to them, to discover what they hope will happen, and to learn what they fear. I believe a face-to-face meeting works best, but I have also used telephone conference calls, followed by conversations with specific individuals.
At the beginning of the meeting or conference call, I introduce myself, describing whatever personal credentials I feel are appropriate to the specific situation. I then go on to define the goals of a retrospective, to briefly describe some of my experiences with retrospectives I believe to be relevant, and finally to summarize the results that can be achieved. My objectives in this part of the introductory session are to
• give the managers a chance to develop a sense of who I am and of how I work
• present a general idea of what will happen at the retrospective
• convey the idea that a retrospective is a learning process rather than a fault-finding experience
• establish a vision of the possibilities
By means of this session, I hope to eliminate any lingering fear, to increase the managers’ understanding of how a retrospective can help with the particular project’s problems, and to secure support. In particular, I need the managers’ help in presenting the idea of a retrospective to their people and I want them to be able to answer questions that get asked most frequently.
Then, I change the focus. Because I want to know exactly what the managers think about holding a retrospective, I ask questions such as
• What would you like to accomplish by having a retrospective?
• What would a successful retrospective look like to you?
• What would you like to have happen after the retrospective?
• Why are you asking me to lead this retrospective?
• What are your greatest fears about this retrospective?
• What topics need to be addressed?
• What topics do I need to be careful with—and why?
• What details about people can you share that I should know?
Usually, my final question is, “What questions do you have for me—about my background, the retrospective process, or anything else?”
After I have answered any questions asked of me, I describe how I think the retrospective might proceed, and I invite the managers to suggest ideas as well. I explain that, as a team, we will work together to develop the retrospective goals. I call goals from the first session “initial goals” since they are likely to change as we work through the retrospective. Typical initial goals are listed below.
• Explore why we had such difficulty in making a schedule we could trust.
• Discuss how to discover if a programmer is blocked and how to react without committing the sin of micro-management.
• Determine whether we spent the right amount of time in the analysis phase.
• Discuss how we might reorganize the team for the next project.
• Learn how to do a better job at interviewing, managing, and overseeing contract help.
• Decide whether we should put more effort into patterns and components for the next project or if we should scrap the idea.
• Find ways to help Bob in Testing get his job done.
This list provides a good starting point, and as we deviate from these initial goals, we use the list as a management tool—scratching out, adding, and changing goals as required.
One practice we must agree upon is that managers will not participate heavily in the retrospective discussion. Their main responsibility is to listen and learn. A second responsibility of the management team is to provide insight from their perspective to address issues raised by software developers and other participants. Before the retrospective, I work out a system of signals with the managers that I can use during the event. For example, I will use one signal to say to the manager, “You are participating too much right now—try just listening for a while.” A manager will have another signal to tell me, “I really need to comment right now!”
Often, it is hard for managers to stop leading and to play the role of observer. I know from personal experience that when I have been the designated leader on a project, I have found it very difficult to shift my role to that of participant during the retrospective. In those situations, I usually need to be reminded about consciously suspending my leadership role. For managers who are really having a hard time just listening, I suggest that they take detailed notes that can serve as triggers for later discussion. I tell them I will meet with them at break time to look over the notes to see if we need to do any redirecting of the process. I also tell managers that they can hold “staff meetings” at meal breaks if, during the retrospective, they need to discuss any issues about work on current projects.
Map the Community
As a way to further understand the project’s dimensions, I have the managers help me develop a “map”—that is, a version of an organizational chart—identifying the community that worked on the project. The map helps me understand working relationships between project team members, and the act of creating it provides an informal setting in which the managers can tell me their view of what went right and what went wrong.
To create the map, I need to know the following:
• What was the formal reporting structure? Did it change over time? If so, how? Who filled each position, and what were their responsibilities? When did they start in this role and when did they stop?
• What were the important informal relationships that existed during the project? How strong were those relationships? Did they aid or hinder project progress?
As the map is developed, and as personnel and roles are discussed, you may hear unfamiliar acronyms or abbreviations, such as CHE, GMAS, BIM, and so forth. This is the perfect time to write down what these letter combinations mean. My preference is to include them as a legend on the map. Instead of using a standard notation, I usually make something up that best fits the individual situation. However, there are a few conventions I use frequently—both for formal and informal reporting relationships:
For informal or special relationships, I use a different type of line:
Sometimes, I make up the notation as I hear the story. For example, a line with many bars dissecting it shows a relationship with problems.
Below is an example of a representative organizational map:
A real company’s organizational chart would differ from my example in at least two ways: First, there would be more informal relationships; and second, the diagram would not be as neat! Usually, a chart depicting a real project’s organization becomes messy rather quickly, since the relationships exist in a way that defies a clean drawing. Also, reorganizations are common, and capturing both the “before” and “after” of an organization can be a real challenge.
There are many reasons why I recommend doing an organizational map, but prime among them are the following:
• The map helps the facilitator understand relationships among people in the organization.
• Drawing the map prompts the managers to begin to tell the story from their perspective (such as, how people were deployed).
• The activity is an introduction to a bit of project history—when people joined, left, were reassigned, and so forth.
• The map helps identify who should attend the retrospective.
• Legend annotations help clarify the abbreviations and acronyms used on this particular project.
Collect Effort Data
Slipped schedules and cost overruns are all too common on software projects. Given this fact, I’m surprised at how few people actively measure and record the money and time expended on a project. I often ask clients how much they spend to produce a line of code. They rarely know. It seems to me that anyone who is serious about improving cost and schedule estimates needs accurate data from past projects.
In cases in which the client doesn’t know the cost, I incorporate the task of capturing effort data into the retrospective process as a benefit to the client, but effort data also is useful to me. As the retrospective leader, I know that solid numbers on effort and cost help me understand the size and difficulty of the project. As I work with my clients to collect the data, I look for answers to the following questions:
• What did this project really cost?
• How many people did it really take?
• How long did it really take?
• When did people really start and when did they leave or stop?
• How much actual work did the team accomplish?
• What level of quality did the product achieve?
• How did cost and schedule estimates compare with the actual results?
I like to collect as much useful information as possible before the retrospective begins, but I don’t expect miracles. I’m not looking for a Herculean effort from management or from team members who, while recuperating from the pressured end of the project, may resemble brain-dead zombies. I’m looking for coarse-grain data, something that can help with quick approximations, and one of my preferred pieces of data to collect is one I mentioned earlier—cost per line of code. For other types of effort data, I usually turn to the management team, the systems integrator, and the defects tracker.
The Management Team: Many of my questions can be answered by the management team. I ask managers to search through their records to come up with the most accurate numbers they have for cost, time, head count over time, and quality of work produced.
When preparing the management team for a retrospective, I discuss what questions we might ask project staff in order to obtain the most useful effort data. Every project is different, so there are no universally applicable data. Determining appropriate questions depends on such factors as the maturity of the team’s process, the project’s size, and the complexity of components (hardware, software, contractors, new technology, and third-party components, for example). Occasionally, managers will suggest collecting information that is impossible to get, but they usually back off when they realize that I’m expecting them to be responsible for obtaining the data.
The collection of management-oriented effort data seems to work best when one person is appointed to collect data about the whole project. This person can be a manager, but often someone who is interested in moving into management could be assigned the job as it provides a tremendous opportunity to learn about the details of managing.
Validation of future schedules is one reason I capture effort data, but I also want to see whether there are any interesting trends to investigate during the retrospective. Often when I contact a manager to find out how the data collection is going, I’m pleasantly surprised to discover that he or she has spent hours looking at many different aspects of the system, and has made some interesting discoveries.
With some clients, I discover that key project information is not available. This discovery tells me a lot about what kind of control these managers had over their project. During the retrospective, I raise the issue of data availability to see whether team members have ideas on how to capture better data on the next project.
The Systems Integrator: Usually, there is one person who stands out in a small group of people who can give a detailed view of the finished system through answers to the following questions:
• How many lines of code were built?
• How many routines were built?
• What were the sizes of the routines?
• What percentage of the code was acquired rather than built?
• Which routines had trouble going through systems integration?
• What patterns did you see?
The systems integrator, who also may be the person who directs the configuration management system, is the person who can answer these questions.
Sample Effort Data Collection E-mail
With the managers’ approval, I introduce myself to this person on the phone and explain what information and data I hope to capture. I usually follow the phone call with e-mail to summarize what we have discussed. A sample e-mail follows:
Howdy Kyle,
As we discussed, Ellen Banks has indicated that you are the person to help me collect some effort data on the recent BIM project. Most software organizations have a long-term optimistic memory. This means a project from the past is remembered to be easier than it actually was. The end of a project is a perfect time to figure out how much effort really went into the project. This effort-data collection may aid project members in “learning what to do differently next time,” but another benefit is a corporate memory of past projects. This corporate memory is useful as a reality check during future scheduling efforts. Comparing the expectations for the future against past projects is a good way to validate or question pending schedule plans.
Of course, sometimes this kind of information is just not available; in which case, we should have a discussion of what data we want to capture next time around. Here are some questions I’d like to pose to you. Please provide as much detail as possible.
1) What were the total number of lines of code produced in your project as reported by your compiler? (The compiler documentation might list lines of code as “program statements.”)
2) What were the meaningful “chunks” of work for which we want to/can capture effort information? (“Schedule-tracked activity” is a good chunk. Other chunks include work done by the Red Team, by the Blue Team, or by each specific person. If possible, get the information by the schedule-tracked activity item on the schedule.)
3) How many lines of code were there for each chunk?
4) How does the effort spent on this project compare with a similar project?
5) In terms of “lines of code,” what percent of code was not written from scratch and what effort did it take to incorporate this “foreign” code?
6) What was the calendar time and number of people required for each chunk? (This might be difficult information to capture. Sometimes, an activity changes over its life as people come and go. The chunk might have started out as one chunk and turned into three chunks, for example. The effort people put into that chunk might not be clear, because they “put in 60 hours per week.” Do the best you can. If there are a great number of changes in the schedule, to the point where it can’t be tracked, then that is interesting to look at in the retrospective, and precise numbers become less important than discussion about what to do differently. Likewise, if effort is not clear, and overwork seems to be a major problem, we need to address this in the retrospective. Also, for each chunk, please get a count of how many schedule slips occurred—at least the important ones.)
I expect that you will be able to answer Question 1 fairly easily. It would be great if you can also answer Questions 2 and 3. The answer to Question 4 might be harder to get. Frankly, I’ll be surprised if you can reconstruct enough information for it. Given the time frame until the retrospective, I doubt answers to Questions 5 and 6 are reasonable to expect, but please think about capturing whatever meaningful data you can.
By the way, I suggested the “lines of code” measure because compilers usually can answer that question. I know how inaccurate using such a metric is—so if you have a better metric, I’m open to it (for example, objects, use cases, or function points). What I’m looking for here is
• some measure of effort that the team can use to discover “interesting” things about this project
• questions that we know we cannot answer this time around, but that would be useful for us to have data about next time
• a demonstration that team members worked harder than they realize
These questions are guidelines. Brainstorm with your colleagues to formulate other interesting questions and then try to get answers. Let’s talk on the phone and figure out what really makes sense after you have had a chance to think about this. I’ll call you soon to see how things are going.
—Norm
I let the systems integrator or whoever is serving as data collector decide on the exact nature of the metrics he or she captures. Determining the best metrics to use depends on the technology. For example, if the team has been using objects, then useful metrics might be
• number of classes
• methods per class
• number of class instantiations
• profile of inheritance structure
• number of library classes used
On the other hand, a relational-database-intensive or framework-driven project might call for very different metrics.
If the systems integrator is comfortable with public speaking, I ask him or her to prepare and give a verbal report to the retrospective community. Otherwise, I present the results.
The Defects Tracker: On a project, there is usually someone who runs the bug-tracking system and makes sure that defect reports get entered and processed through various stages. This defects tracker generates summary reports, and generally knows what bugs are unresolved and which ones are real problems. This person also knows a great deal about what kinds of defects showed up over the life of the project, which defects recurred, which ones were difficult to fix, and which were deferred to be fixed at a later time—and why.
I meet with the defects tracker, and together we explore what kinds of questions to ask and what kinds of data to collect. From this information, I can get a sense of the quality of the project by studying the defects. While doing this, I need to be very careful not to single out or embarrass anyone. If this happens, then the defects-tracking system will never again contain valid data. In addition, I ask the defects tracker a battery of important but sensitive questions, the answers to which will be kept between the two of us.
Some examples of basic questions include
• What was the total number of reported defects?
• How many defects were repaired?
• How many defects were deferred?
• What was the pattern of defects over time in relation to the major milestones?
• How many releases were tested?
• How much time was allocated to test each release?
• How much confidence did the testers have in each release?
• How much time was scheduled for testing and how much time did it actually take?
Some examples of more sensitive questions include
• How many defects were created by each group or person?
• How many defects were found by each group or person?
• How many defects were found by each kind of test?
• As you look at the defects data now, what interesting patterns emerge?
• Did the severity of deferred defects change as a function of time or because of proximity to a milestone?
• Did the severity of deferred defects depend upon who made the decision to defer?
• Did the nature of reported defects change as a function of time or project phase?
• Did the nature of reported defects depend upon who was doing the testing?
After the defects tracker has studied the collection of data on defects, we decide together what information should be shared publicly and what information needs to be handled more carefully. In deciding what to disclose, keep in mind the following:
We are not out to blame anyone. We assume that all group members did the best they could, given what they knew at the time. Our goal here is to learn how to do a better job next time. What is the best way to make this happen?
Ready the Team
We have detailed how to prepare the managers for the retrospective. Now, let’s turn our attention to the team. Here is a review of the objectives that I, as a facilitator, want to pursue as part of the preparation effort:
• I want all participants to understand that a retrospective is a positive learning experience, not an opportunity for blame and counter-blame.
• I want participants to be confident that I will be able to lead this process, that I will understand the nuances of the discussion, and that I will provide a safe and confidential arena for them to talk about their experience.
• I want participants to be able to review the entire project. At the immediate end of a project, people can easily recall the prior few weeks, but the earlier months may be vague, at best. Participants need to review the entire project to be able to bring a complete perspective to the retrospective.
Three Sample Retrospective Handouts
In most instances, it is the managers who introduce the team to the idea of a retrospective. Since I am not usually on site at the time the retrospective is announced, I do as much as I can to help prepare the managers for the task. Some team members will find the thought of a review intimidating, and it’s important that their introduction to the idea be handled well. The managers need to explain what a retrospective is, why they are doing it, and when it will be held. I provide them with three handouts to augment the announcement. The first is an informational sheet explaining what a retrospective is, the second is my biographical sketch, and the third contains questions for the group to consider before coming to the retrospective.
Besides describing a retrospective, the first handout sets the tone that recognizes that project members did the best that they could and explains what people need to do to get ready.
What Is a Retrospective? How Should You Get Ready?
In the movie Dances with Wolves, a tribe of Native Americans celebrates the success of a buffalo hunt by telling and retelling the story of the hunt around a campfire. Telling the story is a very important ritual because it provides a lesson for all the hunts to come. It is the way wisdom is passed on. In the software engineering field, a retrospective works much the same way—its purpose is to help you review your most recent project, in order to understand what happened, what worked well, and what to do differently next time. It is not an activity of finding fault with anyone, but rather an activity for learning from our experiences.
To have gotten your project to its current stage, you undoubtedly expended a great deal of effort and made many sacrifices. In a sense, the effort and sacrifice are the tuition that you paid. Now the question is, What did you learn? Learning is what this retrospective is all about: It is about improving your practice by reflecting on your recent experiences.
During the retrospective, we will review the entire project from many different perspectives, using as much factual information as can be discovered. The review is much like an archaeological research project in that we want to learn and remember, in part, by collecting and analyzing “artifacts” from the project. By artifacts, I mean memos, meeting notes, old schedules (all of them), calendars, white papers, budgets, project plans, personnel loading charts, and so forth. Artifacts can be anything that will remind you of what happened during the project and when it happened.
A key artifact is your calendar from the beginning of the project, so please bring it. (Reviewing your calendar should help refresh your memory and may provide clues about where else to find artifacts.) In addition to bringing the calendar, please search for artifacts in your desk, briefcase, old e-mail files, notebooks, bug status reports, old conference room schedules, notice boards, the recycle pile, and wherever else you can think of that might help document the project.
At this moment, you are probably worn out and getting away is most likely a higher priority than looking at your recent past. Take some time for yourself—it’s important—and then please put some effort into this artifact dig. We want the next project to go better, and that will happen only if we learn from this one. There are two other reasons for putting effort into the artifact dig. First, we will analyze the artifacts to construct a time line of important events. Second, awards will be given for the most artifacts collected, the most creative or unusual artifact, and the most important artifact.
Please also answer the questions on the “Retrospective Pre-work Handout” and fax or e-mail it back to me. Your answers will help me get prepared, and might begin the review process for you.
Congratulations on getting to the end of the project! I look forward to working with you.
—Norm
Although each manager needs to announce the coming of the retrospective in his or her own way, distribution of my written explanation can reinforce the information that managers remember to convey to team members from their preliminary discussions with me.
The artifacts search I ask for in the handout makes the preparation a bit of a game and also encourages creativity. The best time for project members to begin the search is after the product has been released, but before the retrospective is held. The search has a beneficial side effect—team members clean up their desks! Let’s face it—developers are usually too tired to do much of anything else at the end of a project, and so clearing their desks to uncover artifacts is a suitable activity.
The true importance of the artifacts search will be discussed later, but a simple, early justification for the activity is that as people look for these artifacts and recall the stories behind them, they begin the process of remembering, discovering, and learning from the project. The search for artifacts is a subtle but significant activity that is actually the beginning of the review.
The memo acknowledges the fact that project members have worked hard, are tired, and probably have a few more important things to do than getting ready for what they undoubtedly perceive as “more work.” Giving them permission ostensibly to do something else (which is unusual in this business!) makes team members feel more relaxed, and frees them to search for artifacts in a leisurely, non-pressured manner. By this acknowledgment, I’m trying to establish a tone that says, “Whatever you can manage to do is okay.”
For the second handout, the biographical sketch, I intentionally use a folksy style so that I look less like a snooty expert and more like a regular Joe. I usually tailor this handout to fit with the team’s temperament and experiences as I understand them. The following sketch is similar to one I used with a team that was laid-back and a bit counterculture:
Who Is Norm Kerth?
I know there are a number of you whom I have not had the pleasure of meeting. I suppose you are wondering, “Who is this guy who is coming in to lead the retrospective?”
“After all,” you may be thinking, “I heard he’s over forty, never wears a tie, has a beard, and lives on a houseboat—but can he be trusted?”
Those details are true, but I guess I should tell you a bit more about myself. I graduated from Cal-Berkeley with a degree in EE and CS in 1974. While a student, I worked for Hewlett-Packard in Quality Assurance for the hand-held calculators division.
After leaving H-P, I spent ten years with Tektronix in Beaverton, Oregon, engineering software for test equipment, for integrated manufacturing test systems, and for embedded systems applications. Eventually, I moved to the research organization, to create, lead, and manage the Software Engineering Research Group. Our responsibility was to look at ways to improve software development. Issues we addressed included walkthroughs, methodologies, CASE tools, building reusable software components, object-oriented technologies, software-project scheduling, and effective software management.
I joined the faculty at the University of Portland in 1984, where I taught software engineering and operating systems courses. At the same time, I started my consulting practice, which I continue to this day. My goal in this consulting practice is to help companies get better at developing software on time, within budget, and in a manner that is both enjoyable and rewarding.
Oh, I guess I also should tell you that I am a Smalltalk bigot, but I recognize the fact that C++ and Java are here to stay. Frankly, I believe that which language you use is less important than your mastering the analysis, design, and quality-assurance processes.
I hope this answers a few questions about me. I look forward to trading stories about our experiences.
—Norm
The reason I tailor my description to fit the firm’s culture is that doing so helps me establish myself as a “safe” person. If I am going to be an effective facilitator, the participants need to see me as a trustworthy, strong, and competent leader who is capable of keeping discoveries confidential.
My third handout asks each participant to answer, in confidence, questions that will help me understand goals as well as the concerns any participant might have about the retrospective. The answers tell me where to probe to uncover problems specific to the project.
Retrospective Pre-work Handout
November 30 to December 2, I will be working with your group to review what went on during the BIM project. As I explained in my “What Is a Retrospective? How Should You Get Ready?” handout, the goal of this review is to understand what happened, what worked well, and what to do differently next time.
To help me prepare for this retrospective, I have a number of questions that I would like you to answer. You can respond by sending me a letter, e-mail, or a fax.
Norm Kerth 503-233-2163
P.O. Box 82907
Portland, OR E-mail to:
97282-0907 [email protected]
Your answers will be kept in strict confidence. I will review everyone’s comments and discuss any patterns I see, but no individual response will be singled out.
So, here are my questions:
1. For us to learn the most from this experience, what topics need to be discussed?
2. What do you hope can happen for you during the retrospective?
3. What long-term impact do you hope this retrospective will have?
4. What reservations, concerns, or worries do you have about this retrospective?
5. What emotions do you feel as you think about this meeting?
6. What else should I ask and how would you respond?
So that I can do a good job getting prepared, I need your answers by November 25. I thank you in advance for your effort.
—Norm
The act of answering the questions posed in the “Retrospective Pre-work Handout” helps the team members to organize information they think is important—whether it pertains to dreams, hopes, or purely practical concerns. This activity may lead them to find the courage for deeper sharing during the retrospective because the tough issues will have already been expressed on paper, and therefore it may seem less scary to discuss them publicly. Furthermore, answering the questions may prompt some team members to share views with “safe allies” before the retrospective, thereby increasing the probability that important issues will be raised during the review.
When to Get the Legal Department Involved
In the early stages of preparing, make sure you ask whether there is any litigation pending on the project. It’s a simple question, but the answer is very important if it is yes.
A True Story
I had a client who farmed out a major piece of work to a subcon-tractor. After months of craziness with nothing delivered, my client stopped the project and refused to make any payments to the subcontractor. Blame was high on both sides. A lawsuit looked very likely.
The subcontractor held a review of its portion of the project and delivered a less-than-honest appraisal of the situation to my client. Actually, the report was a whitewash carefully prepared by the subcontractor’s legal department.
With my client, I set about to hold a retrospective to see what could be learned about the client company’s interactions with subcontractors and about its own work. When my client’s attorney learned of our plans to hold a retrospective, he used his authority as legal counsel to cancel it just as we were starting. We were shocked, but the attorney explained that if we had held the review, everything we wrote or said would become “discoverable evidence.” The lawyers for the subcontractor would have had an easy job if we gave them a list titled “Things We Learned” and another one titled “What to Do Differently Next Time.”
However, canceling the retrospective was not the end of the story. The attorney recognized the benefit of spending the next three days with all the participants on the project talking about what really happened. Up until then, he had only heard the story from senior management. As legal counsel, he requested that all participants convene to help him prepare for the case. All discussions were to be considered privileged communication between attorney and client and only the attorney could take notes.
We held the retrospective as we had planned, the only difference being that sessions were held exclusively in the presence of the attorney. The retrospective report was produced by the legal staff with the help of the participants. Neither the report nor the list of things learned could be shared with people outside the project until the litigation was concluded, and all documents generated for or during the retrospective were held confidential by the attorney.
In the end, the attorney discovered so many “smoking guns” that he could use in court that he was sure he could get the other side to back down. He was handed artifacts containing telephone logs of conversations proving cover-ups of poor quality, subcontractor memos explaining that it was “hacking junk and delivering it simply to solve an internal cash-flow problem,” a written request from the subcontractor that my client’s QA group “soften defect reports in order to assure that the project stay on schedule,” and numerous other reports of points at which the specifications in the contract were not followed by the subcon-tractor. He had proof that the subcontractor had not used the methodology required by the contract.
The attorney’s reaction to the retrospective was a glowing testimonial: “It was an amazingly efficient way to get prepared for my case. For once in my life, I felt I was getting an honest and complete picture of a situation. I think retrospectives might become a tool I use on even some of my toughest cases.”
Track Details with a Checklist
Retrospectives are complex activities. To be successful, you need to focus your attention on the larger issues—the people, the story, and the process, and you may feel you don’t have time to deal with the smaller details. Be forewarned, however, that a neglected detail can ruin a retrospective.
To prevent my overlooking something small but important, I’ve developed a checklist to remind me of the activities that need to happen but that I don’t want to spend a lot of time monitoring. Every facilitator’s checklist will probably be unique in some regard, but here’s mine to use as a guideline:
Retrospective Checklist
• Five flip-chart stands with pads (or, one flip-chart stand and pad for every five attendees)
When we break into sub-teams, each team will need its own place to work. Sub-team work will be centered around a flip chart. When we prepare the time line, we will be collecting information from five categories.
• 100 feet of butcher paper about 36″ wide to use for the time line
You can get this at most art-supply stores for about $.25 per yard.
• Five-to-six rolls of masking tape (or whatever adhesive the meeting-room owner prefers be used on the walls)
Facility managers may have specific requirements for how they will permit paper to be hung on their walls. Find out whether hanging paper is an issue and what to do if it is. Make sure in advance that the room will have a space where the time line can be hung (30′ to 50′ long and 5′ to 6′ high).
• 1,000 5″ × 8″ file cards
• One 9″ × 12″ (or larger) envelope that can be used to save the “to do” lists
• Marker pens in a variety of colors, with at least one pen for each person
• One box of tissues
• One pair of scissors
• One package of note pads
• One box of pens
• Nametags
• Awards for the artifacts hunt and “Repair Damage Through Play” Exercise (such as certificates or trophies)
• CD Player and CD’s (for music during the breaks)
• Instant camera and six packages of film (to record moments that should be remembered)
• A copy of this handbook (to refer to for ideas)
• VCR, television, and the videotape Flight of the Phoenix (to be used for the “Passive Analogy” Exercise)
• Instruction list for the manager of the facility to explain how you want the room arranged
• Copies of “Preparing a Proposal for Management”
• Select start and end times, and make sure you can have access to the room an hour before and after sessions are scheduled
Some facilities prefer to control access to the rooms between sessions. Occasionally, this can become a serious problem. Also, consider how confidential this retrospective is and whether the room needs to be secure during off hours.
• Special-meals list; smoking and non-smoking room assignments
Be sure the facility’s manager knows in advance and is handling individual requests for room and meal preferences.
When to Arrive for the Retrospective
Although I always assign pre-retrospective work to the participants, only about 40 to 60 percent of team members actually complete the work. This is a fact of life, so I plan to arrive a day early to walk around the office, talk with individuals, and get a better sense of what occurred during the project.
For those who didn’t do the pre-retrospective work, I let them know that I understand that they have other commitments, but I then say, “Now would be a good time for you to tell me your thoughts and any observations about the project.” For those who did submit pre-retrospective work, I discuss it with them. They appreciate the fact that I have read their comments when they see my yellow highlighting marks. Occasionally, when an exceptionally good piece has been sent to me—a clever poem or song, for example—I ask the author whether I can share it with the rest of the participants.
During this walk-around, I remind everyone about the hunt for artifacts, and generally let people get to know me. This getting-acquainted aspect may seem insignificant, but it is very important. It is during this time that I hear people’s most private concerns.
Sometimes, participants want me to speak for them, to present the issues they are afraid to raise. I carefully explain that in my role as facilitator I will help get the story out, but I will not be a conduit for the anonymous submission of issues. I clarify that I do not know the whole background and cannot speak accurately for others, but that it is my job to find a safe way to allow issues to surface during the retrospective. I ask team members to work with me, but encourage them to share only what feels comfortable.
This kind of casual conversation can be very informative. If I hear about issues in confidence that do not come out in the retrospective, then I know the group has a problem sharing sensitive topics, and I will raise this as a major problem for the team to explore during the review.