At the beginning of the iteration, look at your stories and decide whether there are any aspects that programmers might misunderstand. You don’t need to provide examples for everything. Customer tests are for communication, not for proving that the software works. (See No Bugs” in Chapter 7.)

For example, if one of your stories is “Allow invoice deleting,” you don’t need to explain how invoices are deleted. Programmers understand what it means to delete something. However, you might need examples that show when it’s OK to delete an invoice, especially if there are complicated rules to ensure that invoices aren’t deleted inappropriately.

Once you’ve identified potential misunderstandings, gather the team at a whiteboard and summarize the story in question. Briefly describe how the story should work and the rules you’re going to provide examples for. It’s OK to take questions, but don’t get stuck on this step.

For example, a discussion of invoice deletion might go like this:

After a brief discussion of the rules, provide concrete examples that illustrate the scenario. Tables are often the most natural way to describe this information, but you don’t need to worry about formatting. Just get the examples on the whiteboard. The scenario might continue like this:

Your discussion probably won’t be as smooth and clean as in this example. As you discuss business rules, you’ll jump back and forth between describing the rules and demonstrating them with examples. You’ll probably discover special cases you hadn’t considered. In some cases, you might even discover whole new categories of rules you need customer tests for.

One particularly effective way to work is to elaborate on a theme. Start by discussing the most basic case and providing a few examples. Next, describe a special case or additional detail and provide a few more examples. Continue in this way, working from simplest to most complicated, until you have described all aspects of the rule.

You don’t need to show all possible examples. Remember, the purpose here is to communicate, not to exhaustively test the application. You only need enough examples to show the differences in the rules. A handful of examples per case is usually enough, and sometimes just one or two is sufficient.

When you’ve covered enough ground, document your discussion so the programmers can start working on implementing your rules. This is also a good time to evaluate whether the examples are in a format that works well for automated testing. If not, discuss alternatives with the programmers. The conversation might go like this:

Don’t formalize your examples too soon. While you’re brainstorming, it’s often easiest to work on the whiteboard. Wait until you’ve worked out all the examples around a particular business rule (or part of a business rule) before formalizing it. This will help you focus on the business rule rather than formatting details.

In some cases, you may discover that you have more examples and rules to discuss than you realized. The act of creating specific examples often reveals scenarios you hadn’t considered. Testers are particularly good at finding these. If you have a lot of issues to discuss, consider letting some or all of the programmers get started on the examples you have while you figure out the rest of the details.

Programmers, once you have some examples, you can start implementing the code using normal test-driven development. Don’t use the customer tests as a substitute for writing your own tests. Although it’s possible to drive your development with customer tests—in fact, this can feel quite natural and productive—the tests don’t provide the fine-grained support that TDD does. Over time, you’ll discover holes in your implementation and regression suite. Instead, pick a business rule, implement it with TDD, then confirm that the associated customer tests pass.

One of the most common mistakes in creating customer tests is describing what happens in the user interface rather than providing examples of business rules. For example, to show that an account rep must not delete a mailed invoice, you might make the mistake of writing this:

What happened to the core idea? It’s too hard to see. Compare that to the previous approach:

Good examples focus on the essence of your rules. Rather than imagining how those rules might work in the application, just think about what the rules are. If you weren’t creating an application at all, how would you describe those rules to a colleague? Talk about things rather than actions. Sometimes it helps to think in terms of a template: “When (scenario X), then (scenario Y).”

It takes a bit of practice to think this way, but the results are worth it. The tests become more compact, easier to maintain, and (when implemented correctly) faster to run.

Team members, watch out for a common pitfall in customer testing: no customers! Some teams have programmers and testers do all the work of customer testing, and some teams don’t involve their customers at all. In others, a customer is present only as a mute observer. Don’t forget the “customer” in “customer tests.” The purpose of these activities to bring the customer’s knowledge and perspective to the team’s work. If programmers or testers take the reins, you’ve lost that benefit and missed the point.

In some cases, customers may not be willing to take the lead. Programmers and testers may be able to solve this problem by asking the customers for their help. When programmers need domain expertise, they can ask customers to join the team as they discuss examples. One particularly effective technique is to ask for an explanation of a business rule, pretend to be confused, then hand a customer the whiteboard marker and ask him to draw an example on the board.

Programmers may use any tool they like to turn the customers’ examples into automated tests. Ward Cunningham’s Fit (Framework for Integrated Test),^[41] is specifically designed for this purpose. It allows you to use HTML to mix descriptions and tables, just as in my invoice auditing example, then runs the tables through programmer-created fixtures to execute the tests.

Fit is a great tool for customer tests because it allows customers to review, run, and even expand on their own tests. Although programmers have to write the fixtures, customers can easily add to or modify existing tables to check an idea. Testers can also modify the tests as an aid to exploratory testing. Because the tests are written in HTML, they can use any HTML editor to modify the tests, including Microsoft Word.

Programmers, don’t make Fit too complicated. It’s a deceptively simple tool. Your fixtures should work like unit tests, focusing on just a few domain objects. For example, the invoice auditing example would use a custom ColumnFixture. Each column in the table corresponds to a variable or method in the fixture. The code is almost trivial (see Example 9-1).

  public class InvoiceAuditingFixture : ColumnFixture
  {
      public InvoiceStatus Sent;
      public UserRole User;

      public Permission OkayToDelete() {
          InvoiceAuditer auditer = new InvoiceAuditer(User, InvoiceStatus)
          return auditer.DeletePermission;
      }
  }

Using Fit in this way requires a ubiquitous language and good design. A dedicated domain layer with Whole Value objects^[42] works best. Without it, you may have to write end-to-end tests, with all the challenges that entails. If you have trouble using Fit, talk to your mentor about whether your design needs work.

When do programmers run the customer tests?

Once the tests are passing, make them a standard part of your 10-minute build. Like programmers’ tests, you should fix them immediately if they ever break.

Should we expand the customer tests when we think of a new scenario?

Absolutely! Often, the tests will continue to pass. That’s good news; leave the new scenario in place to act as documentation for future readers. If the new test doesn’t pass, talk with the programmers about whether they can fix it with iteration slack or whether you need a new story.

What about acceptance testing (also called functional testing)?

Automated acceptance tests tend to be brittle and slow. I’ve replaced acceptance tests with customer reviews (see Customer review” later in this chapter) and a variety of other techniques (see A Little Lie” in Chapter 3).

When you use customer tests well, you reduce the number of mistakes in your domain logic. You discuss domain rules in concrete, unambiguous terms and often discover special cases you hadn’t considered. The examples influence the design of the code and help promote a ubiquitous language. When written well, the customer tests run quickly and require no more maintenance than unit tests do.

Don’t use customer tests as a substitute for test-driven development. Customer tests are a tool to help communicate challenging business rules, not a comprehensive automated testing tool. In particular, Fit doesn’t work well as a test scripting tool—it doesn’t have variables, loops, or subroutines. (Some people have attempted to add these things to Fit, but it’s not pretty.) Real programming tools, such as xUnit or Watir, are better for test scripting.

In addition, customer tests require domain experts. The real value of the process is the conversation that explores and exposes the customers’ business requirements and domain knowledge. If your customers are unavailable, those conversations won’t happen.

Finally, because Fit tests are written in HTML, Fit carries more of a maintenance burden than xUnit frameworks do. Automated refactorings won’t extend to your Fit tests. To keep your maintenance costs down, avoid creating customer tests for every business rule. Focus on the tests that will help improve programmer understanding, and avoid further maintenance costs by refactoring your customer tests regularly. Similar stories will have similar tests: consolidate your tests whenever you have the opportunity.

Some teams have testers, not customers, write customer tests. Although this introduces another barrier between the customers’ knowledge and the programmers’ code, I have seen it succeed. It may be your best choice when customers aren’t readily available.

Customer tests don’t have to use Fit or FitNesse. Theoretically, you can write them in any testing tool, including xUnit, although I haven’t seen anybody do this.

Fit for Developing Software [Mugridge & Cunningham] is the definitive reference for Fit.

“Agile Requirements” [Shore 2005a], online at http://www.jamesshore.com/Blog/Agile-Requirements.html, is a series of essays about agile requirements, customer testing, and Fit.