Quality Control and Integrated Testing

This section addresses test execution—another aspect of the quality control process. The integrated approach to testing supports the goals of quality control, summarized here as keeping errors out of the development process and preventing errors from reaching the customer.

Traditionally, functional characteristics separated test organizations from development organizations. The partition encouraged organizational silos that did not share resources or knowledge across functional lines. The increasing expectations for organizational efficiency and the growing focus on compliance with regulatory standards are good reasons to consider merging the development and test methodologies and processes.

Integrated Testing

Integrating testing methodology and development methodology into a single methodology is neither a new concept nor is the implementation particularly common. Regardless of the reasons for change, no merger will be successful without a business case and executive-level support.

One approach to an integrated test and development methodology is to incorporate the testing steps and tasks into the development process by adding or modifying the development tasks. Usually, the person who is responsible for the development methodology integrates testing processes. If no one is directly responsible for the development methodology, then the test manager assumes responsibility for integrating the processes.

Another approach is to make the integration part of the quality assurance function. Executive management should not tell the quality assurance team which aspects of the software testing standard should be adopted; how that testing methodology should be integrated into the existing design methodology; and the amount of time and effort needed to perform the task. Executive management should set the tone by stating that the integrated approach will be the basis for testing in the organization.

Step 1: Organize the Test Team

Creating a combined methodology begins with organizing the team for the task. To make a new methodology work, key people who understand testing and development must be appointed to manage it. The group should consist of three to seven individuals who are respected by their peers. With fewer than three members, the interaction and energy necessary to successfully introduce the testing methodology may not occur. With more than seven members, management of the team becomes unwieldy. An experienced chairperson works with the executive sponsor to sanction the team’s mission. The project sponsor should ensure that the test management team:

1. ■ Understands testing concepts and the standard for software testing discussed in this manual.

2. ■ Customizes and integrates the standard for software testing into the organization’s systems design and maintenance methodology.

3. ■ Encourages adherence to and support of the integrated test methodology, and agrees to perform testing in the manner prescribed by the methodology.

Step 2: Identify the Tasks to Integrate

Section 3 describes different development and test methodologies that reflect the relationship between the business technology, system architecture, and organizational structure. The same relationships must be evaluated to determine whether or not to integrate the design methodology. When the team performs this step, the members must arrive at a consensus on the general objectives of testing and how the design methodology affects the test methodology. The design methodology may be addressed by design standards, so the team may accept the design tasks as part of the integrated methodology, or the team may decide that the design methodology and tasks remain outside the integrated methodology.

Step 3: Customize Test Steps and Tasks

The team should customize the steps and tasks covered in this text so that they are consistent with the organization’s development and test methodologies. The team can either perform the customization itself or assign it to others (e.g., to the group in charge of design methodology).

Customization usually includes the following:

1. ■ Standardizing vocabulary—Vocabulary should be consistent throughout the design methodology. If staff members understand and use the same vocabulary, they can easily move from job to job within the organization. Vocabulary customization may mean changing vocabulary in the testing standard or integrating the testing vocabulary into the systems development methodology.

2. ■ Changing the structure of presentation—The way the testing steps and tasks have been described may differ from the way other parts of the design methodology are presented. For example, this manual has separate sections for forms and text descriptions of the software testing tools. If the systems development methodology integrates them into single units, they may need to be rearranged or reordered to make them consistent with the development manual.

During test planning, the test team will determine which test standards, procedures, tasks, worksheets, and checklists are applicable to the system being developed. The team should customize the process for either individual application systems, or for a particular development function.

The team can also choose to create a smaller version of the process for the purpose of validating whether or not the process worked as expected.

Step 4: Select Integration Points

This step involves selecting where to integrate the test steps and tasks into the development methodology. This step requires a thorough understanding of the development methodology and tasks. The two key criteria for determining where to integrate these tasks are the following:

1. ■ What data is needed—The test task can be inserted into the design methodology only after the point at which the needed information has been produced.

2. ■ Where the test products are needed—The testing tasks must be completed before the products produced by that task are needed in the systems development methodology.

Applying these criteria will determine both the earliest and latest points at which the tasks can be performed. The tasks should be inserted into the development methodology at these points.

Step 5: Modify the Development Methodology

At this point, all of the information is that is needed to modify the systems development methodology is available. This step requires someone who is familiar with the design process; he or she inserts the test processes and steps into the development methodology documentation.

Step 6: Test Methodology Training

This step involves training analysts, users, and programmers in use of the test methodology. Once testing is integrated into the systems development methodology, people must be trained and motivated to use the test methodology, a more difficult job.

Test management team members play an important role in convincing their peers to accept and use the new methodology—first, by their example, and second, by actively encouraging coworkers to adopt the methodology. An important part of this step is creating and conducting testing seminars that should cover the following:

1. ■ Testing concepts and methods—This part of the training recaps the material in Appendix F.

2. ■ Test standards—Individuals responsible for testing must know the standards against which they are measured. The standards should be taught first, so team members know why they are performing certain tasks (e.g., test procedures), and the procedures second. If they feel that the procedures are just one way of testing, they may decide there are better ways. On the other hand, if they know the purpose of performing the test procedures (e.g., meeting test standards), they are more likely to take an interest in learning and following the test procedures.

3. ■ Test methodology—The methodology incorporated into the systems development methodology should be taught step by step and task by task. An analyst, user, or programmer should initially perform tasks under the direction of an instructor. This helps ensure that these professionals fully understand how the task should be performed and what results should be expected.

Until the individuals responsible for testing have been trained and have demonstrated proficiency in testing processes, management should allow for some testing errors. In addition, until individuals have demonstrated mastery of the test procedures, they should be closely supervised during the execution of those procedures.

The next part of this section presents a procedure for defect recording and analysis when the testing process is integrated into the development methodology. This procedure requires categorizing defects and ensuring that they are appropriately recorded throughout the development methodology.

Step 7: Incorporate Defect Recording

The quality control function is an integral part of the tester’s workbench. Defects must be recorded and analyzed to determine how to improve the integrated process. This process is the equivalent of problem reporting in operational application systems. The test manager must be able to capture information about the problems or defects that occur; without this information, it is difficult to improve testing.

The most difficult part of defect recording is convincing development staff members that this information will not be used against them. This information is gathered strictly for the improvement of the test process and should never be used for performance appraisals or any other individual evaluations.

The Integrated Team

Although the project and test managers are the people who focus on completing the project within the constraints of the project budget and schedule, the outcome of the execution phase has always been in the hands of the developers and testers. The integrated team approach does not change that condition. The change that will come, if the integration is done for the right reasons and with the best interest of quality in mind, is the opportunity for developers to understand how testers think, and for testers to become more mindful of the developers’ perspective.

In the course of pooling our technical knowledge, the greatest challenge will remain unchanged—understanding the customer’s point of view.