Chapter 19
Prepare for the Next Spiral (Act)
You will recall that in the spiral development environment, software testing is described as a continuous improvement process that must be integrated into a rapid application development methodology. Deming’s continuous improvement process using the PDCA model is applied to the software testing process. We are now in the Act part of the spiral model (see Figure 19.1), which prepares for the next spiral.
Figure 19.2 outlines the steps and tasks associated with the Act part of spiral testing. Each step and task are described, and valuable tips and techniques are provided.
Step 1: Refine the Tests
See Appendix F21, “Impact Analysis Checklist,” which can be used to help analyze the impacts of changes to the system.
Task 1: Update the Function/GUI Tests
The objective of this task is to update the test design to reflect the new functional requirements. The Test Change Function Test Matrix, which cross-references the tests to the functions, needs to be updated. The new functions are added in the vertical list, and the respective test cases are added to the horizontal list. The test case name is recorded on the matrix along with the number. (See Appendix E5, “Function/Test Matrix.”)
Figure 19.1 Spiral testing and continuous improvement.
Figure 19.2 Prepare for the next spiral (steps/tasks).
Next, any new GUI/function test cases in the matrix need to be documented or scripted. The conceptual test cases are then transformed into reusable test scripts with test data created. Also, any new GUI requirements are added to the GUI tests. (See Appendix E7, “GUI-Based Functional Test Matrix.”)
Finally, the tests that can be automated with a testing tool need to be updated. Automated tests provide three benefits: repeatability, leverage, and increased functionality. Repeatability enables automated tests to be executed more than once, consistently. Leverage comes from repeatability from tests previously captured and tests that can be programmed with the tool, which might not have been possible without automation. As applications evolve, more and more functionality is added. With automation, the functional coverage is maintained with the test library.
Task 2: Update the System Fragment Tests
In a prior task, the system fragment tests were defined. System fragment tests are sample subsets of full system tests that can be performed during each spiral loop. The objective of performing a fragment test is to provide early warning of pending problems that would arise in the full system test.
Candidate fragment system tests include function, performance, security, usability, documentation, and procedure. Some of these fragment tests should have formal tests performed during each spiral, whereas others should be part of the overall testing strategy. The objective of the present task is to update the system fragment tests defined earlier based on new requirements. New baseline measurements are defined.
Finally, the fragment system tests that can be automated with a testing tool need to be updated.
Task 3: Update the Acceptance Tests
In Chapter 15, the initial list of acceptance tests was defined. Acceptance testing is an optional user-run test that demonstrates the ability of the application to meet the user’s requirements. The motivation for this test is to demonstrate rather than be destructive, that is, to show that the system works. If performed, acceptance tests typically are a subset of the system tests. However, the users sometimes define “special tests,” such as intensive stress or volume tests, to stretch the limits of the system even beyond what was tested during the system test. The objective of the present task is to update the acceptance tests defined earlier on the basis of new requirements.
Finally, the acceptance tests that can be automated with a testing tool need to be updated.
Step 2: Reassess the Team, Procedures, and Test Environment
Task 1: Evaluate the Test Team
Between each spiral, the performance of the test team needs to be evaluated in terms of its quality and productivity. The test team leader directs one or more testers to ensure that the right skill level is on the project. He or she makes sure that the test cases are being executed according to the plan, the defects are being reported and retested, and the test automation is successful. The basis for allocating dedicated testing resources is the scope of the functionality and the development time frame. If the testing is not being completed satisfactorily, the team leader needs to counsel one or more team members or request additional testers. On the other hand, if the test is coming to a conclusion, the testing manager needs to start thinking about reassigning testers to other projects.
Task 2: Review the Test Control Procedures
In Chapter 14, the test control procedures were set up before the first spiral. The objective of this task is to review those procedures and make appropriate modifications. The predefined procedures include the following:
■ Defect recording/tracking procedures
■ Change request procedures
■ Version control procedures
■ Configuration build procedures
■ Project issue resolution procedures
■ Reporting procedures
The purpose of defect recording/tracking procedures is to record and correct defects and record metric information about the application. As the project progresses, these procedures may need tuning. Examples include new status codes or new fields in the defect-tracking form, an expanded defect distribution list, and the addition of more verification checks.
The purpose of change request procedures is to allow new change requests to be communicated to the development and testing team. Examples include a new change control review board process, a new sponsor who has ideas of how the change request process should be implemented, a new change request database, and a new software configuration management tool.
The purpose of version control procedures is to uniquely identify each software component via a labeling scheme and allow for successive revisions. Examples include a new software configuration management tool with a new versioning scheme or new labeling standards.
The purpose of configuration build procedures is to provide an effective means of assembling a software system from the software source components into executable components. Examples include the addition of a new 4GL language, a new software configuration management tool, or a new delta build approach.
The purpose of project issue resolution procedures is to record and process testing issues that arise during the testing process. Examples include a new project manager who requests a Lotus Notes approach, a newly formed issue review committee, an updated issue priority categorization scheme, and a new issue submission process.
The purpose of reporting procedures is to facilitate the communication process and reporting. Examples include a new project manager who requires weekly testing status reports, a new interim test report structure, or an expanded reporting distribution.
Task 3: Update the Test Environment
In Chapter 15, the test environment was defined. A test environment provides a physical framework for testing necessary for the testing activity. During the present task, the test environment needs are reviewed and updated. (See Appendix F22, “Environment Readiness Checklist,” which can be used to verify the readiness of the environment for testing before starting test execution.)
The main components of the test environment include the physical test facility, technologies, and tools. The test facility component includes the physical setup. The technologies component includes hardware platforms, the physical network and all its components, operating system software, and other software, such as utility software. The tools component includes any specialized testing software, such as automated test tools, testing libraries, and support software. Examples of changes to the test environment include the following:
■ Expanded test laboratory
■ New testing tools required
■ Additional test hardware required
■ Additional network facilities
■ Additional test database space required
■ New Lotus Notes log-ons
■ Additional software to support testing
Step 3: Publish Interim Test Report
Task 1: Publish the Metric Graphics
Each spiral should produce an interim report to describe the status of the testing. These tests are geared to the testing team, the test manager, and the development manager, and will help them make adjustments for the next spiral. The following minimal graphical reports are recommended between each spiral test.
Test Case Execution Status
Figure 19.3 shows the status of testing and predicts when the testing and development group will be ready for production. Test cases run with errors have not yet been corrected.
If there are a relatively large number of test cases that have not been run, the testing group needs to increase its productivity or resources. If there are a large number of test cases run with errors that have not been corrected, the development team also needs to be more productive.
Figure 19.3 Test execution status.
Defect Gap Analysis
Figure 19.4 shows the gap between the number of defects that has been uncovered compared to the number that has been corrected. A large gap indicates that development needs to increase effort and resources to correct defects more quickly.
Defect Severity Status
Figure 19.5 shows the distribution of the three severity categories: critical, major, and minor. A large percentage of defects in the critical category indicates that a problem with the design or architecture of the application may exist.
Test Burnout Tracking
Figure 19.6 indicates the rate of uncovering defects. The cumulative, for example, running total number of defects and defects by time period help predict when fewer defects are being discovered. This is indicated when the cumulative curve “bends,” and the defects by time period approach zero.
Figure 19.4 Defect gap analysis.
Figure 19.5 Defect severity status.
Figure 19.6 Test burnout tracking.