Introduction

Numerous textbooks address software testing in a structured development environment. By “structured” is meant a well-defined development cycle in which discretely defined steps provide measurable outputs at each step. It is assumed that software testing activities are based on clearly defined requirements and software development standards, and that those standards are used to develop and implement a plan for testing. Unfortunately, this is often not the case. Typically, testing is performed against changing, or even wrong, requirements.

This text aims to provide a quality framework for the software testing process in traditional structured as well as unstructured environments. The goal is to provide a continuous quality improvement approach to promote effective testing methods and provide tips, techniques, and alternatives from which the user can choose.

The basis of the continuous quality framework stems from Edward Deming’s quality principles. Deming was the pioneer in quality improvement, which helped turn Japanese manufacturing around. Deming’s principles are applied to software testing in the traditional “waterfall” and rapid application “spiral (or agile)” development (RAD) environments. The waterfall approach is one in which predefined sequential steps are followed with clearly defined requirements. In the spiral approach, these rigid sequential steps may, to varying degrees, be lacking or different.

Section 1, “Software Quality in Perspective,” reviews modern quality assurance principles and best practices. It provides the reader with a historical sketch of software testing, followed by a description of how to transform requirements to test cases when there are well-defined or not so well-defined requirements. Basic software testing techniques are discussed, followed by an introduction to Deming’s concept of quality through a continuous improvement process. The Plan, Do, Check, Act (PDCA) quality wheel is applied to the software testing process.

The Plan step of the continuous improvement process starts with a definition of the test objectives, or what is to be accomplished as a result of testing. The elements of a test strategy and test plan are described. A test strategy is a concise statement of how to meet the goals of testing and precedes test plan development. The outline of a good test plan is provided, including an introduction, the overall plan, testing requirements, test procedures, and test plan details.

The Do step addresses how to design or execute the tests included in the test plan. A cookbook approach describes how to perform component, integration, and system acceptance testing in a spiral environment.

The Check step emphasizes the importance of metrics and test reporting. A test team must formally record the results of tests and relate them to the test plan and system objectives. A sample test report format is provided, along with several graphic techniques.

The Act step of the continuous improvement process provides guidelines for updating test cases and test scripts. In preparation for the next spiral, suggestions for improving the people, process, and technology dimensions are provided.

Section 2, “Waterfall Testing Review,” reviews the waterfall development methodology and describes how continuous quality improvement can be applied to the phased approach through technical reviews and software testing. The requirements, logical design, physical design, program unit design, and coding phases are reviewed. The roles of technical reviews and software testing are applied to each. Finally, the psychology of software testing is discussed.

Section 3, “Spiral Software Testing Methodology,” contrasts the waterfall development methodology with the rapid application spiral environment from a technical and psychological point of view. A spiral testing approach is suggested when the requirements are rapidly changing. A spiral methodology is provided, and broken down into parts, steps, and tasks, applying Deming’s continuous quality improvement process in the context of the PDCA quality wheel.

Section 4, “Project Management Methodology,” describes the practices and methods of software testing by describing basic test management processes and organizational approaches that achieve project quality. A Project Framework is outlined to unite quality processes with project phases, and synchronize project quality management with the system, or software—the development approach.

Section 5, “Emerging Specialized Areas in Testing,” describes how modern software testing must view the whole target business holistically, assuring that the pieces of that business process interact according to customers’ expectations. The focus of this section is to discuss other forms of nonfunctional testing such as performance, usability, interoperability, etc. Also discussed are how to evaluate software testing processes, how to set up an automation framework, steps to implement Service Orientated Architecture (SOA) testing, the building blocks of a Testing Center of Excellence (COE), how to test in an Agile development environment, and how to evaluate on-site versus offshore alternatives.

Section 6, “Modern Software Testing Tools,” describes futuristic software testing tools and trends. Next, a list of up-to-date commercial and open-source free software tools is provided, followed by guidelines on when to consider and when not to consider a testing tool. Also provided is a checklist for selecting testing tools, consisting of a series of questions and responses. A detailed methodology for evaluating testing tools is provided, ranging from the initial test goals through training and implementation.