Contents
SECTION 1 SOFTWARE QUALITY IN PERSPECTIVE
1 A Brief History of Software Testing
Historical Software Testing and Development Parallels
Evolution of Automated Testing Tools
Static Capture/Replay Tools (without Scripting Language)
Static Capture/Replay Tools (with Scripting Language)
Verification versus Validation
Components of Quality Assurance
Software Configuration Management
Elements of Software Configuration Management
Software Quality Assurance Plan
Steps to Develop and Implement a Software Quality Assurance Plan
Step 2: Obtain Management Acceptance
Step 3: Obtain Development Acceptance
Step 4: Plan for Implementation of the SQA Plan
Capability Maturity Model (CMM)
Malcolm Baldrige National Quality Award
3 Overview of Testing Techniques
Black-Box Testing (Functional)
White-Box Testing (Structural)
Gray-Box Testing (Functional and Structural)
Manual versus Automated Testing
Taxonomy of Software Testing Techniques
4 Transforming Requirements to Testable Test Cases
Software Requirements as the Basis of Testing
Numerical Method for Evaluating Requirement Quality
Process for Creating Test Cases from Good Requirements
Step 1: Review the Requirements
Step 3: Identify the Test Suite
Step 5: Write Test Case Descriptions and Objectives
Transforming Use Cases to Test Cases
Step 1: Draw a Use Case Diagram
Step 2: Write the Detailed Use Case Text
Step 3: Identify Use Case Scenarios
Step 4: Generating the Test Cases
What to Do When Requirements Are Nonexistent or Poor?
Advantages and Disadvantages of Ad Hoc Testing
The Art of Exploratory Testing
Advantages and Disadvantages of Exploratory Testing
5 Quality through Continuous Improvement Process
Deming’s 14 Quality Principles
Point 1: Create Constancy of Purpose
Point 2: Adopt the New Philosophy
Point 3: Cease Dependence on Mass Inspection
Point 4: End the Practice of Awarding Business on Price Tag Alone
Point 5: Improve Constantly and Ceaselessly the System of Production and Service
Point 6: Institute Training and Retraining
Point 9: Break Down Barriers between Staff Areas
Point 10: Eliminate Slogans, Exhortations, and Targets for the Workforce
Point 11: Eliminate Numerical Goals
Point 12: Remove Barriers to Pride of Workmanship
Point 13: Institute a Vigorous Program of Education and Retraining
Point 14: Take Action to Accomplish the Transformation
Continuous Improvement through the Plan, Do, Check, Act Process
SECTION 2 WATERFALL TESTING REVIEW
Waterfall Development Methodology
Continuous Improvement “Phased” Approach
Psychology of Life-Cycle Testing
Software Testing as a Continuous Improvement Process
The Testing Bible: Software Test Plan
Major Steps in Developing a Test Plan
Step 1: Define the Test Objectives
Step 2: Develop the Test Approach
Step 3: Define the Test Environment
Step 4: Develop the Test Specifications
Step 6: Review and Approve the Test Plan
Technical Reviews as a Continuous Improvement Process
Motivation for Technical Reviews
Step 1: Plan for the Review Process
Step 3: Develop the Review Agenda
Step 4: Create a Review Report
7 Static Testing the Requirements
Testing the Requirements with Ambiguity Reviews
Testing the Requirements with Technical Reviews
Requirements Traceability Matrix
Building the System/Acceptance Test Plan
8 Static Testing the Logical Design
Data Model, Process Model, and the Linkage
Testing the Logical Design with Technical Reviews
Refining the System/Acceptance Test Plan
9 Static Testing the Physical Design
Testing the Physical Design with Technical Reviews
Creating Integration Test Cases
Methodology for Integration Testing
Step 1: Identify Unit Interfaces
Step 2: Reconcile Interfaces for Completeness
Step 3: Create Integration Test Conditions
Step 4: Evaluate the Completeness of Integration Test Conditions
10 Static Testing the Program Unit Design
Testing the Program Unit Design with Technical Reviews
11 Static Testing and Dynamic Testing the Code
Testing Coding with Technical Reviews
SECTION 3 SPIRAL (AGILE) SOFTWARE TESTING METHODOLOGY: PLAN, DO, CHECK, ACT
12 Development Methodology Overview
Limitations of Life-Cycle Development
Psychology of Client/Server Spiral Testing
Project Goal: Integrate QA and Development
Iterative/Spiral Development Methodology
Methodology for Developing Prototypes
Step 2: Demonstrate Prototypes to Management
Step 3: Demonstrate Prototype to Users
Step 4: Revise and Finalize Specifications
Step 5: Develop the Production System
Continuous Improvement “Spiral” Testing Approach
13 Information Gathering (Plan)
Step 1: Prepare for the Interview
Task 1: Identify the Participants
Task 1: Understand the Project
Task 2: Understand the Project Objectives
Task 3: Understand the Project Status
Task 4: Understand the Project Plans
Task 5: Understand the Project Development Methodology
Task 6: Identify the High-Level Business Requirements
Method 1: Judgment and Instinct
Method 3: Identifying and Weighting Risk Attributes
Step 3: Summarize the Findings
Task 1: Summarize the Interview
Task 2: Confirm the Interview Findings
Task 1: Prepare an Introduction
Task 2: Define the High-Level Functional Requirements (in Scope)
Task 3: Identify Manual/Automated Test Types
Task 4: Identify the Test Exit Criteria
Task 5: Establish Regression Test Strategy
Task 6: Define the Test Deliverables
Task 7: Organize the Test Team
Task 8: Establish a Test Environment
Task 9: Define the Dependencies
Task 10: Create a Test Schedule
Task 11: Select the Test Tools
Task 12: Establish Defect Recording/Tracking Procedures
Task 13: Establish Change Request Procedures
Task 14: Establish Version Control Procedures
Task 15: Define Configuration Build Procedures
Task 16: Define Project Issue Resolution Procedures
Task 17: Establish Reporting Procedures
Task 18: Define Approval Procedures
Step 2: Define the Metric Objectives
Task 2: Define the Metric Points
Step 3: Review/Approve the Plan
Task 1: Schedule/Conduct the Review
Task 1: Refine the Functional Test Requirements
Task 2: Build a Function/Test Matrix
Ten Guidelines for Good GUI Design
Task 1: Identify the Application GUI Components
Step 3: Define the System/Acceptance Tests
Task 1: Identify Potential System Tests
Task 2: Design System Fragment Tests
Task 3: Identify Potential Acceptance Tests
Task 1: Schedule/Prepare for Review
Task 1: Script the Manual/Automated GUI/Function Tests
Task 2: Script the Manual/Automated System Fragment Tests
Step 2: Review/Approve Test Development
Task 1: Schedule/Prepare for Review
17 Test Coverage through Traceability
18 Test Execution/Evaluation (Do/Check)
Task 1: Regression Test the Manual/Automated Spiral Fixes
Task 2: Execute the Manual/Automated New Spiral Tests
Task 3: Document the Spiral Test Defects
Step 3: Publish Interim Report
Task 1: Refine the Test Schedule
Task 2: Identify Requirement Changes
19 Prepare for the Next Spiral (Act)
Task 1: Update the Function/GUI Tests
Task 2: Update the System Fragment Tests
Task 3: Update the Acceptance Tests
Step 2: Reassess the Team, Procedures, and Test Environment
Task 1: Evaluate the Test Team
Task 2: Review the Test Control Procedures
Task 3: Update the Test Environment
Step 3: Publish Interim Test Report
Task 1: Publish the Metric Graphics
20 Conduct the System Test (Act)
Step 1: Complete System Test Plan
Task 1: Finalize the System Test Types
Task 2: Finalize System Test Schedule
Task 3: Organize the System Test Team
Task 4: Establish the System Test Environment
Task 5: Install the System Test Tools
Step 2: Complete System Test Cases
Task 1: Design/Script the Performance Tests
Task 2: Design/Script the Security Tests
Task 3: Design/Script the Volume Tests
Task 4: Design/Script the Stress Tests
Task 5: Design/Script the Compatibility Tests
Task 6: Design/Script the Conversion Tests
Task 7: Design/Script the Usability Tests
Task 8: Design/Script the Documentation Tests
Task 9: Design/Script the Backup Tests
Task 10: Design/Script the Recovery Tests
Task 11: Design/Script the Installation Tests
Task 12: Design/Script Other System Test Types
Step 3: Review/Approve System Tests
Task 1: Schedule/Conduct the Review
Step 4: Execute the System Tests
Task 1: Regression Test the System Fixes
Task 2: Execute the New System Tests
Task 3: Document the System Defects
Step 1: Complete Acceptance Test Planning
Task 1: Finalize the Acceptance Test Types
Task 2: Finalize the Acceptance Test Schedule
Task 3: Organize the Acceptance Test Team
Task 4: Establish the Acceptance Test Environment
Task 5: Install Acceptance Test Tools
Step 2: Complete Acceptance Test Cases
Task 1: Identify the System-Level Test Cases
Task 2: Design/Script Additional Acceptance Tests
Step 3: Review/Approve Acceptance Test Plan
Task 1: Schedule/Conduct the Review
Step 4: Execute the Acceptance Tests
Task 1: Regression Test the Acceptance Fixes
Task 2: Execute the New Acceptance Tests
Task 3: Document the Acceptance Defects
22 Summarize/Report Test Results
Step 1: Perform Data Reduction
Task 1: Ensure All Tests Were Executed/Resolved
Task 2: Consolidate Test Defects by Test Number
Task 3: Post Remaining Defects to a Matrix
Step 2: Prepare Final Test Report
Task 1: Prepare the Project Overview
Task 2: Summarize the Test Activities
Task 3: Analyze/Create Metric Graphics
Functions Tested and Not Tested
Acceptance Testing Defect Types
Task 4: Develop Findings/Recommendations
Step 3: Review/Approve the Final Test Report
Task 1: Schedule/Conduct the Review
Task 3: Publish the Final Test Report
SECTION 4 PROJECT MANAGEMENT METHODOLOGY
23 The Project Management Framework
Product Quality and Project Quality
Components of the Project Framework
The Project Framework and Continuous Quality Improvement
Executing, Monitoring, and Controlling Phases
Scoping the Project to Ensure Product Quality
Product Scope and Project Scope
The Role of the Project Manager in Quality Management
The Role of the Test Manager in Quality Management
Avoid Duplication and Repetition
Communicate Issues as They Arise
Always Update Your Business Knowledge
Learn the New Testing Technologies and Tools
The Benefits of the Quality Project Management and the Project Framework
Project Quality Management Processes
Identifying the High-Level Project Activities
Estimating the Test Work Effort
Effort Estimation: Model Project
25 The Defect Management Process
Quality Control and Defect Management
Defect Discovery and Classification
26 Integrated Testing and Development
Quality Control and Integrated Testing
Step 1: Organize the Test Team
Step 2: Identify the Tasks to Integrate
Step 3: Customize Test Steps and Tasks
Step 4: Select Integration Points
Step 5: Modify the Development Methodology
Step 6: Test Methodology Training
Step 7: Incorporate Defect Recording
27 Test Management Constraints
Traits of a Well-Established Quality Organization
Using the Project Framework Where No Quality Infrastructure Exists
Ad Hoc Testing and the Project Framework
Using a Traceability/Validation Matrix
SECTION 5 EMERGING SPECIALIZED AREAS IN TESTING
28 Test Process and Automation Assessment
Process Evaluation Methodology
Step 1: Identify the Key Elements
Step 2: Gather and Analyze the Information
The Requirements Definition Maturity
Test Effort Estimation Maturity
Test Design and Execution Maturity
Step 4: Document and Present Findings
Identify the Applications to Automate
Identify the Best Test Automation Tool
Basic Features of an Automation Framework
Modularize Scripts/Test Data to Increase Robustness
Reuse Generic Functions and Application-Specific Function Libraries
Develop Scripting Guidelines and Review Checklists
Define Error Handling and Recovery Functions
Define the Maintenance Process
Standard Automation Frameworks
Step 1: Identifying the Scope of Security Testing
Step 2: Test Case Generation and Execution
Guidelines for Usability Testing
Accessibility Testing and Section 508
Agile User Stories Contrasted to Formal Requirements
32 Testing Center of Excellence
Continuous Competency Development
Step 2: Determine the Economic Trade-Offs
Step 3: Determine the Selection Criteria
Project Management and Monitoring
Implementing the On-Site/Offshore Model
Benefits of On-Site/Offshore Methodology
On-Site/Offshore Model Challenges
Future of the Onshore/Offshore Model
SECTION 6 MODERN SOFTWARE TESTING TOOLS
Automated Capture/Replay Testing Tools
Necessary and Sufficient Conditions
Test Data Generation Strategies
Generating Data Based on the Database
A Cutting-Edge Test Case Generator Based on Requirements
35 Taxonomy of Software Testing Tools
Testing Tool Selection Checklist
Commercial Vendor Tool Descriptions
Open-Source Freeware Vendor Tools
When You Should Consider Test Automation
When You Should NOT Consider Test Automation
36 Methodology to Evaluate Automated Testing Tools
Step 1: Define Your Test Requirements
Step 3a: Conduct Selection Activities for Informal Procurement
Task 1: Develop the Acquisition Plan
Task 2: Define Selection Criteria
Task 3: Identify Candidate Tools
Task 4: Conduct the Candidate Review
Step 3b: Conduct Selection Activities for Formal Procurement
Task 1: Develop the Acquisition Plan
Task 2: Create the Technical Requirements Document
Task 4: Generate the Request for Proposal
Task 6: Perform the Technical Evaluation
Step 4: Procure the Testing Tool
Step 5: Create the Evaluation Plan
Step 6: Create the Tool Manager’s Plan
Step 7: Create the Training Plan
Step 9: Perform the Acceptance Test
Step 11: Implement Modifications
Step 13: Use the Tool in the Operating Environment
Step 14: Write the Evaluation Report
Step 15: Determine Whether Goals Have Been Met
Appendix A: Spiral (Agile) Testing Methodology
Appendix B: Software Quality Assurance Plan
Appendix C: Requirements Specification
Appendix D: Change Request Form