Chapter 2
Quality Assurance Framework
What Is Quality?
In Webster’s dictionary, quality is defined as “the essential character of something, an inherent or distinguishing character, degree, or grade of excellence.” If you look at the computer literature, you will see that there are two generally accepted meanings of quality. The first is that quality means “meeting requirements.” With this definition, to have a quality product, the requirements must be measurable, and the product’s requirements will either be met or not met. With this meaning, quality is a binary state; that is, a product is either a quality product or it is not. The requirements may be complete or they may be simple, but as long as they are measurable, it can be determined whether quality requirements have or have not been met. This is the producer’s view of quality as meeting the producer’s requirements or specifications. Meeting the specifications becomes an end in itself.
Another definition of quality, the customer’s, is the one we use. With this definition, the customer defines quality as to whether the product or service does what the customer needs. Another way of wording it is “fit for use.” There should also be a description of the purpose of the product, typically documented in a customer’s “requirements specification” (see Appendix C, “Requirements Specification,” for more details). The requirements are the most important document, and the quality system revolves around it. In addition, quality attributes are described in the customer’s requirements specification. Examples include usability, the relative ease with which a user communicates with the application; portability, the capability of the system to be executed across a diverse range of hardware architectures; and reusability, the ability to transfer software components constructed in one software system into another.
Everyone is committed to quality; however, the following show some of the confusing ideas shared by many individuals that inhibit achieving a quality commitment:
■ Quality requires a commitment, particularly from top management. Close cooperation between management and staff is required to make it happen.
■ Many individuals believe that defect-free products and services are impossible, and accept certain levels of defects as normal and acceptable.
■ Quality is frequently associated with cost, meaning that high quality equals high cost. This is a confusion between quality of design and quality of conformance.
■ Quality demands requirement specifications in sufficient detail that the products can be quantitatively measured against those specifications. Many organizations are not capable or willing to expend the effort to produce specifications at the level of detail required.
■ Technical personnel often believe that standards stifle their creativity, and thus do not abide by standards compliance. However, to ensure quality, well-defined standards and procedures must be followed.
Prevention versus Detection
Quality cannot be achieved by assessing an already completed product. The aim, therefore, is to prevent quality defects or deficiencies in the first place, and to make the products assessable by quality assurance measures. Some quality assurance measures include structuring the development process with a software development standard and supporting the development process with methods, techniques, and tools. The undetected bugs in the software that caused millions of losses to business have necessitated the growth of independent testing, which is performed by a company other than the developers of the system.
In addition to product assessments, process assessments are essential to a quality management program. Examples include documentation of coding standards, prescription and use of standards, methods, and tools, procedures for data backup, test methodology, change management, defect documentation, and reconciliation.
Quality management decreases production costs because the sooner a defect is located and corrected, the less costly it will be in the long run. With the advent of automated testing tools, although the initial investment can be substantial, the long-term result will be higher-quality products and reduced maintenance costs.
The total cost of effective quality management is the sum of four component costs: prevention, inspection, internal failure, and external failure. Prevention costs consist of actions taken to prevent defects from occurring in the first place. Inspection costs consist of measuring, evaluating, and auditing products or services for conformance to standards and specifications. Internal failure costs are those incurred in fixing defective products before they are delivered. External failure costs consist of the costs of defects discovered after the product has been released. The latter can be devastating because they may damage the organization’s reputation or result in the loss of future sales.
The greatest payback is with prevention. Increasing the emphasis on prevention costs reduces the number of defects that go to the customer undetected, improves product quality, and reduces the cost of production and maintenance.
Verification versus Validation
Verification is proving that a product meets the requirements specified during previous activities carried out correctly throughout the development life cycle. Validation confirms that the system meets the customer’s requirements at the end of the life cycle. It is a proof that the product meets the expectations of the users, and it ensures that the executable system performs as specified. The creation of the test product is much more closely related to validation than to verification. Traditionally, software testing has been considered a validation process, that is, a life-cycle phase. After programming is completed, the system is validated or tested to determine its functional and operational performance.
When verification is incorporated into testing, testing occurs throughout the development life cycle. For best results, it is good practice to combine verification with validation in the testing process. Verification includes systematic procedures of review, analysis, and testing, employed throughout the software development life cycle, beginning with the software requirements phase and continuing through the coding phase. Verification ensures the quality of software production and maintenance. In addition, verification imposes such an organized, systematic development practice that the resulting program can be easily understood and evaluated by an independent party.
Verification emerged about 20 years ago as a result of the aerospace industry’s need for extremely reliable software in systems in which an error in a program could cause mission failure and result in enormous time and financial setbacks, or even life-threatening situations. The concept of verification includes two fundamental criteria: the software must adequately and correctly perform all intended functions, and the software must not perform any function that either by itself or in combination with other functions can degrade the performance of the entire system. The overall goal of verification is to ensure that each software product developed throughout the software life cycle meets the customer’s needs and objectives as specified in the software requirements document.
Verification also establishes tractability between the various sections of the software documentation and the associated parts of the requirements specification. A comprehensive verification effort ensures that all software performance and quality requirements in the specification are adequately tested and that the test results can be repeated after changes are installed. Verification is a “continuous improvement process” and has no definite termination. It should be used throughout the system life cycle to maintain configuration and operational integrity.
Verification ensures that the software functions as intended and has the required attributes (e.g., portability), and increases the chances that the software will contain few errors (i.e., an acceptable number in the final product). It provides a method for closely monitoring the software development project and provides management with a detailed status of the project at any point in time. When verification procedures are used, management can be assured that the developers have followed a formal, sequential, traceable software development process, with a minimum set of activities to enhance the quality of the system.
One criticism of verification is that it increases software development costs considerably. When the cost of software throughout the total life cycle from inception to the final abandonment of the system is considered, however, verification actually reduces the overall cost of the software. With an effective verification program, there is typically a four-to-one reduction in defects in the installed system. Because error corrections can cost 20 to 100 times more during operations and maintenance than during design, overall savings far outweigh the initial extra expense.
Software Quality Assurance
A formal definition of software quality assurance is that it is the systematic activities providing evidence of the fitness for use of the total software product. Software quality assurance is achieved through the use of established guidelines for quality control to ensure the integrity and long life of software. The relationships between quality assurance, quality control, the auditing function, and software testing are often confused.
Quality assurance is the set of support activities needed to provide adequate confidence that processes are established and continuously improved to ensure products that meet specifications and are fit for use. Quality control is the process by which product quality is compared with applicable standards and action taken when nonconformance is detected. Auditing is the inspection/assessment activity that verifies compliance with plans, policies, and procedures.
Software quality assurance is a planned effort to ensure that a software product fulfills these criteria and has additional attributes specific to the project, for example, portability, efficiency, reusability, and flexibility. It is the collection of activities and functions used to monitor and control a software project so that specific objectives are achieved with the desired level of confidence. It is not the sole responsibility of the software quality assurance group, but is determined by the consensus of the project manager, project leader, project personnel, and users.
Quality assurance is the function responsible for managing quality. The word assurance means that if the processes are followed, management can be assured of product quality. Quality assurance is a catalytic function that should encourage quality attitudes and discipline on the part of management and workers. Successful quality assurance managers know how to make people quality conscious and to make them recognize the benefits of quality to themselves and to the organization.
The objectives of software quality are typically achieved by following a software quality assurance plan that states the methods the project will employ to ensure that the documents or products produced and reviewed at each milestone are of high quality. Such an explicit approach ensures that all steps have been taken to achieve software quality and provides management with documentation of those actions. The plan states the criteria by which quality activities can be monitored rather than setting impossible goals, such as no software defects or 100 percent reliable software.
Software quality assurance is a strategy for risk management. It exists because software quality is typically costly and should be incorporated into the formal risk management of a project. Some examples of poor software quality include the following:
■ Delivered software frequently fails.
■ Consequences of system failure are unacceptable, from financial to life-threatening scenarios.
■ Systems are often not available for their intended purpose.
■ System enhancements are often very costly.
■ Costs of detecting and removing defects are excessive.
Although most quality risks are related to defects, this only tells part of the story. A defect is a failure to comply with a requirement. If the requirements are inadequate or even incorrect, the risks of defects are more pervasive. The result is too many built-in defects and products that are not verifiable. Some risk management strategies and techniques include software testing, technical reviews, peer reviews, and compliance verification.
Components of Quality Assurance
Most software quality assurance activities can be categorized into software testing (that is, verification and validation), software configuration management, and quality control. However, the success of a software quality assurance program also depends on a coherent collection of standards, practices, conventions, and specifications, as shown in Figure 2.1.
Software Testing
Software testing is a popular risk management strategy. It is used to verify that functional requirements were met. The limitation of this approach, however, is that by the time testing occurs, it is too late to build quality into the product. Tests are only as good as the test cases, but they can be inspected to ensure that all the requirements are tested across all possible combinations of inputs and system states. However, not all defects are discovered during testing. Software testing includes the activities outlined in this text, including verification and validation activities. In many organizations, these activities, or their supervision, are included within the charter for the software quality assurance function. The extent to which personnel independent of design and coding should participate in software quality assurance activities is a matter of institutional, organizational, and project policy.
The major purpose of verification and validation activities is to ensure that software design, code, and documentation meet all the requirements imposed on them. Examples of requirements include user requirements; specifications derived from and designed to meet user requirements; code review and inspection criteria; test requirements at the modular, subsystem, and integrated software levels; and acceptance testing of the code after it has been fully integrated with hardware.
During software design and implementation, verification helps determine whether the products of one phase of the software development life cycle fulfill the requirements established during the previous phase. The verification effort takes less time and is less complex when conducted throughout the development process.
Quality Control
Quality control is defined as the processes and methods used to monitor work and observe whether requirements are met. It focuses on reviews and removal of defects before shipment of products. Quality control should be the responsibility of the organizational unit producing the product. It is possible to have the same group that builds the product perform the quality control function, or to establish a quality control group or department within the organizational unit that develops the product.
Quality control consists of well-defined checks on a product that are specified in the product quality assurance plan. For software products, quality control typically includes specification reviews, inspections of code and documents, and checks for user deliverables. Usually, document and product inspections are conducted at each life-cycle milestone to demonstrate that the items produced satisfy the criteria specified by the software quality assurance plan. These criteria are normally provided in the requirements specifications, conceptual and detailed design documents, and test plans. The documents given to users are the requirement specifications, design documentation, results from the user acceptance test, the software code, user guide, and the operations and maintenance guide. Additional documents are specified in the software quality assurance plan.
Quality control can be provided by various sources. For small projects, the project personnel’s peer group or the department’s software quality coordinator can inspect the documents. On large projects, a configuration control board may be responsible for quality control. The board may include the users or a user representative, a member of the software quality assurance department, and the project leader.
Inspections are traditional functions of quality control, that is, independent examinations to assess compliance with some stated criteria. Peers and subject matter experts review specifications and engineering work products to identify defects and suggest improvements. They are used to examine the software project for adherence to the written project rules at a project’s milestones and at other times during the project’s life cycle as deemed necessary by the project leader or the software quality assurance personnel. An inspection may be a detailed checklist for assessing compliance or a brief checklist to determine the existence of such deliverables as documentation. A report stating the purpose of the inspection and the deficiencies found goes to the project supervisor, project leader, and project personnel for action.
Responsibility for inspections is stated in the software quality assurance plan. For small projects, the project leader or the department’s quality coordinator can perform the inspections. For large projects, a member of the software quality assurance group may lead an inspection performed by an audit team, which is similar to the configuration control board mentioned previously. Following the inspection, project personnel are assigned to correct the problems on a specific schedule.
Quality control is designed to detect and correct defects, whereas quality assurance is oriented toward preventing them. Detection implies flaws in the processes that are supposed to produce defect-free products and services. Quality assurance is a managerial function that prevents problems by heading them off, and by advising restraint and redirection.
Software Configuration Management
Software configuration management is concerned with labeling, tracking, and controlling changes in the software elements of a system. It controls the evolution of a software system by managing versions of its software components and their relationships.
The purpose of software configuration management is to identify all the interrelated components of software and to control their evolution throughout the various life-cycle phases. Software configuration management is a discipline that can be applied to activities including software development, document control, problem tracking, change control, and maintenance. It can provide high cost savings in software reusability because each software component and its relationship to other software components have been defined.
Software configuration management consists of activities that ensure that design and code are defined and cannot be changed without a review of the effect of the change itself and its documentation. The purpose of configuration management is to control code and its associated documentation so that final code and its description are consistent and represent those items that were actually reviewed and tested. Thus, spurious, last-minute software changes are eliminated.
For concurrent software development projects, software configuration management can have considerable benefits. It can organize the software under development and minimize the probability of inadvertent changes. Software configuration management has a stabilizing effect on all software when there is a great deal of change activity or a considerable risk of selecting the wrong software components.
Elements of Software Configuration Management
Software configuration management identifies a system configuration to systematically control changes, maintain integrity, and enforce tractability of the configuration throughout its life cycle. Components to be controlled include planning, analysis, and design documents, source code, executable code, utilities, job control language (JCL), test plans, test scripts, test cases, and development reports. The software configuration process typically consists of four elements: software component identification, software version control, configuration building, and software change control, as shown in Figure 2.2.
Component Identification
A basic software configuration management activity is the identification of the software components that make up a deliverable at each point of its development. Software configuration management provides guidelines to identify and name software baselines, software components, and software configurations.
Software components go through a series of changes. To manage the development process, one must establish methods and name standards for uniquely identifying each revision. A simple way to name component revisions is to use a series of discrete digits. The first integer could refer to a software component’s external release number. The second integer could represent the internal software development release number. The transition from version number 2.9 to 3.1 would indicate that a new external release, 3, has occurred. The software component version number is automatically incremented when the component is checked into the software library. Further levels of qualifiers could also be used as necessary, such as the date of a new version.
A software configuration is a collection of software elements that comprise a major business function. An example of a configuration is the set of program modules for an order system. Identifying a configuration is quite similar to identifying individual software components. Configurations can have a sequence of versions. Each configuration must be named in a way that distinguishes it from others. Each configuration version must be differentiated from other versions. The identification of a configuration must also include its approval status and a description of how the configuration was built.
A simple technique for identifying a configuration is to store all its software components in a single library or repository. The listing of all the components can also be documented.
Version Control
As an application evolves over time, many different versions of its software components are created, and there needs to be an organized process to manage changes in the software components and their relationships. In addition, there is usually a requirement to support parallel component development and maintenance.
Software is frequently changed as it evolves through a succession of temporary states called versions. A software configuration management facility for controlling versions is a software configuration management repository or library. Version control provides the tractability or history of each software change, including who did what, why, and when.
Within the software life cycle, software components evolve, and at a certain point each reaches a relatively stable state. However, as defects are corrected and enhancement features are implemented, the changes result in new versions of the components. Maintaining control of these software component versions is called versioning.
A component is identified and labeled to differentiate it from all other software versions of the component. When a software component is modified, both the old and new versions should be separately identifiable. Therefore, each version, except the initial one, has a predecessor. The succession of component versions is the component’s history and tractability. Different versions also act as backups so that one can return to previous versions of the software.
Configuration Building
To build a software configuration, one needs to identify the correct component versions and execute the component build procedures. This is often called configuration building.
A software configuration consists of a set of derived software components. An example is executable object programs derived from source programs. Derived software components are correctly associated with each source component to obtain an accurate derivation. The configuration build model defines how to control the way derived software components are put together.
The inputs and outputs required for a configuration build model include the primary inputs such as the source components, the version selection procedures, and the system model, which describes how the software components are related. The outputs are the target configuration and respectively derived software components.
Software configuration management environments use different approaches to select versions. The simplest approach to version selection is to maintain a list of component versions. Other approaches entail selecting the most recently tested component versions, or those modified on a particular date.
Change Control
Change control is the process by which a modification to a software component is proposed, evaluated, approved or rejected, scheduled, and tracked. Its basic foundation is a change control process, a component status reporting process, and an auditing process.
Software change control is a decision process used in controlling the changes made to software. Some proposed changes are accepted and implemented during this process. Others are rejected or postponed, and are not implemented. Change control also provides for impact analysis to determine the dependencies.
Modification of a configuration has at least four elements: a change request, an impact analysis of the change, a set of modifications and additions of new components, and a method for reliably installing the modifications as a new baseline (see Appendix D, “Change Request Form,” for more details).
A change often involves modifications to multiple software components. Therefore, a storage system that provides for multiple versions of a single file is usually not sufficient. A technique is required to identify the set of modifications as a single change. This is often called delta storage.
Every software component has a development life cycle. A life cycle consists of states and allowable transitions between those states. When a software component is changed, it should always be reviewed and further modifications should be disallowed (i.e., it should be frozen) until a new version is created. The reviewing authority must approve or reject the modified software component. A software library holds all software components as soon as they are frozen and also acts as a repository for approved components.
A derived component is linked to its source and has the same status as its source. In addition, a configuration cannot have a more complete status than any of its components, because it is meaningless to review a configuration when some of the associated components are not frozen.
All components controlled by software configuration management are stored in a software configuration library, including work products such as business data and process models, architecture groups, design units, tested application software, reusable software, and special test software. When a software component is to be modified, it is checked out of the repository into a private workspace. It evolves through many states, which are temporarily beyond the scope of configuration management control.
When a change is completed, the component is checked into the library and becomes a new software component version. The previous component version is also retained.
Software Quality Assurance Plan
The software quality assurance (SQA) plan is an outline of quality measures to ensure quality levels within a software development effort. The plan is used as a baseline to compare the actual levels of quality during development with the planned levels of quality. If the levels of quality are not within the planned quality levels, management will respond appropriately as documented within the plan.
The plan provides the framework and guidelines for development of understandable and maintainable code. These ingredients help ensure the quality sought in a software project. An SQA plan also provides the procedures for ensuring that quality software will be produced or maintained in-house or under contract. These procedures affect planning, designing, writing, testing, documenting, storing, and maintaining computer software. It should be organized in this way because the plan ensures the quality of the software rather than describing specific procedures for developing and maintaining it.
Steps to Develop and Implement a Software Quality Assurance Plan
Step 1: Document the Plan
The software quality assurance plan should include the following sections (see Appendix B, “Software Quality Assurance Plan,” which contains a template for the plan):
■ Purpose Section—This section delineates the specific purpose and scope of the particular SQA plan. It should list the names of the software items covered by the SQA plan and the intended use of the software. It states the portion of the software life cycle covered by the SQA plan for each software item specified.
■ Reference Document Section—This section provides a complete list of documents referenced elsewhere in the text of the SQA plan.
■ Management Section—This section describes the project’s organizational structure, tasks, and responsibilities.
■ Documentation Section—This section identifies the documentation governing the development, verification and validation, use, and maintenance of the software. It also states how the documents are to be checked for adequacy. This includes the criteria and the identification of the review or audit by which the adequacy of each document will be confirmed.
■ Standards, Practices, Conventions, and Metrics Section—This section identifies the standards, practices, conventions, and metrics to be applied, and also states how compliance with these items is to be monitored and assured.
■ Reviews and Inspections Section—This section defines the technical and managerial reviews, walkthroughs, and inspections to be conducted. It also states how the reviews, walkthroughs, and inspections are to be accomplished, including follow-up activities and approvals.
■ Software Configuration Management Section—This section is addressed in detail in the project’s software configuration management plan.
■ Problem Reporting and Corrective Action Section—This section is addressed in detail in the project’s software configuration management plan.
■ Tools, Techniques, and Methodologies Section—This section identifies the special software tools, techniques, and methodologies that support SQA, states their purposes, and describes their use.
■ Code Control Section—This section defines the methods and facilities used to maintain, store, secure, and document the controlled versions of the identified software during all phases of development. This may be implemented in conjunction with a computer program library or may be provided as a part of the software configuration management plan.
■ Media Control Section—This section describes the methods and facilities to be used to identify the media for each computer product and the documentation required to store the media, including the copy and restore process, and protects the computer program physical media from unauthorized access or inadvertent damage or degradation during all phases of development. This may be provided by the software configuration management plan.
■ Supplier Control Section—This section states the provisions for ensuring that software provided by suppliers meets established requirements. In addition, it should specify the methods that will be used to ensure that the software supplier receives adequate and complete requirements. For previously developed software, this section describes the methods to be used to ensure the suitability of the product for use with the software items covered by the SQA plan. For software to be developed, the supplier will be required to prepare and implement an SQA plan in accordance with this standard. This section will also state the methods to be employed to ensure that the developers comply with the requirements of this standard.
■ Records Collection, Maintenance, and Retention Section—This section identifies the SQA documentation to be retained. It states the methods and facilities to assemble, safeguard, and maintain this documentation, and will designate the retention period. The implementation of the SQA plan involves the necessary approvals for the plan as well as development of a plan for execution. The subsequent evaluation of the SQA plan will be performed as a result of its execution.
■ Testing Methodology—This section defines the testing approach, techniques, and automated tools that will be used.
Step 2: Obtain Management Acceptance
Management participation is necessary for the successful implementation of an SQA plan. Management is responsible for both ensuring the quality of a software project and for providing the resources needed for software development.
The level of management commitment required for implementing an SQA plan depends on the scope of the project. If a project spans organizational boundaries, approval should be obtained from all affected departments. Once approval has been obtained, the SQA plan is placed under configuration control.
In the management approval process, management relinquishes tight control over software quality to the SQA plan administrator in exchange for improved software quality. Software quality is often left to software developers. Quality is desirable, but management may express concern as to the cost of a formal SQA plan. Staff should be aware that management views the program as a means of ensuring software quality, and not as an end in itself.
To address management concerns, software life-cycle costs should be formally estimated for projects implemented both with and without a formal SQA plan. In general, implementing a formal SQA plan makes economic and management sense.
Step 3: Obtain Development Acceptance
Because the software development and maintenance personnel are the primary users of an SQA plan, their approval and cooperation in implementing the plan are essential. The software project team members must adhere to the project SQA plan; everyone must accept it and follow it.
No SQA plan is successfully implemented without the involvement of the software team members and their managers in the development of the plan. Because project teams generally have only a few members, all team members should actively participate in writing the SQA plan. When projects become much larger (i.e., encompassing entire divisions or departments), representatives of project subgroups should provide input. Constant feedback from representatives to team members helps gain acceptance of the plan.
Step 4: Plan for Implementation of the SQA Plan
The process of planning, formulating, and drafting an SQA plan requires staff and word-processing resources. The individual responsible for implementing an SQA plan must have access to these resources. In addition, the commitment of resources requires management approval and, consequently, management support. To facilitate resource allocation, management should be made aware of any project risks that may impede the implementation process (e.g., limited availability of staff or equipment). A schedule for drafting, reviewing, and approving the SQA plan should be developed.
Step 5: Execute the SQA Plan
The actual process of executing an SQA plan by the software development and maintenance team involves determining necessary audit points for monitoring it. The auditing function must be scheduled during the implementation phase of the software product so that improper monitoring of the software project will not hurt the SQA plan. Audit points should occur either periodically during development or at specific project milestones (e.g., at major reviews or when part of the project is delivered).
Quality Standards
The following section describes the leading quality standards for IT.
Sarbanes–Oxley
The Sarbanes–Oxley Act of 2002, also known as the Public Company Accounting Reform and Investor Protection Act of 2002 and commonly called SOx or Sarbox, is a U.S. federal law enacted on July 30, 2002, in response to a number of major corporate and accounting scandals: Enron, Tyco Internation, Adelphia, Peregrine Systems, and WorldCom.
The Sarbanes–Oxley Act is designed to ensure the following within a business:
■ There are sufficient controls to prevent fraud, misuse, or loss of financial data/transactions. In many companies, most of these controls are IT-based.
■ There are controls to enable speedy detection if and when such problems occur.
■ Effective action is taken to limit the effects of such problems.
Table 2.1 Top CQBIT Controls
Not only must controls be in place; they must be effective, and it must be possible to note exceptions caught by the controls and follow audit trails to take appropriate action in response to those exceptions. This requirement puts new pressure on IT that until now few IT departments have faced.
The ISACA subset of COBIT ensures that the key IT aspects related to Sarbanes–Oxley are being tested. The top COBIT controls, as recommended in the ISACA study, are in Table 2.1, along with a list of tactical solutions that satisfy those controls.
The COBIT objectives are specifically designed to aid the effective management of information and IT, with particular emphasis on IT governance. They provide management with a framework for implementing internal control systems that support core business processes. They clarify areas of responsibility and due diligence by all individuals engaged in the management, use, design, development, maintenance, and operation of a company’s information systems.
Table 2.2 attempts to break the 318 COBIT controls down into areas of activity, to try to make the task more manageable. This helps you to understand the key areas that will need to be addressed, either through the introduction of internal controls, through automated solutions, or both.
Table 2.2 CQBIT Controls by Areas of Activity
Table 2.3 Companion ISO Standards
ISO9000
ISO9000 is a quality series and comprises a set of five documents developed in 1987 by the International Standards Organization (ISO). ISO9000 standards and certification are usually associated with non-IS manufacturing processes. However, application development organizations can benefit from these standards and position themselves for certification, if necessary. All the ISO9000 standards are guidelines and are interpretive because of their lack of stringency and rules. ISO certification is becoming more and more important throughout Europe and the United States for the manufacture of hardware. Software suppliers will increasingly be required to have certification. ISO9000 is a definitive set of quality standards, but it represents quality standards as part of a total quality management (TQM) program. It consists of ISO9001, ISO9002, or ISO9003, and it provides the guidelines for selecting and implementing a quality assurance standard.
ISO9001 is a very comprehensive standard and defines all the quality elements required to demonstrate the supplier’s ability to design and deliver a quality product. ISO9002 covers quality considerations for the supplier to control design and development activities. ISO9003 demonstrates the supplier’s ability to detect and control product nonconformity during inspection and testing. ISO9004 describes the quality standards associated with ISO9001, ISO9002, and ISO9003 and provides a comprehensive quality checklist.
Table 2.3 shows the ISO9000 and companion international standards.
Capability Maturity Model (CMM)
The Software Engineering Institute–Capability Maturity Model (SEI–CMM) is a model for judging the maturity of the software processes of an organization and for identifying the key practices that are required to increase the maturity of these processes. As organizations enhance their software process capabilities, they progress through the various levels of maturity. The achievement of each level of maturity signifies a different component in the software process, resulting in an overall increase in the process capability of the organization. The Capability Maturity Model for Software describes the principles and practices underlying software process maturity and is intended to help software organizations improve the maturity of their software processes in terms of an evolutionary path from ad hoc chaotic processes to mature, disciplined software processes.
The CMM is organized into five maturity levels (see Figure 2.3):
Initial. The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics.
Repeatable. Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications.
Defined. The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the organization. All projects use an approved, tailored version of the organization’s standard software process for developing and maintaining software.
Managed. Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled.
Optimizing. Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies.
Level 1: Initial
The organization typically does not provide a stable environment for developing and maintaining software. This period is chaotic without any procedure and process established for software development and testing. When an organization lacks sound management practices, ineffective planning and reaction-driven commitment systems undermine the benefits of good software engineering practices.
In this phase, projects typically abandon planned procedures and revert to coding and testing. Success depends entirely on having an exceptional manager and effective software team. The project performance depends on capable and forceful project managers. However, when they leave the project, their stabilizing influence leaves with them. Even a strong engineering process cannot overcome the instability created by the absence of sound management practices.
Level 2: Repeatable
During this phase, measures and metrics will be reviewed to include percentage compliance with various processes, percentage of allocated requirements delivered, number of changes to requirements, number of changes to project plans, variance between estimated and actual size of deliverables, and variance between actual PQA audits performed and planned and number of change requests processed over a period of time. The following are the key process activities during Level 2:
■ Software configuration management
■ Software quality assurance
■ Software subcontract management
■ Software project tracking and oversight
■ Software project planning
■ Requirements management
Level 3: Defined
During this phase measures and metrics will be reviewed to include percentage of total project time spent on test activities, testing efficiency, inspection rate for deliverables, inspection efficiency, variance between actual attendance and planned attendance for training programs, and variance between actual and planned management effort. Level 3 compliance means an organization’s processes for management and engineering activities have been formally defined, documented, and integrated into a standard process that is understood and followed by the organization’s staff when developing and maintaining software. Once an organization has reached this level, it has a foundation for continuing progress. New processes and tools can be added with minimal disruption, and new staff members can be easily trained to adapt to the organization’s practices. The following are the key process areas for Level 3:
■ Peer reviews
■ Intergroup coordination
■ Software product engineering
■ Integrated software management
■ Training program
■ Organization process definition
■ Organization process focus
The software process capability of Level 3 organizations can be summarized as standard and consistent because both software engineering and management activities are stable and repeatable. Within established product lines, cost, schedule, and functionality are under control, and software quality is tracked. This process capability is based on a common organizationwide understanding of the activities, roles, and responsibilities in a defined software process.
Level 4: Managed
This phase denotes that the processes are well defined and professionally managed. The quality standards are on an upswing. With sound quality processes in place, the organization is better equipped to meet customer expectations of high-quality/high-performance software at reasonable cost and committed deliveries. Delivering consistency in software work products and consistency throughout the software development life cycle, including plans, process, requirements, design, code, and testing, helps create satisfied customers. Projects achieve control over their products and processes by narrowing the variation in their process performance within acceptable quantitative boundaries. Meaningful variations in process performance can be distinguished from random variations (noise), particularly within established product lines. The risks involved in moving up the learning curve of a new application domain are known and carefully managed:
■ Software quality management
■ Quantitative process management
The software process capability of Level 4 organizations can be summarized as predictable because the process is measured and operates within measurable limits. The level of process capability allows an organization to predict trends in process and product quality within the quantitative bounds of these limits. When these limits are exceeded, action is taken to correct the situation. Software products are of predictably high quality.
Level 5: Optimized
A continuous emphasis on process improvement and defect reduction avoids process stagnancy or degeneration and ensures continual improvement translating into improved productivity, reduced defect leakage, and greater timeliness. Tracing requirements across each development phase improves the completeness of software, reduces rework, and simplifies maintenance. Verification and validation activities are planned and executed to reduce defect leakage. Customers have access to the project plan, receive regular status reports, and their feedback is sought and used for process tuning. The KPA at Level 5 are:
■ Process change management
■ Technology change management
■ Defect prevention
Software project teams in Level 5 organizations analyze defects to determine their causes. Software processes are evaluated to prevent known types of defects from recurring, and lessons learned are disseminated to other projects. The software process capability of Level 5 organizations can be characterized as continuously improving because Level 5 organizations are continuously striving to improve the range of their process capability, thereby improving the process performance of their projects. Improvement occurs both by incremental advancements in the existing process and by innovations using new technologies and methods.
People CMM
The People Capability Maturity Model (People CMM) is a framework that helps organizations successfully address their critical people issues. On the basis of the best current practices in fields such as human resources, knowledge management, and organizational development, the People CMM guides organizations in improving their processes for managing and developing their workforces. The People CMM helps organizations characterize the maturity of their workforce practices, establish a program of continuous workforce development, set priorities for improvement actions, integrate workforce development with process improvement, and establish a culture of excellence. Since its release in 1995, thousands of copies of the People CMM have been distributed, and it is used worldwide by organizations small and large.
The People CMM consists of five maturity levels that establish successive foundations for continuously improving individual competencies, developing effective teams, motivating improved performance, and shaping the workforce the organization needs to accomplish its future business plans. Each maturity level is a well-defined evolutionary plateau that institutionalizes new capabilities for developing the organization’s workforce. By following the maturity framework, an organization can avoid introducing workforce practices that its employees are unprepared to implement effectively.
CMMI
The CMMI Product Suite provides the latest best practices for product and service development and maintenance (Andrews and Whittaker, 2006). The CMMI models are the best process improvement models available for product and service development and maintenance. These models extend the best practices of the Capability Maturity Model for Software (SW-CMM®), the Systems Engineering Capability Model (SECM), and the Integrated Product Development Capability Maturity Model (IPD-CMM).
Organizations reported that CMMI is adequate for guiding their process improvement activities and that CMMI training courses and appraisal methods are suitable for their needs, although there are specific opportunities for improvement. The cost of CMMI is an issue that affected adoption decisions for some, but not for others. Finally, return-on-investment information is usually helpful to organizations when making the business case to adopt CMMI.
Malcolm Baldrige National Quality Award
As the National Institute of Standards and Technology (NIST) says:
In the early and mid-1980s, many industry and government leaders saw that a renewed emphasis on quality was no longer an option for American companies but a necessity for doing business in an ever-expanding, and more demanding, competitive world market. But many American businesses either did not believe quality mattered for them or did not know where to begin (Arthur, 1993).
Public Law 100-107, signed into law on August 20, 1987, created the Malcolm Baldrige National Quality Award. The Award Program led to the creation of a new public–private partnership. Principal support for the program comes from the Foundation for the Malcolm Baldrige National Quality Award, established in 1988. The Award is named for Malcolm Baldrige, who served as Secretary of Commerce from 1981 until his tragic death in a rodeo accident in 1987.
The Baldrige Award is given by the President of the United States to businesses—manufacturing and service, small and large—and to education and health care organizations that apply and are judged to be outstanding in seven areas: leadership, strategic planning, customer and market focus, information and analysis, human resource focus, process management, and business results. . . . While the Baldrige Award and the Baldrige recipients are the very visible centerpiece of the U.S. quality movement, a broader national quality program has evolved around the award and its criteria. A report, “Building on Baldrige: American Quality for the 21st Century,” by the private Council on Competitiveness, said, ‘More than any other program, the Baldrige Quality Award is responsible for making quality a national priority and disseminating best practices across the United States.’
Figure 2.4
Each year, more than 300 experts from industry, educational institutions, governments at all levels, and nonprofit organizations volunteer many hours reviewing applications for the award, conducting site visits, and providing each applicant with an extensive feedback report citing strengths and opportunities to improve. In addition, board members have given thousands of presentations on quality management, performance improvement, and the Baldrige Award (Arthur, 1993).
The Baldrige performance excellence criteria are a framework (see Table 2.4) that any organization can use to improve overall performance. Seven categories make up the award criteria.
The system for scoring examination items is based on these evaluation dimensions:
Approach: Approach indicates the method that the company uses to achieve the purposes. The following are the factors to decide on the correct approach: the degree to which the approach is prevention-based; the appropriateness of the tools, techniques, and methods; the effectiveness of their use; whether the approach is systematic, integrated, and consistently applied; effective self-evaluation and feedback; quantitative information gathered; and the uniqueness and innovativeness of the approach.
Deployment: This concerns the areas where the approach is deployed. It evaluates whether the approach is implemented in all the products and services and all internal processes, activities, facilities, and employees.
Results: This refers to the outcome of the approach. The quality levels demonstrated, rate of quality improvement, breadth, significance, and comparison of the quality improvement and the extent to which quality improvement is demonstrated are the key factors involved.
Table 2.4 Baldrige Performance Framework
As compared to other programs such as ISO, Japan’s Deming award and America’s Baldrige Award:
■ Focus more on results and service
■ Rely on the involvement of many different professional and trade groups
■ Provide special credits for innovative approaches to quality
■ Include a strong customer and human resource focus
■ Stress the importance of sharing information
Notes
1. http://www.sei.cmu.edu/cmmi/adoption/cmmi-start.html.
2. http://www.nist.gov/public_affairs/factsheet/baldfaqs.htm.