Verification is the ongoing process we recommend to ensure that each step of the development is correct, meets the needs of the next step, and is not superfluous to the needed activities. This chapter examines traceability techniques you can use to support all stages of verification of the project, right from the start.
A significant factor in quality software implementation is the ability to trace the implementation through the stages of specification, architecture, design, implementation, and testing. The ability to track relationships and to relate them when change occurs forms a key thread throughout many modern high-assurance software processes, particularly in life-critical medical products or business or mission-critical activities. The reason is that historical safety data has shown that the impact of change is often missed and that small changes to a system can create significant safety and reliability problems.
For example, the latest U.S. Food and Drug Administration (FDA) guidance for traceability in medical software development activities is contained in the FDA Office of Device Evaluations (ODE) Guidance Document (1996b). In addition, the Design Controls section of the new medical Current Good Manufacturing Practices (CGMP) document (FDA 1996a), Subpart C of the CGMP, defines the obligations of the system designers to be able to trace relationships between various work products within the lifecycle of the product's development.
IEEE (1994) provides two working definitions of traceability:
"The degree to which a relationship can be established between two or more products of the development process, especially products having a predecessor-successor or master-subordinate relationship to one another; for example, the degree to which the requirements and design of a given software component match." (IEEE 610.12-1990 §3)
"The degree to which each element in a software development product establishes its reason for existing; for example, the degree to which each element in a bubble chart references the requirement it satisfies." (IEEE 610.12-1990 §3)
A key element of traceability is the "traceability relationship." We find it convenient to define the relationship in terms of a simple "traced-to" and "traced-from" model. For example, we can easily imagine that one or more software requirements are created in the system in order to support a given feature specified in the Vision document. Thus, we can say that a software requirement is traced from one or more features (see Figure 31-1).
Additional meaning can be placed on the relationship from the context of the requirement types that are created. For example, a software requirement that is traced to a test case would suggest that the software requirement is "tested by" the test case that it is "traced to." An object description that is traced from a software requirement would imply that the requirement is "implemented by" the referenced object. Indeed, your project is likely to have one-to-many, many-to-one, and many-to-many interrelationships among these project elements.
Building on what we've learned about the various means of expressing requirements and the requirements organization we discussed in Chapter 25, your project might be structured with the full set of relationships as shown in Figure 31-2.
In Figure 31-2, we observe that your project team explicitly adds the relationships between the elements. That is, we define explicit traceability as the development of relationships stemming from external considerations supplied by your team. So, for example, the linkage, or relationship, between a product feature and a use case that supports that feature is determined solely by the team's deciding that such a relationship has meaning. There is no intrinsic relationship between the elements; only external decisions can establish the relationships.
On the other hand, the methodology and the structure of the model may provide implicit traceability relationships. For example, Team Skill 5 discussed the notion of parent-child requirements that exhibited a formal parent-child hierarchical relationship. In the case of hierarchical requirements, there is an implicit traceability between the parent requirements and the related child requirements. This implicit relationship need not be explicitly stated as an explicit relationship; indeed, it should not be stated explicitly, because of possible confusions that may arise.
Other cases of implicit traceability may arise from the modeling paradigm used. For example, the modeling tools used in the development process may automatically provide other traceability relationships among the modeling elements. If, for example, your modeling tool provides implicit links between use-case modeling elements and the actors that interact with the use case, you have a significant opportunity to exploit those implicit traceability relationships. You can further extend the traceability into the implementation by tracing the use-case collaborations that trace to the implementation objects.
In summary, we make little distinction between the two classes of traceability. The only caution we offer is to make sure that you are aware of all of the forms of traceability your modeling tools offer. If the tools do provide certain forms of implicit traceability, use them. If the tools do not offer implicit traceability in areas of interest to you, you will need to generate explicit traceability linkages as required to support your development efforts.
Traceability can frequently help in understanding other parts of a project. We have often added less traditional elements to a project and included them in the traceability processes because they add value to our understanding of the project.
For example, you might find it useful to define a new element called an "issue" and to keep all of your running, unresolved issues as part of your project elements. By using traceability techniques, you can link your issues to the items they are referring to. For example, if you have an unresolved issue about the product's functionality, you can link that issue to the product features and the software requirements related to that issue. By maintaining the "issue" links, you have an easy way to go back into your project and find all of the elements related to an issue you may have just resolved. Other nontraditional elements you may want to incorporate into your project traceability processes include
Assumptions and rationales
Action items
Requests for new/revised features
Glossary and acronym terms
Bibliographic references
The key point is that you can use traceability to help you understand the relationships among "things" in your project. Let your imagination roam over the possibilities and add "things" that will help your team understand the project better. Feel free to trace all of useful relationships among "things." For example, there may be unresolved issues with a particular definition of a glossary item. In such cases, linking an issue with a glossary element (and, perhaps, other features) may be a useful aid to reminding the team that unresolved matters still exist in the project. A typical traceability structure might be augmented with additional traceability items, as shown in Figure 31-3.
A caution: . Do not go overboard with this notion. We have found that adding too many "things" to the traceability process becomes a maintenance burden. As always, you should strive for a good balance between the value of the extra elements you wish to trace and the cost of maintaining them.
Powerful software development tools offer a simple user-guided procedure to "point and click" through the relationships that may exist between two elements of the lifecycle. We have had extensive experience with the RequisitePro tool offered by Rational Software and have chosen to use that tool for all of the tool illustrations in this book. Your choice of tool may differ, but the end results should be similar to our examples. (Refer to Appendix A for a more detailed view of the HOLIS project artifacts that we will be using in the following examples.)
Using a tool offers you many ways to obtain additional insights into your project. For example, after we have defined the relationships between the features and the software requirements on our HOLIS project, we can display a matrix version of those relationships, as shown in Figure 31-4.
Interpreting the traceability matrix in Figure 31-4 is straightforward. For example, consider the intersection of FEA1 ("Easy to program control …") and SR3 ("HOLIS shall support up to ….") At the intersecting cell, the arrow indicates that a relationship traces from FEA1 to SR3, meaning that SR3 in some way satisfies the feature defined as FEA1.
After using a tool to establish all known relationships, an instructive requirements management activity—strongly supported by not only governmental regulatory guidance but also our own experiences—is to examine a traceability matrix for two potential indications of error:
If inspection of a row fails to detect any traceability relationships (no "arrows"), a possibility exists that no software requirement is yet defined to respond to a feature required in the Vision document. This may be acceptable if, for example, the feature is not software ("The case shall be of nonbreakable plastic"). Nevertheless, empty rows are potential red flags and should be checked carefully. Modern requirements management tools should have a facility to automate this type of inspection.
If inspection of a column fails to detect any traceability relationships, a possibility exists that a software requirement has been included for which there is no known product feature that requires it. This may indicate a misunderstanding of the role of the software requirement, or a weakness in the original Vision document, or it may indicate dead code or code that is not in support of the system requirement or a programmer's "Easter Egg," in which case it should be weeded out immediately. (We will discuss Easter Eggs in more detail in Chapter 34.) In any case, careful checking is required.
In addition to providing a set of tools to query the relationships you have established, your development tool should provide a simple means to store the queries and to recall them later. This feature allows you to revisit the relationships at a later time, perhaps after changes have been made, and to quickly requery the relationships to detect potential trouble spots. As you will see in Chapter 34, this virtually guaranteed occurrence arises from the continual project changes.
Simple and obvious application of such techniques will enable you to relate many elements of your project. You should consider linking and relating
Software requirements and use cases to test plans/test specs/test results
User needs to product features
Product features to software requirements and use cases
Software requirements and use cases to implementation units such as functions, modules, objects, and collaborations
Implementation units to test plans/specs/results
After linking the various elements of the various documents, you should have a relationship setup similar to that shown in Figure 31-5.
Your requirements management tool should also be able to display the full set of traceability relationships within a project. Figure 31–6 shows such a "tree" view. Note that the (partial) tree view allows you to view simultaneously all of the currently established relationships in your project. You should use the tree view to help you comprehend the overall relationships within your project. For example, Figure 31-6 reveals that FEA5 ("Vacation settings") links to SR1, SR3, and UC1.
Once you have linked the elements, your tool should maintain those linkages. You may then use the full power of the tool to examine relationships among the project elements as you desire. You've probably already figured out that we can use the traceability relationships to support our verification efforts.
Of course, you may not have a tool specifically constructed to support the types of operations identified in the preceding section. Without such a tool, how should you proceed?
In the pretool days, we used spreadsheets and databases to maintain the traceability relationships. After all, many of the matrix relationships could be easily handled with a simple spreadsheet. We used spreadsheets extensively during the 1980s and early 1990s and found them to be a useful aid in managing projects.
The problem with spreadsheets, however, was maintenance, especially in extensive hierarchies of relationships. For instance, we found that changing a single linkage could have far-flung impacts in the relationships at issue, as well as other relationships in other parts of the hierarchy. Truth to tell, it was usually a nightmare if extensive changes to the linkages had to be made.
Since it was so difficult to manually keep track of the changes and their "ripple effects," we found that we either
Fell into a pattern of resisting any discussions to change the relationships, or
Abandoned the matrices as the work became too overwhelming.
Of course, we always came to regret both of these behaviors, as they inevitably led to subsequent problems in the project. Imagine our excitement when modern tools began to arrive and to help in this activity!
The other alternative was to use a database. We used relational databases extensively and found it fairly easy to construct them and to input the data. Indeed, in the pretool days, we used such tools to run human-critical medical device development projects. Relational databases worked pretty well. Even though it was more difficult to expand the database application to include tracking ripple effects from changes, it could be done. The problem, however, was that we ended up spending disproportionate amounts of time improving the tool's capability. Good for the ego but bad for the project resources who should have been doing something else.
So, although you can use spreadsheets or databases to maintain the traceability relationships, it won't be easy. If you have a small project, the pain and suffering will be minimal, and it might be worth considering simpler alternatives. On the other hand, we do not recommend tackling larger projects without the use of specialized requirements management tools.
In this case, you are looking for cases in which the rows of the traceability matrix show you that a particular feature is not linked to some software requirements/use cases. For example, inspection of the matrix in Figure 31-7 shows that Product Feature 4 (FEA4) is not linked to any Software Requirement. And, although we didn't show it, there are no links to any use case (UC) either.
Upon detecting a "hole" in the relationships, you should review the original set of product requirements and the related software requirements/use cases.
You may find that a link was accidentally missed in establishing the traceability. In such cases, simply adding a new link and recomputing the trace matrix will solve the problem. This type of omission frequently occurs in the early stages of a project.
On the other hand, you might find that the development of the software requirements simply failed to consider the needs of one of the required product features. In this case, a project review may be necessary to consider the addition of suitable requirements to respond to the product feature. Or, the project review may determine that the omitted feature should be moved to a "future" category or removed entirely.
In any event, the important facet of verification is to ensure that no linkages have been left out and that all lower-level elements, such as the software requirements/use cases, have been properly related to the higher-level product requirements.
Verification may also uncover the opposite issue. That is, inspection of the columns of the trace matrix may reveal a column that is not linked to any row elements. In Figure 31-8, for example, use case 3 (UC3) is not linked to any product feature (FEA).
This type of situation indicates that you have created a use case (or requirement) for which there was no related product feature. That is, the requirement appears to be superfluous to the product features. As before, you should review the trace relationships
You may find that a link was accidentally missed in establishing the traceability. In such cases, simply adding a new link and recomputing the trace matrix will solve the problem.
On the other hand, you might find that the determination of the product features simply failed to consider the needs of one of the required software requirements. This situation may occur if, for example, you have certain design constraints that are required in the implementation but that change the features of the product. Here, a project review may be necessary to consider the feasibility and need for the requirements. You might wish to remove the requirement(s) or place them on a "future" list. Or, the project review may determine that the omitted feature should be moved to a "future" category or added now. In our example, we have reviewed the project and have determined that UC3 was a superfluous specification that should not have been in the project.
This type of verification inspection focuses on ensuring that gratuitous elements have not crept into the project. Experience has shown that such elements typically increase the project scope and rarely contribute to the overall quality of the result.
The problem with considering omitted/excess relationships is that it is very mechanical. That is, it is all too tempting to say, "Well, we looked at all of the rows and columns, and everything looks OK. So, we're verified. Let's move on."
The fallacy in this argument is that we have not considered whether or not we've correctly and completely considered all of the links that should (and should not) be established. We have found that deeper consideration of the initial traceability linkages always leads to some revisions in the linkages. New linkages may be added, and existing linkages may be revised or removed.
We urge you to consider the initial linkages as merely a starting point in verification. Formal or informal reviews should be held after the initial linkages have been established and the initial row/column inspections completed. After at least one full-scale review is completed and changes posted, then you may consider that verification for the current phase is valid.
Of course, change happens. Thus, you will need to treat all linkages as living linkages that will be revised as the project matures and evolves. You should consider verification reviews whenever it becomes apparent that future, current, or past phases have had significant changes in the traceability linkages. Retrospective reviews are frequently needed whenever new understandings of the project cause old linkages and relationships to be reconsidered. The crucial point is that verification requires thinking, and you should not fall into the trap of rote mechanical processing.
Verification is an important technique your team can apply in the struggle to help ensure that you are designing and implementing the right systems. To our amazement, we continue to find many projects that don't use a verification strategy to make sure that the project is heading in the right direction. Inevitably, those projects "fall off the rails," and the team members don't even know it. Of course, they eventually find it out but, by then, it is very late in the game, and bad feelings and desperate measures ensue. Projects that undergo frequent verification challenges are much less subject to such surprises.
But all the verification in the world will not guarantee that the final result works as intended. So, we need to include another concept to help us ensure that the right system is being built. That concept is called validation and is the subject of our next chapter.