So far, we have not talked much about the overall software development process and its relationship to the team's ability to achieve the results it desires. However, effective requirements management cannot occur without the context of a reasonably well-defined software process that defines the full set of activities that your team must execute to deliver the final software product. Some software processes are relatively formal, and some are informal, but always, a process is at work, even though it may not be rigorous or documented.
Your team's software development process defines who (which member of the team) is doing what (what activity is being performed, when (that activity is done in relation to other activities), and how (the details and steps in the activity) in order for your team to reach its goal. Software processes have a material effect on your team's ability to develop software on time and on budget. In this chapter, we look at some of the higher-level aspects of various types of software processes, namely, the time-dependent phases and major types of activities in those phases, and we then analyze how the various software processes affect the scope management challenge.
Boehm (1988a) points out that, as early as the 1950s, the software industry, recognizing the cost of discovering software defects late in the cycle, adopted a logical, stepwise process model, which progressed from a requirement phase to a design phase to a coding phase, and so on. This was a major improvement over the earlier, two phase "code and fix" model, whereby programmers first wrote the code and then fixed it until it couldn't be fixed any more.
In the 1970s, Winston Royce (1970), working at TRW, defined what became known as the "waterfall model" of software development. The waterfall model improved on the strictly stepwise model by
Recognizing the need for feedback loops between stages, thereby acknowledging that design affects requirements, that coding the system will cause the design to be revisited, and so on.
Developing a prototype system in parallel with requirements analysis and design activities
As shown in Figure 22-1, in the waterfall model, software activities proceed logically through a sequence of steps. Each step bases its work on the activities of the previous step. Design logically follows requirements, coding follows design, and so on. The waterfall model has been widely followed over the past two decades and has served successfully as a process model for a variety of medium- to large-scale projects.
Note that as commonly applied, Figure 22-1 does not reference the prototyping activity prescribed. This is an unfortunate mistake in history that we'll return to shortly.
The waterfall model has been somewhat successful in reinforcing the role of requirements as the necessary first step in software development, the basis for design and coding activities. However, this strength became a source of difficulty, as it tended to emphasize fully elaborated requirements and design documents as a barrier to exit from each phase. Also, through perhaps its misapplication by overzealous development teams, this model has come to represent a fixed, rigid approach to development, wherein requirements are "frozen" for the life of the project, change is anathema, and the process of development takes on a life of its own. In this case, over time, the team may be completely disengaged from the real world that the project was originally based on.
The waterfall model comes under additional pressure when it is aligned to the scope management challenge (Figure 22-2). Specifically, if the waterfall model is applied to a project that is initiated with 200 percent scope, the results can be disastrous. At deadline time, nothing really works, unit test and system integration are forced or abandoned, and significant investments have been made in the specification, design, and coding of system features that are never delivered. The result: nothing deliverable, chaos, poor quality, and software scrap.
Primarily for these reasons, the waterfall model has become less popular. One unfortunate result has been the tendency to leap right into code, with an inadequate understanding of the requirements for the system, which was one of the main problems the waterfall model was trying to solve!
Barry Boehm's pivotal study (1988a) recommended a different framework for guiding the software development process. His "spiral model" of software development is a role model for those who believe that success follows a more risk-driven and incremental development path (Figure 22-3).
In the spiral model, development is initially driven by a series of risk-driven prototypes; then a structured waterfall-like process is used to produce the final system. Of course, when misused, the spiral model can exhibit as many problems as the misused waterfall model. Projects sometimes fall into a cut-and-try approach, providing incremental deliverables that must be expanded and maintained with the addition of bubble gum and baling wire, of what some refer to as the process of creating instant legacy code, progress being measured by our newfound ability to create nonmaintainable and nonunderstandable code two to three times as fast as with earlier technology!
When you look at the spiral model more carefully, however, it provides a sensible road map that helps address some of the requirements challenges noted in this book. Specifically, the spiral model starts with requirements planning and concept validation, followed by one or more prototypes to assist in early confirmation of our understanding of the requirements for the system. The main advantage of this process is the availability of multiple feedback opportunities with the users and customers, which is intended to get the "Yes, Buts" out early. Opponents of this rigorous approach note that in today's environment, the luxury of time implied by full concept validation, two or three prototypes, and then a rigorous waterfall methodology is simply not available.
Again, what happens in the spiral model when a project is initiated with 200 percent scope? Figure 22-4 illustrates the result. One might argue that the result is not much better than a disastrous waterfall plan; others might note that at least a prototype or two is operational and that one has customer feedback at the deadline time. (Of course, a lot of this feedback will focus on the lack of availability of any production-deployable software!)
In the past decade, many teams have migrated to a new approach, one that combines the best of waterfall and spiral models and is a hybrid of the two. This new approach also incorporates some additional constructs from the advancing discipline of software process engineering. The "iterative approach" that was introduced in Kruchten (1995) has now been well described in a number of texts, including Kruchten (1999) and Royce (1998). This approach has proved effective in a wide variety of project types and can exhibit a number of advantages over the waterfall and the spiral models of development.
In the traditional software development process models, time moves forward through a series of sequential activities, with requirements preceding design, design preceding implementation, and so on. This seems quite sensible. In the iterative approach, the lifecycle phases are decoupled from the logical software activities that occur in each phase, allowing us to revisit various activities, such as requirements, design, and implementation, at various iterations of the project. In addition, like the spiral model, each iteration is designed to mitigate whatever risks are present in that stage of development activity.
The interactive approach consists of four lifecycle phases: inception, elaboration, construction, and transition, corresponding to fairly natural "states" of the project at these times. (See Figure 22-5.).
In the inception phase, the team is focused on understanding the business case for the project, the scope of the project, and the feasibility of an implementation. Problem analysis is performed, the Vision document is created, and preliminary estimates of schedule and budget, as well as project risk factors, are defined.
In the elaboration phase, the requirements for the system are refined, an initial executable architecture is established, and an early feasibility prototype is typically developed and demonstrated.
In the construction phase, the focus is on implementation. Most of the coding is done in this phase, and the architecture and design are fully developed.
Beta testing typically happens in this transition phase, and the users and the maintainers of the system are trained on the application. The tested baseline of the application is transitioned to the user community and deployed for use.
Within each phase, the project typically undergoes multiple iterations (Figure 22-6). An iteration is a sequence of activities with an established plan and evaluation criteria, resulting in an executable of some type. Each iteration builds on the functionality of the prior iteration; thus, the project is developed in an "iterative and incremental" fashion.
Iterations are selected according to a number of criteria. Early iterations should be designed to evaluate the viability of the chosen architecture against some of the most important and risk-laden use cases.
In the iterative approach, the activities associated with the development of the software are organized into a set of workflows. Each workflow consists of a logically related set of activities, and each defines how the activities must be sequenced to produce a viable work product, or "artifact." Although the number and kind of workflows can vary, based on the company or project circumstances, there are typically at least six workflows, as Figure 22-7 illustrates.
During each iteration, the team spends as much time as appropriate in each workflow. Thus, an iteration can be regarded as a miniwaterfall through the activities of requirements, analysis and design, and so on, but each miniwaterfall is "tuned" to the specific needs of that iteration. The size of the "hump" in Figure 22-7 indicates the relative amount of effort invested in a workflow. For example, in the elaboration phase, significant time is spent on "refining" the requirements and in defining the architecture that will support the functionality. The activities can be sequential (a true miniwaterfall) or may execute concurrently, as is appropriate to the project.
From the requirements management perspective, the iterative approach has two significant advantages
Getting the "Yes, Buts" out early. . Each iteration produces an executable release such that, very early in the project, the customers have the opportunity to see the work product. And, of course, their reaction will be "Yes, But," but at this early stage, only a minimal investment has been made. With each successive iteration, the size of the "Yes, But" should decrease, and you and your customer will eventually converge on the right system.
Better scope management. . If the first iteration is missed by 30 percent, that's an indicator that the project may be badly scoped, and adjustments can be made. Even if scope is not well managed, multiple executable iterations have been developed by the time the deadline is reached, and the last may even be deployed. Even though it lacks some functionality, the release will deliver value to the user, if the features have been picked and prioritized carefully, allowing your customer to meet objectives, at least in part, as you continue with further development iterations. And, if the architecture is robust and addresses the key technical issues, your team will have a solid platform on which to base the additional functionality.
One of the premises of this book is that getting the "Yes, Buts" out early is one of the highest-leverage activities of the software process.
How many times do you have to do it?
Do the customer's demands change at each and every prototype?
Are we doomed no matter what process we follow?
Answers: at least once, yes, and no. Yes, customers will ask for change every time they see it. No, you are not doomed. The reason is that the amount of change that occurs after a customer has the opportunity to see, touch, and interact with the intended implementation is small compared to the customer's response to the first tangible artifact of the process.
So, although we prefer the iterative model in our development projects, no matter what software development process model you use, it is mandatory that the development activity provide at least one robust evaluation prototype for customer feedback before the majority of the design and coding activity is performed. (Remember the prototype activity that Royce (1970) suggested in the initial waterfall model?) By reducing the amount of change to a manageable level, the developer succeeds by making incremental changes, typically in user interfaces, reports, and other outputs, to a robust and high-quality design and implementation. Thereafter, a rigorous process of finalizing design, coding, unit tests, and system integration activities will provide a solid foundation for the product while simultaneously greatly assisting the quality assurance and test activities.
Team Skill 4 Summary
In Team Skill 4, Managing Scope, we learned that the problem of project scope is endemic. Projects typically are initiated with approximately twice the amount of functionality that the team can reasonably implement in a quality manner. This shouldn't surprise us, as it is the nature of the beast: customers want more, marketing wants more, and we want more, too. We just need to make sure that we put ourselves on a diet sufficient to make sure that we can deliver something on time.
We looked at various techniques for setting priorities, and we defined the notion of the baseline—an agreed-to understanding of what the system will do—as a key project work product—our touchstone and reference point for decisions and assessment. We learned that if scope and the concomitant expectations exceed reality, in all probability, some bad news is about to be delivered. We decided on a philosophy of approach that engages our customer in the hard decisions. After all, we are just the resources, not the decision makers; it's the customer's project. So the question is, What exactly must be accomplished in the next release, given the resources that are available to the project?
Even then, we expect to do some negotiating; all of life, and certainly all of business, is a negotiation in a sense, and we shouldn't be surprised by this either. We briefly mentioned a few negotiation skills and hinted that the team may need to use these skills on occasion.
We cannot expect that this process will make the scope challenge go away, any more than any other single process will solve the problems of the application development world. However, the steps we have outlined can be expected to have a material effect on the scope of the problem, allowing application developers to focus on critical subsets and to incrementally deliver high-quality systems that meet or exceed the expectations of the user. Further, engaging the customer in helping solve the scope management problem increases commitment on the part of both parties and fosters improved communication and trust between the customer and application development teams. With a comprehensive definition of the product (Vision document) in hand and scope managed to a reasonable level, while it's too early to start bragging, we at least have the opportunity to succeed in the next phases of the project.