CONTENTS OF THIS CHAPTER

This chapter covers the following topics:

• introduction to system development lifecycles;
• what we mean by ‘system development lifecycle’;
• comparing linear and evolutionary approaches;
• lifecycles based on the linear approach;
• lifecycles based on the evolutionary approach;
• the impact of Agile;
• hybrid approaches;
• how to decide on an approach.

INTRODUCTION TO SYSTEM DEVELOPMENT LIFECYCLES

A system development lifecycle (SDLC) is a framework describing a process for understanding, planning, building, testing and deploying an information system. The process can apply to both hardware and software systems, as a system can be composed of hardware only, software only, or a combination of both.

Figure 2.1 The main stages of systems development

In any system development, there are a number of stages that must be undertaken. Although shown as a set of sequential steps, they do not need to be treated as such. Later in this chapter we will explore when and how often they occur in different lifecycles.

The stages are defined as:

Feasibility study

Before system development can commence, funding is required. The feasibility study involves investigation and research in order to evaluate the project’s potential for success to support decision-making. Such a study concerns itself with understanding objectively the resources and costs involved and weighing these against the value that will be attained following completion of the project or system. This is called return on investment (ROI) and only those projects and systems that return a reasonable ROI will be supported. The time spent in this stage will depend on how big and complex is the problem that needs to be solved within the organisation or business.

Requirements engineering

This stage aims to secure understanding of what the business needs the proposed system to do. To do this, effort is required from business analysts so that requirements can be elicited, analysed, documented and validated. Also, consideration is given to how the requirements will be stored and managed throughout the system development lifecycle, so that they that are accessible to those who need to see or use them and any changes can be managed. The requirements produced during this stage become the input to system design and so it is critical that the requirements can be traced all the way through the system development lifecycle from source to implementation. The amount of requirement detail captured when and where during requirements engineering can be affected by the lifecycle approach to be followed.

Design

In the design stage, possible solutions that meet the requirements are considered and weighed against each another. The chosen design is then elaborated in sufficient detail to allow the developers to implement it.

Development (programming)

Development is the phase where the technical components (both hardware and software) are created, procured or configured. The developers follow the design to ensure that the system does what is needed for the business or customer.

Testing

During testing, the components produced during development are tested to make sure they are working properly and that the system does what it is supposed to do. There are different levels of testing including unit testing, component testing, system testing and acceptance testing.

Implementation

Before an IT system is ready to use it must be commissioned into the ‘live’ or ‘operational’ environment. Up until this point in a system’s development, a reference or test environment may have been used in order to protect other systems from faults within the new system. Moving into the live environment takes careful planning and needs to be understood and managed as part of the overall development lifecycle.

As well as describing how to implement the main stages, a system development lifecycle also describes other elements that together comprise a particular approach that can be used to develop a system. For example, they may describe processes and methods, roles and deliverables, as well as tools and techniques. The elements of a SDLC are illustrated in Figure 2.2.

Figure 2.2 Elements of the system development lifecycle

The elements of the system development lifecycle to be considered are:

Context

Before beginning to develop a system, it is wise to consider the context in which we are operating. There are many aspects that can affect how we develop a system. Some examples are: Are we expected to deliver one complete system or will the end users expect multiple releases, or phased delivery? What skills and expertise do we currently have within the development team, and what is their preferred delivery style? Are the team and the key stakeholders in the same location? Is there a requirement for audit, quality or regulatory approval? Is our organisation expecting us to move to Agile techniques? How complex is the system we are expecting to deliver? Are the requirements likely to be well understood or could they change? Is this a tried and tested technology, or are we breaking new ground?

Understanding the context we are operating within help us select the right SDLC, and helps us to use it properly.

Lifecycle

The lifecycle describes the stages we will typically follow to plan, design, build, test and deliver information systems. Lifecycles are either linear or iterative. The linear lifecycle, often referred to as the ‘traditional’ or ‘step-by-step’ approach, takes the form of a sequence of steps, completed in order, with the final step being implementing the system into the operational environment. The ‘Waterfall’ lifecycle is one well-known example. An alternative to this is the evolutionary lifecycle, often referred to as ‘iterative’ development, whereby the system evolves through early prototyping and progressive versions of the system.

Process

In its simplest form, a process is a set of actions or steps that, if followed, will deliver a particular outcome. Although the lifecycle itself provides key development stages, it is also important to structure, plan and control the processes around those stages. To do this, we adopt methods and standards that apply a framework around managing the processes within the SDLC.

There are many different types of software approach to choose from, some of which lend themselves to a more linear lifecycle and others that are more suited towards an evolutionary lifecycle approach. Some are more prescriptive than others too, in that they mandate which lifecycle they are part of. Some also define specific roles, deliverables, techniques and even the modelling method. Others are more agnostic of lifecycle, and can be used in various types of SDLC.

When deciding upon which system development processes to use, it is important to make sure they are in line with the ‘context’ and ‘lifecycle’ that has been chosen or is adopted within the organisation.

Further information about software processes can be found in the section ‘How to decide on an approach’, where you can find useful tips to help decide which process might best suit your needs.

Roles

In order to progress through an SDLC, it is imperative that there are people who can carry out the tasks required. Many system development approaches outline the roles required for us. However, it is worth noting that many of the role titles vary between the different approaches. Below is a generic list of the key roles required within the system development lifecycle:

• business roles;
  • sponsor or senior responsible owner;
  • business analyst;
  • domain expert;
  • end users;
• project roles;
  • project manager;
  • team leader;
  • work package manager;
• technical roles;
  • technical architect;
  • solution developer;
  • solution tester;
• implementation and support roles;
  • release manager;
  • database administrator;
  • system administrator.

Deliverables

These are the documents, models, designs and hardware or software components that are required during the different stages of the SDLC. Again, different lifecycles and processes require different deliverables. The important thing to remember is that many of the deliverables are a way of detailing what is understood about the system in terms of what it needs to do, how it needs to do it, how well it does it and how it gets delivered. The deliverables include things such as:

• requirement documents;
• models such as:
  • class or entity relationship models;
  • use case models;
  • process models;
  • state transition diagrams;
  • sequence diagrams;
  • component diagrams;
• test plans;
• test scripts;
• implementation plans;
• system components and working software.

Techniques

There is a huge number of techniques that can be adopted during the development of a system. Which specific ones are used varies depending on team and organisation preferences, life cycle choice, and system development approach selected. For example:

• An organisation with a strong military product line may insist on the use of MoDAF (Ministry of Defence Architecture Framework – UK) or DoDAF (Department of Defense Architecture Framework – USA).
• Teams used to rapid prototyping or Test Driven Development may struggle with a linear approach and work better in an Agile model.
• Commercial projects with payment milestones that require formal sign-off and structured review dates may benefit from the rigour provided by a linear ‘Waterfall’ lifecycle.
• Detailed UML designs may not be appropriate where team members and project sponsors do not understand them.
• Another factor to consider when selecting the techniques is whether there are any formal standards or procedures that need to be adopted within the organisation.

Over the years, many development lifecycles have been created and, in some cases, marketed. They each have their strengths and weaknesses, and some work better for some types of systems and projects than others.

It is important to stress, however, that the lifecycle chosen must not be seen as a recipe to be followed exactly to the letter. Blindly following an SDLC without considering the context or system development approach could actually result in project failure. Each system development project is different in terms of scale and complexity and therefore the lifecycle approach used will often benefit from tailoring.

However, this tailoring must be done with caution, and changing an approach without considering the implications can also result in project failure. Used (and tailored) properly, most lifecycles can be used successfully with most types of system. But it is true that some will be a naturally better fit for some types of system development that for others. This chapter explores the various approaches that have been developed over the years and helps you choose the right one for your project/system.

WHAT WE MEAN BY ‘SYSTEM DEVELOPMENT LIFECYCLE’

There are a great many different system development lifecycles in use today. However, they are underpinned by just five basic lifecycles and two approaches.

The following two sections describe these seven core elements and help us to understand them. This is because understanding and being able to identify the core attributes of a lifecycle is critical to being able to properly evaluate its suitability for a project.

The first fundamental elements describe whether a lifecycle is linear or evolutionary:

• A linear approach describes a sequence of tasks that are completed in order, only moving to the next step once the previous step is complete.
• An evolutionary approach evolves the solution through progressive versions, each more complete than the last, and often uses a prototyping approach to development.

Then, there are five basic system development lifecycles:

• Waterfall:
• ‘V’ model;
• Incremental;
• Iterative;
• Spiral.

Before we explore these in more depth, we must first consider the scope of a system development lifecycle. Earlier in this chapter, we introduced the main stages of systems development, where the final stage is implementation. However, we know that the development of a system doesn’t end when it is first released, but rather continues as the system is updated, faults are identified and fixed and improvements are made until, eventually, the system is decommissioned. Throughout this period, the system also requires other maintenance and support activities such as backup, patching, audit and security protection.

However, most SDLCs only cover the development and transition to operating capability and do not include these ongoing activities and costs to support and maintain the system.

The additional costs beyond system development are sometimes referred to as ‘through life costs’ because they are the costs that will be spent on the system until it reaches its end of life. In most cases, these costs eventually vastly exceed the original development cost.

This is the difference between the whole system lifecycle, and the system development lifecycle, and can be categorised as follows:

System development lifecycle

The planning, understanding, analysing, building, testing and implementation of a new system. Delivery is often managed through a project structure, where time and budget are provided to deliver the system. Success is when the system is implemented and transitioned into the ‘live’ or operating environment. Once implementation occurs, the project is considered complete.

System lifecycle

The ongoing support and maintenance of a system that has been implemented into the ‘live’ or operational environment right the way through to its end of life or decommissioning. This is often managed through system managers or service owners and often funded through the business-as-usual (BAU) budget rather than project funding.

Not surprisingly, through life support often includes adding features and improvements to the system, which require some of the same tasks as developing the initial system. Such updates to the system often come in the form of service pack releases or system enhancements. The main difference is often one of time, budget and scope. Complex or expensive changes might necessitate a new system development phase, whereas smaller, simpler or urgent changes will be applied to the original system.

Since systems can have long lifetimes, particularly in some contexts (such as military), the costs accrued during the system lifecycle can far outweigh the costs spent during system development. Therefore, for many systems it is important to consider the through life costs of the system during the feasibility stage of SDLC and perhaps to modify the design accordingly. More information on this is provided in Chapter 13 ‘Maintenance and evaluation’.

Comparing linear and evolutionary approaches

We can categorise SDLCs into whether they are linear or evolutionary.

Linear approaches

Linear approaches follow the SDLC stages in a defined order. Each step must be completed, understood and agreed before the next step can be started.

In large, complex systems, this agreement between steps requires formal reviews by lots of people before the stage can be signed-off. It also places very definite break points in development that can be linked to project, payment or decision milestones. This rigour and involvement through formal documentation and review leads to a high quality product, as all the mistakes are captured and corrected before the next step starts. This is suitable for developing systems where the requirements are well understood, can be agreed up front and are unlikely to change.

In large, complex systems, where development is distributed or outsourced, a linear approach helps to control cost and scope, and allows development to be split between suppliers.

For smaller projects, a linear approach can be simple to implement, as the steps logically follow one another and can be easily understood by a small team. A smaller team can transition through the steps quite quickly too, so the documentation can be light as the information for each stage will be fresh in the team’s minds.

Some of the strengths and weaknesses of a linear approach are:

Strengths

• breaks down the problem into distinct stages, each with a clear purpose;
• everything is agreed in advance of being used, with no need to revisit later;
• helps provide structure to complex systems, making distributed or outsourced development easier to manage;
• suits a very detailed design decomposition and detailed specification approach;
• locking down each stage in advance of the next makes it easier to control cost and scope creep;
• simple and intuitive for smaller problems (people who don’t think they are using an SDLC are probably taking a linear approach).

Weaknesses

• depends greatly on each stage being done properly, as it is hard or impossible to go back and change it later;
• for complex problems, the time required to be thorough at each stage leads to long timescales;
• doesn’t cope well with changing requirements;
• customer or business value is not available until the end;
• if the project is stopped early there is little of business value to show for the cost.

Evolutionary approaches

Evolutionary approaches focus on progressing understanding through delivering early prototypes or iteration releases. The assumption here is that the requirements are not well understood and cannot be well articulated early in the lifecycle, or that early delivery is more important than completeness. Evolutionary approaches are the basis of ‘Agile’ development today.

Evolutionary development is ideal for new products, which might be innovation projects or green field developments, where the users find it difficult to express their requirements. Using this approach enables the system to evolve, uncovering more and more understanding through the demonstration of working software until the solution is reached. They are also ideal for large, complex systems, where early benefit must be realised in order to justify funding or to get quickly to market. In these examples, high value threads of benefit can be identified for implementation first, with richer functionality following later. This is common in web-based services or products.

An evolutionary approach is also used where there is high business or technical risk. Early versions can focus on exposing or mitigating that risk. If it turns out that the project is not feasible, it can be stopped without having spent much time on it, and all the work delivered so far has value.

Some of the strengths and weaknesses of an evolutionary approach are:

Strengths

• early delivery of value to the customer – either working versions or knowledge of project risk;
• copes well with complex requirements – fast changing, uncertain or complicated;
• encourages collaboration with the users throughout, so customer buy-in is higher;
• allows ‘just enough’ to be done knowing that it can be refined later on.

Weaknesses

• can be hard to project manage due to multiple iterations; especially hard with multiple iteration teams and complex products;
• without careful management, the evolving requirements can result in scope creep;
• overall costs can be higher due to the additional integration and test across multiple teams and iterations;
• easy to over-promise on early functionality.

LIFECYCLES BASED ON THE LINEAR APPROACH

Let us now explore the linear lifecycles. Of the five basic lifecycles, three follow a linear approach and these are:

• Waterfall;
• ‘V’ model;
• Incremental.

The Waterfall lifecycle

The most common linear lifecycle is ‘Waterfall’. This was one of the first lifecycles introduced back in the 1970s and it still holds much weight and value today. The main stages of the Waterfall lifecycle are illustrated in Figure 2.3, which shows a sequence of steps or stages that cascade down giving it a Waterfall-like look, hence the name.

The principles behind Waterfall are that each step should be completed before moving onto the next step. So, the feasibility study must be completed before the requirements engineering step can be started, and the requirements engineering must be completed before the design can be started, and so on. Completing the steps in this way ensures that the system is well understood through formal documentation and review and therefore should lead to a quality product.

The dotted lines and arrows show that if problems occur in any of the steps, then communication and understanding must also flow backwards to rectify the problems. For example, if during programming a problem occurred whereby a particular requirement could not be satisfied, then the programming would halt whilst the design step was reconsidered in light of the problem and if necessary the requirements engineering step would also be revisited.

Successful Waterfall development requires strong project management, accurate estimation and sound design. This implies that the system must be well understood prior to the start. Otherwise, additional time can be spent in the earlier stages of the lifecycle, getting the requirements and design right and leaving less time in areas in the latter stages, especially testing. Development can take longer than expected due to ambiguity or error in the design, and this can also increase pressure on testing. This tends to result in a poorer quality product; timescales being pushed out, leaving customers to wait even longer for the delivery; or costs increasing as the project goes into overdrive to meet quality and/or timescale demands.

Since requirements are elicited, documented and signed off very early in the lifecycle, Waterfall projects are vulnerable to changes in requirements. Once the requirements stage is complete, changes become increasingly expensive, as the team must revisit the requirements stage and all stages afterwards. To manage this, formal change control procedures and processes are implemented to manage change or, in some cases, even protect the project from change. In fast-moving customer environments, this can present a challenge.

Figure 2.3 The Waterfall lifecycle

The strengths and weaknesses of the Waterfall lifecycle are:

Strengths

• good for developing systems where the requirements and likely solution are well understood and unlikely to change;
• rigour and control around each stage makes it easier to project manage, with obvious hooks for project milestones;
• simple to understand and easy to use due to rigidity of the stages.

Weaknesses

• can be costly to manage change, especially if change to requirements is identified during testing;
• working software not delivered until the final stage, implementation;
• poor estimation can severely impact project cost and schedule;
• customer collaboration is limited to requirements and user testing; on long projects this can lead to a lack of customer buy-in, particularly where stakeholders change.

The ‘V’ model

The ‘V’ model was introduced almost as an extension to the Waterfall model to deal with the problem whereby time allocated to testing could often get squeezed during a Waterfall project if time and money started to run out. To address this, the ‘V’ model, shown in Figure 2.4, changes the way that the development stages are represented from a ‘Waterfall’ to a ‘V’.

This makes a distinction between the activities on the left hand side of the ‘V’ (project definition) and the right hand side of the ‘V’ (verification and validation). The bottom of the ‘V’ is therefore the actual development that takes place to translate the project definition into working software that can be verified and validated.

The dotted lines connecting the left-hand side with the right are a fundamental difference from the Waterfall model, and demonstrate the relationship between each stage of definition with its associated stage of testing. The model is implemented from the top down, meaning that test artefacts are created at the same time as the corresponding requirements, helping ensure that the testing stages will be testing the right things.

This early thought and rigour around testing leads to an approach that focuses on delivering a high-quality system.

Figure 2.4 The ‘V’ model

For this reason the ‘V’ model is particularly good for safety-critical systems where a single release of the system is still anticipated, but high quality and safety are critical. The design of stages, particularly the separation of high-level design and unit/component design, and the inclusion of ‘integration testing’ lends itself well to complex problems where there are multiple teams or suppliers involved in the development stage.

It is simple to see how system development responsibilities can be split between teams or suppliers in order to divide up a complex or large-scale task.

An extended ‘V’ model, illustrated in Figure 2.5, was later introduced to include the feasibility and benefits realisation stages. This produces a way of measuring and testing how well the actual business benefits were delivered, as opposed to testing when the requirements were delivered. After all, we could have a system that verifies and validates all the requirements, and yet fails to meet the need of the business because it does not deliver the business benefit.

For example, a system to maximise staff deployment based on skills and experience could easily result in a system that records employees skills and experience. This would probably meet many of the requirements but not the benefit of improved staff deployment.

The extended ‘V’ model tends to be more popular today than the simple Waterfall, and is used in many system engineering projects. This shows its versatility and ability to scale massively. When used with formal systems engineering frameworks like MoDAF, the ‘V’ model can be used to manage extremely large and complex systems like aircraft or ships.

Figure 2.5 The extended ‘V’ model

Strengths

• additional focus on testing leads to a high quality product;
• ideal for systems with complex or high quality attributes (such as safety critical, challenging to integrate or very large);
• easy to plug in highly complex supply chains and distributed teams due to clear interfaces and boundaries;
• rigour and control around each stage makes it easier to project manage;
• easy to scale.

Weaknesses

• heavy testing and integration cost that can be unnecessarily expensive for simpler systems;
• cost and impact of change increases significantly as further progress down and up the ‘V’ is made;
• early stages take longer to complete as there are test artefacts to complete;
• working system not delivered until the final stage, which could be a long time after the customer was last involved;
• development teams can often omit to put enough effort into planning the right-hand side as they progress down the left; This significantly increases risk, particularly in the integration of complex systems.

Incremental lifecycle

The Incremental lifecycle, shown in Figure 2.6 is still a linear approach in so much as the first three stages of the SDLC are conducted in sequence, each being completed before the next is started.

Where the Incremental lifecycle differs from the Waterfall or ‘V’ model is that after fully understanding and designing the system, decisions can then be made to deliver some parts of the system before other parts. This enables high-priority requirements to be delivered first, thus delivering elements of functionality to the customer or business early. Increments can be delivered in parallel, as the core design is complete.

In this way, the customer is able to get a working solution quite early in the development process. This addresses the problem of one final stage delivery, whereby the user has to wait a long time between providing requirements and seeing the results. For this reason this lifecycle is often referred to as Incremental delivery.

The Incremental lifecycle is ideal where early capability is required, perhaps for rapid time to market, but where the core architecture is stable. It is also useful where the functionality required is clear, but there are options over what aspect is most important to be delivered first. Because the high-level system design is independent of delivery order, the decision on delivery order can be left until nearer the time.

Figure 2.6 The Incremental lifecycle

This lifecycle still requires rigorous analysis and design work in the early stages, as these provide the foundation for the last releases, not just the first. The additional releases also add some cost, as release costs (such as user training, deployment, system outages, etc.) are incurred multiple times. Subsequent releases must be regression tested against earlier functionality. This means that features in the first release are re-tested in every subsequent release (to ensure that they still work properly), adding to the test cost. This cost must be weighed up against the value of early and more frequent releases.

However, the Incremental approach can also reduce costs. If the organisation decides that the delivered functionality of the earlier increments is sufficient, then the project can be stopped at that point and no further cost is incurred.

Strengths

• delivers working software to users early;
• easier to manage risks because risky pieces can be identified and handled in early increments;
• easier to test and debug smaller increments;
• increments can be built in parallel, reducing overall time to market.

Weaknesses

• hard to manage parallel deliveries;
• still difficult to manage change, as requirements are agreed before increments decided;
• overall costs can be higher due to additional overheads of parallel implementation and additional ‘regression testing’.

LIFECYCLES BASED ON THE EVOLUTIONARY APPROACH

The final two core lifecycles we will describe are evolutionary approaches. Evolutionary approaches have been around since the 1980s and underpin the more popular ‘Agile’ approaches today. Although there are many processes and approaches that follow the evolutionary approach, there are really only two fundamental lifecycles to explore – they are:

• Iterative;
• Spiral.

Iterative development

Iterative development is a lifecycle whereby the overall project is composed of several small time-boxed developments, called iterations, which take place in sequence.

Each iteration is a self-contained mini project composed of activities such as requirements, analysis, design, development and test; in this respect, they can look a bit like mini-linear projects. Requirements are elicited during each iteration and then developed to produce a working system that delivers some value to the project or customer. On big projects, this could be multiple teams all working on the same iteration.

In doing this, the understanding of the overall solution evolves as the requirements evolve through the iterations. At the end of each iteration, all development across the teams is integrated and tested into an iteration release that may not, initially, be released externally.

As Figure 2.7 illustrates, a minimum of three iterations would typically be required to achieve an external release, and these iterations would consist of:

Investigate

The initial iteration is used to investigate things such as risks around the technology or risks around user functionality. Once this iteration is completed there will be more understanding about what is required to deliver the external iteration or first release.

Refine

During this iteration the requirements are fully detailed and designed and the system is developed to meet the business goal. There are sometimes multiple refinement iterations.

Consolidate

The final iteration, prior to external release, ensures that the code and design developed is stabilised and fully integrated with any other components or iterations so that the overall business goal is achieved in the working system, the external release.

Figure 2.7 The Iterative lifecycle

The earlier iterations are mostly for the development team to prove technology or explore and reduce risk and so are not always released. Due to this, the approach is often thought of as Iterative development, as delivery can still be through one big final stage release. To avoid this final stage release, Iterative development is often combined with an Incremental delivery approach and is referred to as Iterative Incremental Delivery (IID). Combining Iterative development with Incremental delivery is very common and is widely used today.

Iterative lifecycles are good where the problems or solutions are not clear at the outset, or where the business does not know yet if it wants to fund the project. The early iterations can give greater clarity over what is feasible and how much it may cost. It is also an approach well suited to prototyping, testing the market or proving new technology.

Strengths

• requirements evolve through each iteration;
• encourages collaboration with the users throughout so customer buy-in is higher;
• change is easier to manage as requirements are not locked down early;
• cost can be controlled.

Weaknesses

• can be hard to project manage due to multiple iterations and multiple iteration teams;
• without careful management, the evolving requirements can result in scope creep;
• if poorly managed, can still appear like a single, final release;
• overall costs can be higher due to the additional integration and test across multiple teams and iterations.

Spiral

The Spiral lifecycle, shown in Figure 2.8, was developed by Barry Boehm in 1986. The Spiral model combines Iterative development with prototyping to test understanding and evolve the requirements so that risks can be addressed throughout. The Spiral model, like the Iterative approach, does not need requirements to be defined in advance. This approach starts with a concept and repeatedly revisits the four main phases of development until a suitable working product is complete.

The quadrants of the Spiral model are as follows:

Determine objectives

In the first stage, the development team and business owners identify the objectives for the development and agree prioritisation of requirements. Throughout the project this understanding is constantly reviewed and revisited.

Identify and resolve risks

At this stage, the developers explore technical possibilities to meet the business goal and assess any risks that the technology may bring. An early prototype is a key enabler here and will be produced, in collaboration with the business users, to address the risks.

Development and test

Reviewing the prototypes enables the development team to assess how well the requirements and risks were addressed and look at improvements. Designs are then developed for another prototype, which is agreed, developed and tested. After a few iterations this will produce the operational prototype.

Plan the next iteration

During this stage, the working system is released into the live operational environment. This may just be an Incremental release and therefore further releases will be planned and developed through subsequent iterations and releases until the complete system is developed and the business objectives are fully met.

Strengths

• risks are addressed early so risk avoidance is enhanced;
• requirements evolve, so additional functionality can be added at a later stage;
• collaboration with users throughout is key, so customer buy-in is good;
• a working system is produced early in the development lifecycle.

Figure 2.8 Boehm’s Spiral lifecycle

Weaknesses

• there is a high chance of scope creep due to evolving nature of requirements;
• overall costs can be higher due to the many implementations of working software into the operational environment;
• as the focus is on a working system, documentation of the system can become de-prioritised, making the system harder to maintain post-project delivery;
• it can seem like a lot of iterations are required, so can be a poor choice for small teams or simple problems.

THE IMPACT OF AGILE

Agile is a popular term now given to evolutionary development approaches following the ‘Agile Manifesto’ in 2001. The manifesto was developed by 17 leading IT professionals, who felt that IT system development was failing to meet the growing challenges expected by businesses today. The Agile Manifesto states:

The Agile Manifesto led to a whole new way of thinking about software development and a new craze in Agile development methods, processes and lifecycles emerged, as shown in Figure 2.9.

Linear lifecycles such as Waterfall, ‘V’ model and Incremental fell out of favour in place of more evolutionary approaches such as Scrum and the Agile Unified Process (AgileUP).

For many development teams, this provided a means of throwing out bureaucracy and formal documentation so as to focus more on producing working software. Methods such as Scrum became very popular and lightweight approaches were attractive as they were easy to pick up and start using almost immediately.

Figure 2.9 Evolution of Agile

However, like all development methods, the key to success is to understand the method and apply it properly. In order to be successful, many Agile methods depend on high rigour and adherence to rules in order to succeed. While they may not depend on high levels of documentation in advance of development, they do depend on following highly rigorous processes.

Some of the lightweight Agile methods focus purely on the development of the software – and do not cover the stages prior to coding – or on the overarching governance and architecture. Conversely, most lifecycles do not define how the system is developed. This means that, in many cases, Agile approaches can integrate well into more traditional approaches, particularly where the principles of the Agile Manifesto are maintained.

Because the history of many Agile approaches is in small, highly collaborative teams, they work very well in those kinds of environments. However, they are much harder to work well where that environment is challenged. There are a number of factors that can make Agile much harder to work in practice:

• larger teams, particularly where they are geographically distributed;
• complex systems;
• audit, regulatory or safety critical requirements that expect higher quality assurance;
• projects with strict contractual commitments;
• complex user environments, or where the end user is not available;
• sub-contracting into a project being run in a non-Agile way.

In these situations, Agile approaches can still bring considerable benefits, but the additional risk that they also bring must be mitigated in some other way. This can be through the combination of Agile with other approaches, so that the whole scope of the systems development lifecycle is covered and additional rigour can be applied where necessary.

HYBRID APPROACHES

As we have seen, there are many popular lifecycles that are used and followed today and all of them have their roots in one or more of the linear or evolutionary lifecycles already discussed. Some of them are pure versions of one of the core lifecycles described earlier, but there are many more that are a mix of the seven elements already described.

DEVELOPMENT APPROACHES AND METHODS

This section reviews some of the more popular lifecycles and provides some history and background into each.

SSADM

The Structured Systems Analysis and Design Method (SSADM) was first released in 1983, although its origins go back as far as 1980. SSADM is a linear Waterfall approach, which focuses on the analysis and design for developing information systems and was produced for the public sector for the Office of Government Commerce. The main difference between SSADM and Waterfall is that SSADM adopts principles from Peter Checkland’s Soft Systems Methodology, which considers wider influences on the system than just the IT or application development. The stages of SSADM are:

• stage 0 – feasibility study;
• stage 1 – investigation of the current environment;
• stage 2 – business system options;
• stage 3 – requirements specification;
• stage 4 – technical systems options;
• stage 5 – logical design;
• stage 6 – physical design.

Taking the wider business approach and adding in the additional stages 2 and 3 meant that SSADM provided a more through understanding of the wider business perspective and processes. However, some SSADM projects ran into difficulties as the ever-changing business environment meant that earlier stages had to keep being revisited, causing delays to the later stages and hence delivery. Later versions of SSADM introduced a more dynamic approach to combat this. Modelling is key to SSADM and the three key models are:

• logical model – models the data requirements and business rules;
• data flow model – models how data moves around an information system;
• entity event model – models the behaviour of the entity, or data, and the events that cause the entity to change.

These techniques are prevalent in system development today by the same name or in the form of Object Oriented Analysis and Design (OOA&D).

DSDM

The Dynamic Systems Development Method (DSDM) is an evolutionary Iterative and Incremental approach that was originally developed in 1994 as an Agile project delivery framework to support the trend towards ‘rapid application development’. With its roots in Agile, DSDM was extended in 2003 to support the Agile programming process called ‘Extreme Programming’ (XP). Shortly afterwards, DSDM recognised that IT projects can often fail because of the people and process changes rather than the IT itself, and this led to the launch of DSDM Atern (now re-named as simply DSDM again).

DSDM has three main phases called pre-project phase, project lifecycle phase, and post-project phase. The project lifecycle phase is further sub-divided into five stages that include: feasibility study, business study, functional model iteration, design and build iteration and implementation. As well as the phases and stages outlined, DSDM also defines eight core principles that are at the heart of the framework which are:

1. focus on the business need;
2. deliver on time;
3. collaborate;
4. never compromise on quality;
5. build Incrementally from firm foundations;
6. develop iteratively;
7. communicate continuously and clearly;
8. demonstrate control.

DSDM is a flexible process that is highly configurable. It doesn’t provide guidance on how the development should be done, leaving that to the team to decide. The core elements specify a time-boxing approach and a business focused governance. These make it a good choice for business focused projects, where the implementation methods are less important.

Needless to say, these principles, coupled with the evolutionary and Incremental approach, are what makes DSDM a key player in the Agile lifecycles choice. However, the amount of focus on time-boxing and governance, has led some to believe that DSDM Atern is more a project management framework than a development lifecycle.

Scrum

Scrum is probably one of the most well known of the Agile approaches used today and was first published in 1995 by Ken Schwaber and Jeff Sutherland. Scrum is an Iterative and Incremental process that follows an evolutionary approach to the development of software systems. It introduces some unique language and concepts that govern how work is planned and monitored by a small team of multi-disciplinary software engineers. Scrum only proposes three roles which are:

• product owner – represents the business and is the voice of the customer;
• development team – cross-functional, seif-organising team of 5–9 individuals who deliver the working software;
• Scrum master – person accountable for removing impediments so that the development team can deliver the goals.

The core element of Scrum is a ‘Product Backlog’, a prioritised list of requirements that the team will be delivering. The ‘Product Owner’ holds control over the priority of the items on the backlog.

Iterations are called ‘Sprints’ and will generally be short – perhaps two weeks – and focus on delivering a working version of the product, as defined through agreement with the product owner. The sprint is monitored by the eponymous ‘Scrum’, a short daily, highly-structured meeting that is focused on removing barriers to success. The team all work together to deliver the items on the Backlog, which could relate to any stage of the development lifecycle (requirements, analysis, design, implementation, etc.).

Scrum works really well for project teams of 5–9 developers, where the team is capable of doing all the work necessary for the product, and where there is often candidate architecture already in mind. Problems can arise, however, if the combined team do not possess all of the skills required. Additionally, as Scrum begins with the Backlog, the wider business approach can sometimes be overlooked, which can lead to innovative products that don’t necessarily deliver the expected business benefits.

Scrum can be scaled, but this needs to be done with caution as it can be easy to break. Where Scrum works less well is where there is poor stakeholder engagement – if the team needs to stand in for the Product Owner, then alarm bells should ring.

Rational Unified Process 1998

The Rational Unified process (RUP), developed originally in 1998 by Rational Software (now IBM), is a software engineering process. It is a risk-driven, evolutionary iterative and Incremental process, built upon the foundations of the Spiral model, that assigns and manages tasks and responsibilities in a disciplined way within a development organisation. RUP is also a process framework that be adapted and tailored to suit the needs of the organisation or project team. RUP consists of four major phases, which are Inception, Elaboration, Construction and Transition as well as six core engineering workflows which are; business modelling, requirements, analysis and design, implementation, test and deployment.

Although the workflows are similar to those within a Waterfall approach, the iterative nature of the process means that the workflows are revisited again and again throughout the lifecycle and across all four major phases of RUP.

Because RUP is a fully-described process that can be used to develop highly complex systems, it is important to tailor it to the scope, risk and size of each project. However, the complexity and flexibility of RUP can make this tailoring hard to do, and some teams end up using more of the process that they really need to, with consequential increases in cost and complexity.

Being a software process, RUP is popular with teams of developers, and makes extensive use of UML (the Unified Modelling Language) for Object Oriented Analysis and Design (OOA&D) and case modelling. This can make it overwhelming to newcomers and business stakeholders, but does encourage strong design and good traceability.

Lean Software Development

Lean Software Development (LSD) has its roots in lean manufacturing, pioneered by Toyota. This is a set of techniques and tools that can be applied to eliminate waste in manufacturing and hence improve productivity. Lean Software Development was introduced by Mary and Tom Poppendieck in their book Lean software development: an Agile toolkit (2003), and translates lean manufacturing into software development principles. These principles are:

• focus on customers;
• energise workers;
• eliminate waste;
• learn fast;
• deliver fast;
• build quality in;
• keep getting better;
• optimize the whole.

Although Lean Software Development does provide tools and techniques, its focus is on reducing waste within the process of developing a product. It does not provide details as to the stages you need to take or the development techniques you should use, as it expects that you already have an approach you are following. Due to this, Lean Software Development is an ideal approach to be combined with other methods or approaches, such as Scrum for instance.

There are more development approaches than the ones mentioned in this chapter. Below is a list of some of the additional approaches you may wish to investigate:

• Kanban:
• Xtreme Programming;
• Agile Unified Process (AgileUP);
• Disciplined Agile Development (DAD);
• Rapid Application Development;
• Crystal Methods;
• Scrumban.

HOW TO CHOOSE AN APPROACH

This chapter has introduced core elements that can be used to describe any systems development lifecycle. it has also introduced some of the commonly used processes.

So how do we know which approach to follow? Well the answer is ‘it all depends!’ Used properly, any SDLC should enable us to deliver a successful project; used poorly, none of them will. The trick is to try to identify an SDLC that will give the best chance of the project succeeding.

Some examples of factors to consider are discussed here. Alternatively, we could find someone to discuss the project with, and use the descriptions in this chapter to help us work it out.

Complexity of problem

For simple problems, choose a simple method such as Waterfall.

For complex problems, choose an approach where the complexity is reduced early, and risk is understood early, such as RUP, ‘V’ model or spiral.

Team experience

For risky problems, choosing an approach the team is comfortable with is probably has the best chance of success – unless we think they are failing because of the method. For simple problems, we could take the opportunity to pick something new, knowing that the project is likely to still succeed.

Stability of requirements

If we are confident the requirements will not change through the life of the development, any approach will work, but those that keep revisiting the requirements stage (like the evolutionary ones) will cause unnecessary re-work. For rapidly changing requirements, choose an iterative approach with short iterations (like Scrum). For less rapidly changing requirements, any of the evolutionary or iterative methods should do.

Delivery speed, and quality

If the customer needs the system to be complete before delivery, we have a lot of choice. A linear method would work well, but if we have other reasons to prefer an evolutionary approach (to expose technical risk early, for example) then we can use one but not release each iteration to the customer. We should probably avoid an approach, like Scrum, that requires heavy customer involvement unless they are happy with that. If the customer is anxious to see results quickly, then pick one of the evolutionary approaches, or the Incremental approach if there is a stable architecture.

Customer involvement

For low customer involvement, linear approaches work well, although with an Incremental approach we need to know in what order the customer wants the delivery made. Evolutionary approaches tend to require more customer involvement, particularly Scrum, which needs the customer to be a virtual part of the team – if the customer can’t manage that, Scrum probably isn’t the best choice.

Uniqueness

If the project has never been done before, pick an evolutionary approach that allows us to test architectural risk early (and more cheaply), and to re-scope the problem or solution if required.

High regulatory requirements

More established technologies should work with any approach. If we need to meet regulatory, legal, audit or safety critical requirements, then choose an approach that includes documentation and assurance – Waterfall and ‘V’ model work well – or for Agile development, perhaps DSDM.

REFERENCES

Boehm, B. (1986) ‘A spiral model of software development and enhancement’. ACM SIGSOFT Software Engineering Notes, August.

Checkland, P. (1981) Systems thinking, systems practice. John Wiley and Sons, Chichester.

Kenneth, R. (2012) Essential Scrum: A practical guide to the most popular Agile process. Signature series (Cohn). Addison-Wesley, Boston, MA.

Kruchten, P. (2003) The rational unified process an introduction (3rd edition). Addison-Wesley, Boston, MA.

Poppendieck, M. and Poppendieck, T. (2003) Lean software development: an Agile toolkit. Addison-Wesley, Boston, MA.

Royce, W. (1970) ‘Managing the development of large software systems’. www.cs.umd.edu/class/spring2003/cmsc838p/Process/Waterfall.pdf (accessed 4 June 2014).

FURTHER READING

Ambler, S. W. and Lines, M. (2012) Disciplined Agile delivery: a practitioner’s guide to Agile software delivery in the enterprise. IBM Press, Indianapolis, IN.

DSDM Consortium (2008) DSDM atern: the handbook. DSDM Consortium, Ashford.

Paul, D., Yeates, D. and Cadle, J. (2014) Business analysis (3rd edition). BCS, Swindon.