‘Things change. The only thing constant is change. It’s up to you to be adaptable.’
– Anonymous
Anyone who tries to make a cake simply by guessing the recipe and method will very quickly discover that the outcome may not be a good cake. Most of us will reach for a recipe book. In the same way, when embarking on a software development project, using a methodology to provide a framework and sense of direction will ensure a better outcome. This chapter explores a range of possible approaches.
DEVELOPMENTAL APPROACHES
At the heart of a new software development project is the need to solve an identified problem. But what is the best way to tackle this and communicate ideas?
Unfortunately, there isn’t a single ‘one size fits all’ framework, methodology or technique that will solve all your problems – that would be too easy! Moreover, different organisations adopt different methods and tactics to solve their problems. However, there is a reasonable starting point and that is the software development lifecycle (SDLC).
The software development lifecycle
A software development project usually follows a set process with a series of defined (and predictable) steps. This is known as the SDLC. It usually begins with the identification of an internal problem or aspiration, often in the form of a brief. Reviewing the product at the end of the lifecycle can trigger a new development.
The lifecycle contains a series of phases, each one of which contributes a process to the development (see Figure 4.1).
The length of a SDLC varies greatly depending on the size and complexity of the problem it is attempting to solve. Despite this, the lengths of each stage generally remain proportionate – for example, the coding phase is longer (potentially many months) compared to a shorter design and architecting period (perhaps several weeks).
The basic components of the lifecycle (up to the point of roll-out) are as follows.
Brief
This triggers the development process. It could be based on a problem with an existing system or process, something new that is necessary to meet a business requirement, or simply a response to an emerging opportunity based on a new idea. It may occur as the result of an organisation’s aspiration to provide a better customer experience, improve its processes or reduce costs (or all three!).
Figure 4.1 The traditional phases of the software development lifecycle
Feasibility
This is a top-level investigation into the brief which essentially considers the problem to be solved and investigates potential solutions. These solutions can include:
See ‘How the Client Brief Affects the Development Process’ later in this chapter for more on potential solutions.
Analysis
Once it has been established that a development is required, a full analysis of the problem and the existing system (and how it could contribute to the solution) is undertaken. This is documented to ensure that subsequent decisions can be justified.
Planning
This is a very necessary, formal part of the process which can sometimes be overlooked. It is where expected activities and processes needed to achieve the solution are identified, along with anticipated timeframes. Available resources are then established and allocated to the tasks that will need to be undertaken.
Design
The completion of the analysis leads to the design of the new system. This may or may not be formally documented (depending on the nature of the development and the methodology being used to focus activities).
Development
This phase involves the actual coding of the new solution, system or service.
Testing
A range of testing strategies may be applied in software development. Some are undertaken during development, others when the development is considered relatively complete, and others during a period of final testing before the developed system is handed over to the client.
Roll-out/implementation
Once the code has passed testing, it will work flawlessly in a staging environment and be ready to go ‘live’ and into production. At this point, the system is deployed (installed and prepared) for use by internal and/or external customers.
The client will look to the developers for advice about the implementation of the software or system so that it can be integrated appropriately into the business (often called a ‘changeover strategy’). There is no single right answer to the question of how the solution should be implemented, which is why the development team must consider the needs of the client, the risk factors, and the amount of additional work that will be incurred through the implementation activity in the advice they give.
Common to every changeover strategy is the need for the organisation’s data to be added to the new system so that when it goes live it has something to work with. This may require changes to the data to prepare it for integration, such as converting data formats or reorganising the data to prepare it for transfer. The data itself will usually be moved to the new system at a time when normal business transactions are suspended because the organisation is closed. This can be more challenging if the business operates online.
Table 4.1 explores the most commonly used changeover strategies, the risk factors and additional considerations related to them.
Table 4.1 Changeover strategies
Changeover strategy | Risk factors | Additional considerations |
Direct: once data has been added, the existing system is switched off and the new system takes over. |
| This is the most straightforward changeover strategy because it does not require any additional work. |
Parallel: the existing and new systems are run together. This allows the operation of the new system to be checked by comparing it to the output from the existing system. |
| This is the most labour intensive changeover strategy because every transaction has to be recorded twice. It is only a short-term solution as it creates additional stresses and workload for staff. |
Pilot: the new system is introduced in one area of the organisation first, such as a branch office or shop. When this has been proved successful, the system is rolled out to other branches or shops until all parts of the organisation are using the system. |
| The main issue is that as with a direct changeover, the existing system is switched off, so the risk factors are similar. However, the risk is less as any errors would impact a smaller area of the business. |
Phased: this introduces the new software in steps or stages. As an example, consider the sales function of an organisation where the sales force is split into geographical teams. One team could be moved to the new system and, when this has been proved successful, another team could be added. |
| The main issue is that as with a direct changeover, the existing system is switched off for those being moved across, so the risk factors are similar. However, the risk is less as any errors would impact a single team or a small number of teams in an area. |
A common tactic employed in the IT sector is the use of a so-called soft launch. Prevalent in software releases and commercial websites, in a soft launch the product is made available to a selected audience without any fanfare or marketing.
For a software product this can be used as a form of beta testing, allowing the organisation to test its product ‘in the wild’ without its reputation being affected if the results are not favourable. In addition, it will receive useful bug reports and usage statistics that will help it to resolve identified issues or further improve the product’s performance before full release.
Commercial websites often use such soft launches to roll out new features or make modifications of the user interface. Again, user feedback is invaluable in shaping the next stage of development.
Maintenance
Designed. Coded. Tested. Implementation complete. The new solution is in production and the client appears to be happy. The next step is to ensure ongoing maintenance. Once development is complete and the system has been implemented, a maintenance strategy is put in place to support the system going forward.
Once it is operational, the system is examined for any issues that have arisen as a result of its use and identified errors are fixed. For example, a user might complain about tabbing errors, which is where input boxes are not presented to the user in the right order and the user finds themselves jumping around the screen. This does not indicate that the product is faulty; it is essentially an irritation that really should be eradicated but which probably will have a low priority. If, however, a user discovers that sales records are not being correctly committed to the sales database (for instance), this would be a high-priority issue that would need to be resolved quickly.
Maintenance activities usually fall into one of the following three categories.
Corrective maintenance
Corrective maintenance is reactive. It is usually prioritised because it aims to resolve significant problems with the system that have emerged during its use. Bugs and other defects need to be eradicated quickly for two reasons: firstly, because the product will not be working as it should until these are resolved, and secondly, because users may well view the product with increasing negativity if bugs persist.
Typical bugs include calculations not outputting accurate answers, and button clicks or hyperlinks not working.
Perfective maintenance
Perfective maintenance is also reactive. Following corrective maintenance, perfective maintenance resolves other issues. These issues do not prevent the system working but are known to be irritating to users.
Tabbing errors (as mentioned above) are one example of an issue that would be addressed during perfective maintenance. Another example could be the slow loading of a web application, caused by too many extraneous HTTP (Hypertext Transfer Protocol) requests during initial loading. In such cases a developer will use specialised tools to review and improve the design. For example, a pipeline profiler might be used to analyse the performance of a web page while it loads. This type of activity can often be the developmental equivalent of diving down a very dark and deep rabbit hole, and it may lead to the unearthing of graver concerns, such as CPU load issues and memory usage inefficiencies. Perfective maintenance is time-consuming but it may not garner the same amount of resources as corrective maintenance, which is perceived to be more critical; in other words, perfective maintenance is a ‘nice to have’ but not essential.
From a purist’s point of view (and in an ideal world), anything that negatively impacts a user’s experience of a software product should be considered critical. However, time and money are always at a premium in any development project, so higher-priority issues will typically win out.
Adaptive maintenance
This is a longer-term form of maintenance and is usually a response to the organisation’s ever-changing environment or its future aspirations. New technologies could emerge which the business would like to exploit. The organisation may have expansion plans, such as selling new product lines or services which will mean higher transaction volumes or larger numbers of users. In some cases, adaptive maintenance may even be required in response to the activities of business rivals and the need to be more competitive.
Adaptive maintenance is proactive because the organisation is able to plan the maintenance activities that will be needed to prepare for its future plans.
It is certain that your organisation will be involved in corrective maintenance, and it is likely that it will also be involved in perfective maintenance. Adaptive maintenance may well initiate another development project and the SDLC will start again.
Review
At the end of the SDLC, a review is undertaken which not only examines the success (or failure) of the product but also examines the development process (from the receipt of the client brief through to the implementation of the software), decisions taken and so on. Usually involving a range of stakeholders, this activity provides the development team with an overview of their activities and what went well or badly. It also provides an opportunity to consider how future developments could be improved based on lessons learned.
A good project manager will produce a checklist that covers all of the development activities so that they can be individually inspected. This will provide a useful focus for the evaluation process and a framework for evaluation, enabling subsequent development activities to be examined in the same way. In this way, it can prove or disprove whether lessons learned have really affected the organisation’s development processes.
Here is a very basic checklist of what could or should be considered under each phase:
The review phase of the development should be extensive and the activity should be undertaken with commitment, particularly as there may be a tendency to want to end the current development project so that the next one can begin. Understanding what went well and where improvements could have been made will be invaluable experience for the next development.
HOW DEVELOPMENTS GO WRONG
A suspicious but worldly wise project manager once said, ‘What could possibly go wrong?’ Given the long gestation time of many projects, the answer is likely to be, ‘Quite a lot.’ (For example, see Maughan 2010.)
Here are a few things to think about – always remember, forewarned is forearmed!
We previously discussed the importance of formally agreeing the boundaries and requirements of a project at the outset to manage the client’s expectations and to focus the activities of the development team.
In an effort to keep the client happy, it is very common for developers to try to accommodate requests from the client that emerge during development. Unfortunately, the client often has little or no understanding of the impact of the request they are making in terms of additional work – and even, in extreme cases, how what might appear to be a relatively minor change may have a detrimental impact on other aspects of the development.
For example, adding an additional data field to customer details may not seem particularly problematic. The reality, however, could be that doing so would affect data structures far beyond this entity. For example, on every form and screen where the data was displayed, the code that drew the detail from the customer file would need to be modified, and the code that generated any reports might also need to be changed to accommodate the additional values.
Accommodating these changes would be an example of scope creep. This in itself is not an issue, particularly when you remember that methodologies such as Agile and Scrum (covered later in this chapter) are built on this concept as part of normal development activities. However, the reality is that if the level of scope creep is not carefully managed, there will be two potential outcomes:
To avoid scope creep, all team members involved in a development project should be alert to ensure that additional client requests do not derail the project. However, this does not mean that you should immediately dismiss any additional requests made by the client. Ideally, as a development team, you will have discussed at the outset how you would respond to any client requests that are not specifically included in the requirements specification.
One well-used strategy is to raise an invoice for all but the most minor requests. This outlines the additional cost that would be incurred by the client for the inclusion of the requested changes, together with a revised delivery date for the project. It is then up to the client whether they want the amendments to go ahead or not. If not, the organisation simply raises a credit note to cover the invoice.
On a positive note, accommodating scope creep can be one way to produce extra revenue! The outcomes are then:
The conflicting demands of stakeholders
Never underestimate the impact that the conflicting demands of stakeholders can have on the outcome of a development project. It is essential that any conflict is eradicated as early as possible in the project. This may not be easy and may require a number of meetings involving heated debate before agreement is reached. The stakeholders should not only agree the requirements and boundaries but also the priority of the components, timelines and so on of a project.
As a software developer, you should not attempt to influence this process. However, you should be available to answer any questions and provide technical clarification about what is or is not possible so as to help the stakeholders to reach a consensus.
Missing deadlines and the ramifications of late delivery
Missing deadlines not only impacts the client but can also affect the team members by increasing the amount of stress they experience when they realise that the project is falling behind. This is particularly the case because missed deadlines tend to create a domino effect where one late project reduces the time available for the next project. This applies additional pressure and may ultimately cause instances of team member sickness.
The outcome could be staff cutting corners, producing a poor development outcome which damages the organisation’s reputation. This could affect revenue going forward – not only the remuneration expected at the end of the project but also by reducing the organisation’s chance of securing further work if the failure is made public.
An equally significant potential ramification comes in the form of financial penalties, which may eliminate profit or even result in the organisation delivering the project at a loss. There are certainly documented cases of organisations being destroyed by penalties they have been unable to pay, leading to their demise.
Monitoring the development project is essential to ensure that it stays on track. The activity schedule should be revisited regularly to check milestones are being met, adjusting timelines if required and keeping the client informed if relevant. This is why it is essential to build recovery points into the plan to ensure that there is wiggle room if needed.
Tips for meeting software development deadlines
Other scheduling issues
In addition to the key issues already mentioned, projects can be impacted by many other less predictable issues, such as:
Not monitoring these issues, or simply identifying them too late, can lead to failure.
Regular team meetings will help with the monitoring process, although the frequency of meetings will be dependent on the level and complexity of the project at any particular point in its activities. When meetings are held, you have the option of involving stakeholders, particularly if some ad hoc decisions need to be taken along the way which require their involvement. This is a more attractive option than sending and receiving endless emails.
The importance of the client journey
It is essential to keep the client informed throughout the development process. Why? Because apart from the fact that they are paying the development costs and it’s great customer service to keep them informed, managing the client’s expectations is crucial.
A good manager of a software development project manages the client’s expectations actively rather than reactively. Anticipating problems helps in preparing appropriate responses and planning how to communicate them.
Keeping the client informed throughout the development process reduces the chance of having to be the bearer of unpleasant (‘we can’t solve the problem’) or unexpected (‘we can solve the problem but it’s going to take much longer’) news.
Things to consider as part of your project delivery include the following:
The importance of including all stakeholders in the review
Make certain that representatives from all stakeholder groups have been included in the review phase. This helps to reinforce stakeholders’ view that they have been fully included in the process and may well be instrumental in securing future business.
KEY DESIGN METHODOLOGIES AND LIMITATIONS
A methodology is a set of predefined rules, methods or approaches that can be followed to deliver a desired outcome. Think of it as a recipe. All of the ingredients are gathered together and then the steps are followed to produce the final dish.
Some organisations select and apply a methodology in its entirety. Others choose to loosely follow a methodology, selecting only the most relevant rules and processes to suit the project in hand. In addition, you may well find an organisation has a house methodology which draws on a range of other considerations (e.g. the personal preferences of key staff in a project team).
It would be unusual for a new employee in a development context to ask which methodology was being used as this can look somewhat naive. It might therefore be helpful to know some of the key features and uses of some of them so that you can recognise them when you encounter them.
Methodologies are usually categorised as sequential, incremental or iterative, in relation to their phases and how they are conducted. Additionally, the Agile methodology combines elements of iterative and incremental methodologies.
Bear in mind that, in principle, the SDLC (see above) provides a framework which is recognised as the basis for most of the current methodologies, but the amounts of emphasis on the different components may not be the same, and in some methodologies some parts are largely informal or missing altogether.
Sequential methodologies
Sequential methodologies require each phase of the development to have been fully completed (reviewed and essentially signed off) before the next phase begins. The best known (and most commonly used) sequential model is the Waterfall methodology. Just as a waterfall cannot flow backwards, each of the phases in the Waterfall model must be completed before the next phase begins (see Figure 4.2).
Figure 4.2 The Waterfall model
Benefits of the Waterfall methodology include the following:
Drawbacks of the Waterfall methodology include the following:
Incremental methodologies
Incremental methodologies support development projects where the requirements of the new system can be broken down into a number of modules where each will become a part of the final working system (see Figure 4.3).
Once each module is complete, it can be released to users.
Figure 4.3 The incremental model
Benefits of the incremental methodology include the following:
Drawbacks of the incremental methodology include the following:
Iterative methodologies
Iterative methodologies are based on the quick development of a basic working model: a minimum viable product (MVP) that does the job but that would benefit from refinements either through added functionality or through the incorporation of emerging technologies which provide new possibilities (see Figure 4.4).
Figure 4.4 How to implement the iterative methodology (Source: Kniberg 2016)
This methodology is ideal where a project may require a significant number of refinements during the development process. This is likely to be because the general direction of travel is known but the specifics of the final solution are less well known.
Benefits of the iterative methodology include the following:
Drawbacks of the iterative methodology include the following:
Agile
Agile, and its derivative Scrum, are classified as both iterative and incremental. These are the approaches that are most commonly used in the IT industry. To be involved in this type of development, it is essential to understand the terminology used. The key elements are as follows:
The Agile Manifesto
Written in 2001 by 17 software developers, the Agile Manifesto is centred around four key values:
Individuals and interactions over processes and tools.
Working software over comprehensive documentation.
Customer collaboration over contract negotiation.
Responding to change over following a plan.
(Agile Manifesto Authors 2001)
For each statement, the item on the left has more intrinsic value than that on the right. For further information on Agile, refer to Agile Foundations (Measey et al. 2015).
HOW IS A DEVELOPMENT METHODOLOGY CHOSEN?
Deciding on the right methodology to use in a development project depends on the type of development being undertaken. Take the example of a new stock system for an online retailer. You would have to decide which model is likely to be the most suitable and beneficial when applied to the development. Would Scrum be a suitable model? Should a sequential model be used instead?
You may find it useful to employ a series of established criteria to help you make the decision. Examples of potential selection criteria are detailed in Table 4.2.
When it comes to choosing the methodology to support a development project, there is not always a clear winner – just some obvious wrong answers. As suggested earlier in this chapter, it can also come down to the personal preferences of the manager (what they know) or other key stakeholders (what they have experienced), or even just the preferred in-house approach (what they do).
Table 4.2 Selection criteria for methodologies
Selection considerations | Description |
Required outcome | Some models are more suited to a particular type of development than others. The client brief (see next section) is therefore a key factor in choosing a methodology. For example, the development of a high-risk system (large, complex, potentially liable to fail) should be well analysed, planned and documented. Therefore, the most obvious model in this situation would probably be Waterfall. |
Timescale | Development projects which need to be completed quickly will not benefit from a structured model which places emphasis on investigation, design and documentation. |
Risk | The development of a data-driven system, such as a stock control system, is likely to be relatively low risk so from this perspective most models would be suitable. In comparison, a development project that involved creating software to automate movement (such as self-driving cars or flight or rail control systems, where errors could result in death or destruction) would benefit from a highly documented, structured methodology to ensure that appropriate analysis and design have been undertaken, all risk factors have been analysed and mitigated, and a number of options have been considered before any coding activity begins. |
Accessibility for client and/or users | Methodologies that require a high degree of user interaction may be less suitable for some types of development. A stock system is likely to have future users with a wide range of IT skills. It may therefore be beneficial for a selection of users with varying skills to be involved throughout development, particularly during the design and development stages, to ensure that the system that is produced is accessible to all. |
Stability of requirements | It can be argued that a requirement for some tweaks will almost always emerge during a development project, but developments where innovation is preferred do not lend themselves to models that are sequential. |
Constraints | A requirement for structure and documentation affects the speed with which a development project can occur. Another example constraint is a lack of experience of a particular methodology (such as Scrum). Constraints are covered in more detail later in this chapter in the section ‘System definition’. |
HOW THE CLIENT BRIEF AFFECTS THE DEVELOPMENT PROCESS
Each project has unique features, sculpted by the needs of the client and the technical challenges these needs present. It is important to know the required outcome, the timescales involved, the potential risks to the project, the accessibility needs of clients and users, the stability of the requirements and any constraints (see Table 4.2) before you can even begin to understand the problem itself. You can then begin to ask, ‘What type of problem are we facing and which skills will be needed to solve it effectively?'
Assume from this point that almost everything that follows applies whether your client is internal or external. This is because an internal client should be treated in much the same way as an external client.
All software development projects are triggered by one or more of the following, which give the development a context:
In terms of development, the solution or solutions could fall into any one of the following categories or could even need a combination of the solutions detailed below:
Let’s explore that last one in a little more detail.
There are many types of hosted service, all of which have software products at their core. At the top level, these hosted services fall into one of four categories, as shown in Figure 4.5.
Figure 4.5 Service models
Software as a service (SaaS)
SaaS is where an application is distributed and licensed for use over the internet. This means that the software is not downloaded or installed onto the user’s machine and both the software and the files created on it can be accessed and used from any computer, providing there is an internet connection.
SaaS provides hosting for applications and development tools and provides a level of database management and business analytics. Examples of SaaS include services such as Dropbox, GoToMeeting, Google Apps, Google Docs and OneDrive. These are useful services for small organisations with limited resources. They also provide opportunities for sharing content during short-term collaborations, particularly where applications may need to be accessed using both web and mobile interfaces.
Platform as a service (PaaS)
Built on virtualisation technology, PaaS provides customers with the services that enable them to develop and run their own software without needing to buy and maintain the usual infrastructure associated with these activities. The customer provides the applications and the data while all of the other services are provided by the third party (including servers, storage, networking, run-time libraries and even the operating system). Examples of PaaS include Google App Engine, Apache Stratos and Windows Azure.
Infrastructure as a service (IaaS)
IaaS allows customers to provide almost all of the software components of the system, with the third party providing the hardware, networking, servers and storage. Examples of IaaS include Google Compute Engine, Amazon Web Services and Rackspace.
Amazon and Google both offer extensive cloud computing services that provide an increasingly wide range of services and technologies to customers, including:
System definition
Once the client brief has been understood and the reasons for the software development project have been identified, a system specification is generated which defines key aspects of the development. Typical activities include:
The importance of an audit trail
Documenting the key parts of a software development project is essential. This will:
Matching client needs with team skills: a team leader’s responsibility
It is highly unlikely that an entire development team will be involved in the discussions with the client. In fact, it is possible that some of the team members will never meet the client at all.
The team leader will have had significant contact with the client in the early stages and will have been responsible for producing the system specification. This will have been agreed and signed off by the client as a true and accurate reflection of the requirements of the development. This document is used to focus the activity and is frequently employed as a reference point to ensure that the development project is proceeding in the right direction.
It is important to ensure that the right skills are there when needed. To do this, it is essential for the team leader to know what skills are available to them and what to do if any skills are lacking (e.g. buy in additional expertise or ask team members to undertake training or mentoring, which grow the organisation’s future talent).
Very few organisations have the skills they need at their disposal all of the time. If they did, then there would be no need for training or continuing professional development programmes (in which staff are encouraged or required to improve their existing skills or develop new ones).
Understanding development requirements and best practice
Once the client brief has been crafted as a requirements specification, the nuts and bolts required to support the development should be put in place. When it comes to software development, there are several key requirements:
SUMMARY
In this chapter we have examined various software development methodologies and situations that can arise during the SDLC (both good and bad). We have considered the client brief and how this is fashioned into a requirements specification which will form the basis for the development.
In the next chapter we will examine the various commercial software languages available.