This chapter provides an overview of the activities that are essential to the complex role of software developer in analysing and planning a software development.

ANALYSING THE PROBLEM

It’s an undeniable fact: you can’t really solve a problem until you fully understand it.

The process of analysing a problem largely depends on whether there’s an existing system that is being replaced. If it exists, the system, whether manual or automated/electronic, will usually have entered a period of decay – the point at which it no longer works optimally (e.g. it can’t cope with demand) – or it will no longer meet business requirements (it can’t handle new opportunities).

Start by analysing the current system, asking a few key questions:

• What does the system currently do?
• What functionality needs to be improved?
• What functionality needs to be added?

Clearly the new development should retain any parts of the current system that are tried and tested and that are working well. Parts of the system which are found to be defective or insufficient need to be either rectified or replaced. By answering these questions, you will identify what the system does now and what it needs to do. The difference between the two represents the amount of development work required.

Of course, there may be no system already in place. In this case, there is no template to work from. While this can be liberating from a design perspective, you will need to solve all the problems from scratch.

The next part of the analysis focuses on the functionality of the new system. What does the new system need to do that the current system, if there is one, does not? If there is no existing system, what does it need to do? What problem are you trying to resolve?

As a software developer, you have a range of tools at your disposal to help you to investigate.

Identifying sources of information

When it comes to investigating systems, there are four key techniques that you can apply to obtain information. Not all of them are appropriate in every situation.

• Interviews with individuals and/or groups: one of the most useful techniques, interviews provide a real opportunity to explore the current system with the help of the individuals who use it. Interviews allow you to acquire information, clarify your understanding of facts, explore ideas and gather the opinions of the user base. This in itself can be very motivating for the end users, who will feel included in the development. Interviews can be either structured or unstructured. A structured interview is based on a series of questions that have been agreed in advance and each interview is conducted using this predefined list. Unstructured interviews, however, are much more fluid and so are helpful because the discussion can naturally produce useful information that you may not have thought to ask for.
• Observations of working practices: this technique helps to define the processes that users undertake – for example, establishing how documents flow through the system and how users interact with them.
• Examining existing documents: this is an often-overlooked technique, but most organisations will have a range of documentation that you can access. You should also consider examining organisational policies as this will provide information around security protocols, business continuity and backup procedures, all of which may influence aspects of the development project. Care should always be taken to ensure that any documentation being used for this purpose is up to date and reflects current practices.
• Questionnaires: this is another useful technique which benefits from the fact that it can be carefully prepared in advance to ensure that the right questions are asked to capture relevant responses with an appropriate level of detail. For example, you can decide whether to ask ‘Do you use this feature regularly? Yes or no?’ or whether you need more information, such as how often a feature is used, because that will impact on the volume of use that the feature will need to be able to support.

Choosing the right analysis techniques

When to use which technique is a key concern. Let’s consider this further.

Interviews with individuals and/or groups

The key benefit to using the structured interview technique is that the responses are easy to analyse because all respondents are asked the same questions. The disadvantage, however, is that you may miss essential facts because you did not think to ask questions about a particular aspect.

The unstructured interview allows much more opportunity to explore concepts and gather information that as an analyst you may not have thought to ask for. But, conversely, you might not get answers to questions you really needed to ask but did not think to because there was no structure to the interview. In addition, the information may be quite disparate so it can be very difficult to analyse.

The reality is that most interviews will use a combination of structured and unstructured techniques, employing predefined questions to continually refocus the discussion but not limiting the discussion.

While interviews are clearly a valuable technique, they can be time-consuming to carry out and it is likely that you will not have time to undertake many. For instance, in the context of the stock system example mentioned in Chapter 5, let’s imagine that the system will be used by 1,000 employees across 50 retail outlets. Although the analyst might well set up some interviews with representatives of these groups, they would be unlikely to be able to interview all 1,000 proposed users of the existing (and new) system.

Observations of working practices

The key benefits of observing working practices are that:

• You may observe processes being carried out that users might not have thought to include in a description of their working activities.
• You can see the processes in action in real time.

Observing working practices has downsides, however. A user could speed up their activities because they feel they could be judged not to be doing enough. The opposite is also possible, which is that they slow down their activities dramatically for fear that they may be given more to do. They may also do something extra when they are being observed that would not normally be part of their activity. Therefore, as an analyst you may not be confident that the timings of activities are accurate. In addition, some users may become anxious before or during an observation and in some cases the reaction could be severe. For example, they could refuse to be observed or react negatively to the observation process fearing impact on their job security. This is often connected to fear of change.

For these reasons you would be well advised to use observation only where the investigation would benefit from observing a particular process or from observing the same process carried out by a number of individuals so that average processing times can be calculated.

Examining existing documents

Examining documents such as purchase orders, sales invoices, HR records and production records will provide the most useful information about the types of data that the system will need to store; however, sensitive data may be excluded from your analysis due to privacy concerns. In general, though, examining such documents will enable you to estimate the quantities of data that will need to be handled, the frequency of transactions and the number of processes that will be involved.

Questionnaires

A carefully designed questionnaire is one of the quickest and easiest ways of capturing both simple and complex information from one or even many respondents, which is why questionnaires are heavily used in the investigative phase of software development projects. If the questions are well written, they will enable developers to capture both facts and user opinion.

However, as an investigative technique in the context of software development, questionnaires also have disadvantages. For example, many of us dislike questionnaires simply because we find we are faced with forms to complete on a regular basis (financial forms, job applications, medical questionnaires, insurance documents, security questionnaires, passport applications etc.).

Additionally, badly written questions may result in responses which inappropriately skew the results, leading to poor development decisions being made. For example, it is possible for questions to be written in a way that enables analysts to generate evidence to support their own theories. Consider the following questions. Which is likely to give the most useful (and genuine) response?

1. What are the problems with the way that invoices are currently processed?
2. What is good about the way that invoices are currently processed?
3. How are invoices currently processed?

A and B are leading questions because they direct the respondent to consider the issue from a particular perspective (‘problem’ and ‘good’). The most unbiased question is C, and this is the most likely to generate useful information.

It is generally agreed that, when questionnaires are used in an analysis or investigation, the following are important:

• Be clear what is it you want to find out.
• Choose your respondents carefully – ensure that you ask the right people.
• Choose your questions carefully – for example, do you want to gather facts or opinion?

Next, decide on the type(s) of questions you need to use:

• Open questions, such as ‘How are our invoices currently processed?’, allow respondents to produce an extended response, but there is a danger that the respondent will miss the point of the question altogether and answer a question that is not even being asked. In addition, the responses may well be difficult to analyse because they can be so varied.
• Closed questions are questions where the respondent chooses from a predefined list of possible answers. This produces responses which are much easier to analyse because you simply have to count the number of responses that chose a particular answer to produce a measurable result. However, the danger here is that respondents may well have had yet another answer which you did not consider.

You can also make use of the funnelling technique, where a series of increasingly penetrative questions are used to gather finer points and detail. For example:

• Question 1: Can you tell me about customer complaints? (first question, open)
• Question 2: Which type of complaint is most common?
• Question 3: Can you give me an example of this type of complaint?
• Question 4: Was this successfully resolved? (last question, closed)

Funnelling can also be employed the other way around, starting with quite restrictive, closed questions and then broadening out into more open questioning. This often occurs during in-person interviews when an interviewee is initially quite guarded and reluctant to give detailed answers; over time they relax and are happier to handle more open questions.

Once you have decided on your approach, write the questions and decide their order. Think about the flow of the questions to help respondents move through the questionnaire as easily as possible. Think about it in the same way as you would consider tab order on an electronic form, moving logically from the top to the bottom of the form with each keypress, rather than having to jump around as the tab key moves you erratically up and down the form.

Overall, questionnaires are a very useful and widely employed investigative tool. However, their key flaws are:

• The return rate may be low as respondents who are not particularly interested may well ignore the questionnaire.
• Respondents may lie (particularly if the honest response could cast the respondent in a negative light).
• You may have to deal with irrelevant information that respondents have chosen to include.
• Respondents may not fully answer the question or may simply refuse to answer.

Putting the results of your analysis together

Once you have analysed the data gathered from the investigation, it will focus the development project, enabling the development team to identify the functional requirements of a new (or adapted) system.

You should be aware that if there is no existing documentation, or there are no users to interview or question, identifying the functional requirements can be challenging. Having said that, this situation is usually only the case for a business start-up. Any organisation which is already operating, even if it does not use a computerised solution, will still have documentation to explore and users to talk to.

Establishing requirements

The next stage is to produce a document which sets out the context of the new system and what it will be designed to achieve. This document should list the key components and processes of the existing system which will need to be incorporated into the new system. Key components can include the following.

Purpose

As suggested earlier, at the top level, the purpose of a development project will be one or more of the following:

• resolve a problem or problems with an existing system;
• respond to the changing needs of the organisation;
• respond to business drivers such as reducing costs or improving efficiency;
• take advantage of the availability of new technologies;
• respond to the activities of competitors;
• respond to the identification of emerging markets.

The purpose of the project should be written down in sufficient detail to guide the new system’s designers in relation to the functional needs of the new system. The documentation should consider:

• availability of data;
• types of transaction needed;
• legal and regulatory requirements;
• administration (such as user access);
• audit tracking;
• reporting needs.

In every software development project there are also a number of non-functional requirements. These set out how the system should work or behave and may often include:

• security;
• maintenance of data integrity;
• interoperability (the compatibility of the new system with other existing components);
• scalability;
• performance (issues such as response times, volumes of data to be managed);
• reliability.

The benefit of establishing requirements at the outset is that once these are known, the development team will be able to use them to measure the success of the development project by establishing how well each of these requirements was (or was not) met.

Users

As most systems involve a level of user interaction, they should be developed to accommodate the needs of a wide range of end users. Preparing to meet user needs is something that should be thoroughly explored during the analysis phase of the development project, as understanding the user experience will ultimately generate user satisfaction. A number of factors should be considered, including aesthetics, performance, ergonomics, accessibility, usability, ease of navigation and even how enjoyable the user considers the experience to be.

Even the quality of user prompts will influence the user’s judgements about the system. For example, consider the following prompt:

Input Numbur:

This simple prompt has a significant issue: the misspelling of ‘number’ could be interpreted by users as sloppiness and a lack of care on the part of the development team – someone should have proofread the user interface, and the error doesn’t instil any confidence. Secondly and more importantly, even if it had been spelled correctly, this particular prompt gives the user no indication of what is expected. What sort of number? A whole number? Can it have a decimal part? A number in what range? This could have been improved by simply including more information:

Input a whole number between 1 and 20:

Systems programmed using visual techniques allow developers to make use of components which reduce the amount of user input, such as combo boxes and dropdown lists. This immediately reduces the potential for problems with data integrity because the user is not able to input erroneous data.

Where user input is needed, validation plays a key role in ensuring that users interact correctly with the system by rejecting inappropriate or incorrect inputs and providing guidance on what is expected. In addition, any developments which focus on external interactions with users, whether established businesses or start-ups, must present a good impression to users because failure to do so could mean that customers may not return to the site. Equally, a good impression could drive the business forward.

Inputs, processes and outputs

When it comes to the data component of a system, some development teams almost work in reverse by considering what the system will need to output so that the inputs and processes needed to produce those outputs can be identified. They may also consider the reports that the software will need to generate. For example:

• sales (daily, weekly, monthly, annual);
• stock (fast- or slow-moving stock, stock levels, reorder data);
• financial (income, expenditure, borrowing);
• personnel (hours worked, payroll breakdown, sickness statistics, holiday usage);
• recruitment (advertisements placed, responses).

The types of input and output really depend on the purpose of the software development project. Is it to create an electronic version of something that already exists as a paper-based system (such as an electronic rostering system for hospital staff)? If so, both the inputs and the outputs will be immediately identifiable. Alternatively, if the development project is expected to produce a completely new product, the inputs, outputs and processes may be less well known and may need to be explored.

The inputs to a system may require user interaction or may draw on historical data owned by the organisation. Alternatively, the data may need to be acquired from a third party.

Processing will include simple calculations such as invoice totals and VAT calculations, and will also include the need for data to be read from a file or written back to a file (e.g. before an item can be sold, the system may need to read the amount of stock and then reduce the stock value by the amount sold). However, processing may also require complex analytics, trend analysis, usage statistics and so on.

As a developer, you need clarity around inputs, processes and outputs as these are the heart of any system.

ANTICIPATING POSSIBLE ERRORS AND ISSUES, AND MITIGATING THEM

From the perspective of managing a development, there are a number of possible issues which could emerge. Here are a few of the more common examples.

Deadlines

In Chapter 11 we suggest that you should not ‘over-promise’ and ‘under-deliver’. Doing so is guaranteed to create problems with your client and potentially your users, and it will often occur because unrealistic deadlines were promised at the outset. With experience, you will become better able to estimate the activities that will be needed in each phase and how long each activity will take to complete, but even so it is always a good idea to build an element of redundancy into your plan to ensure that there will be time to manage any unexpected events.

Scope creep

The requirements of the new system will have been defined and agreed at the start of the development, but it is quite common for clients to ask for changes to what was agreed, or they might even request additional features or functionality be included. This is known as scope creep (as already touched upon in Chapter 4).

Avoiding scope creep can be a challenge, particularly if you want to keep your client happy. But responding positively to these requests can drive up the costs of the development due to the time taken to accommodate them. For this reason, smart development teams will often estimate the cost of implementing the request and will raise an invoice for that value which they will share with the client. In reality they may or may not choose to raise an invoice, but producing the invoice will show the client the financial impact of such requests and will make them think more carefully, particularly when they understand that accommodating such changes could mean having to pay more for the development project.

Having said all this, some of the methodologies discussed in Chapter 4, particularly Agile, are based on the notion of the product emerging over time and therefore scope creep is part of the process. As a developer, you will need to monitor these requests and ensure they do not become excessive, entail significant additional cost for the client, or morph the development into something completely different from what was intended.

When it comes to scope creep, more rigid methodologies, such as Waterfall (see Chapter 4), can help you to maintain more control.

Poor team communication

It is now very common for development teams to increase their coding capacity by subcontracting aspects of development to third parties, often overseas. Part of the reason for this is to access a wider talent pool while keeping costs low, because the average hourly rate of programmers in countries such as India and Poland is far lower than that in countries such as the UK.

This can result in communication issues, particularly in terms of ensuring that the subcontractors fully understand what it is you are asking them to do. It is therefore essential to closely monitor the activities of the team to ensure that the development project is progressing as expected. This should be done regularly using a variety of methods, such as code reviews, daily stand-ups, stakeholder briefings and project management platforms, so that any issues can be quickly uncovered and dealt with.

Poor team collaboration

Poor collaboration can result from a lack of, or poor, communication. In addition, your development team could suffer from a variety of issues which could affect their successful performance. For example:

• The size of a team can affect how well its members work together because with a large team you simply have a larger number of personalities.
• Larger teams may need to be divided into sub-teams. This allows each sub-team to be assigned specific tasks and their concerns, input, questions and so on to be communicated via a sub-team leader, effectively streamlining discussions and keeping development meetings suitably focused.
• The team members may not all be prepared to share information.
• Pre-existing conflict can resurface, particularly during times when development activity is stressful. This can be a particular issue if you have domineering personalities in the group.
• A lack of clarity about how the team will be rewarded for their activities can result in a sense of unfairness.
• A lack of focus can result if team members are not on board with the overall objectives and ultimate goal.
• When team members are based in different countries, there can be both physical and cultural barriers which need to be carefully managed.
• There may not be a feeling of mutual trust across the whole team.

Potential ways of promoting collaboration include:

• Promoting a positive attitude among team members is essential to effectively managing a team.
• To avoid individual team member disengagement, all team members should have their ideas heard.
• Promote a positive team culture and avoid the blame game. If a member of the team has problems, the team should pull together to help resolve the situation rather than simply blaming the individual personally for their failure.

Access to staff

One of the most irritating problems when you have put together a development team is when one or more members become unavailable. Holidays, for example, can be factored into the planning, but sickness can usually not be predicted.

There can also be situations where team members are pulled off a project to be assigned to other activities. This in particular destabilises the team when it creates a skills shortage. Finding a replacement can be difficult and, even if you do, you then have the additional problem of bringing the new team member up to speed.

See more on this topic in Chapter 4.

Lack of staff understanding of their own role and responsibilities

Whether the entire team is involved in the decision-making and planning processes or not, there should be clear demarcation of roles and responsibilities. Staff should be supported to ensure that they understand their roles within the team and within the context of the development project. If this does not happen, team members could disengage with the process and put the entire project at risk.

Not being aware of an impending problem

Even if you have taken steps to mitigate possible problems, there are not many development projects that do not suffer from some sort of issue from time to time. To give yourself the best chance of identifying emerging issues and addressing them, make sure to carefully observe team activities and listen carefully to the concerns of both the team and the client or users.

Ideally a development should aim to ‘design out’ any potential operational problems that could occur later. This means proactively reworking ideas so problems don’t occur and therefore do not need to be solved. For example, changing a user input from free text to a selection box of available choices removes many validation-oriented issues.

Designing systems that will be robust is usually achievable if you fully understand your data and the needs of your client and users. But there are other principles that should be considered too, such as scalability. It is very easy to make a product which meets the immediate needs of the client. However, the client’s needs may outgrow the original design during the development period. This could be because the system has not been designed to handle a larger number of future transaction or data volumes, cannot cope with an increased number of users, does not have sufficient storage availability, or generally is unable to handle an increased workload as time passes.

THE ROLE OF DOCUMENTATION

During planning, consideration is given to documentation that can be generated during or at the end of the development. All decisions made and the development activities undertaken are recorded to provide data that will be used in the review. This ensures that lessons can be learned.

The documentation that is typically generated includes:

• development plans and schedules;
• group activity logs with expected activity dates – the logs are then annotated with monitoring detail to record what is complete, what went well or what went badly, and information about changes that had to be made or how emerging problems were solved;
• diagrams representing both the existing system and the design of the proposed system, such as flowcharts, data flow diagrams and system diagrams;
• annotated code;
• test data;
• client feedback received throughout the development project – the plan may be that all feedback will be provided by the client via email (rather than verbal feedback), so that the email or a sequence of emails can provide a permanent record;
• minutes of development meetings (if these occurred);
• photographs of Scrum boards (or archived digital copies) – during planning, decisions will be made about whether physical Scrum boards will be photographed, or whether the Scrum boards will be digitally documented using proprietary software such as Asana, Wrike or Workamaji®, with the advantage that the Scrum boards can also be accessed on devices as the data will be held in the cloud.

Some of the documentation listed above will ultimately be included in the technical guide to support the maintenance of the product. These documents will provide a focus for discussion and enable team members to learn lessons from the development activity.

SUMMARY

In this chapter we have reviewed the essential activities in relation to analysis and planning with which a software developer might engage as part of their role. Perhaps the most important point to remember is that it doesn’t matter how good you are at creating software if you don’t truly understand the underlying problem you are trying to solve.

In the next chapter, we will explore how to write good-quality code.