Reporting Success

Part 1 of this book stated a universal design rule: Features are always and everywhere aspects of integration, not implementation. As such, there are no features in any of the early services. At some point you will have integrated enough to start seeing features. I call that point the system. The system is unlikely to appear at the very end of the project since there may be some additional concluding activities such as system testing and deployment. The system typically appears toward the end because it requires most of the services as well as the clients. When using The Method, this means only once you have integrated inside the Managers, the Engines, the ResourceAccess, and the Utilities can you support the behaviors that the Clients require.

While the system is the product of the integration, not all the integration happens inside the Managers. Some integration happens before the Managers are complete (such as the Engines integrating ResourceAccess) and some integration happens after the Managers (such as between the Clients and the Managers). There might also be explicit integration activities, such as developing a client of a service against a simulator and then integrating the client with the real service.

The problem with the system appearing only toward the end of the project is pushback from management. Most people tasked with managing software development do not understand the design concepts in this book and simply want features. They would never stop to think that if a feature can appear early and quickly, then it does not add much value for the business or the customers because the company or the team did not spend much effort on the feature. Usually, management uses features as the metric to gauge progress and success, and tends to cancel sick projects that do not show progress. As such, the project faces a serious risk: It could be perfectly on schedule but because the system only appears at the end, if the project bases its progress report on features, it is asking to be canceled. The solution is simple:

Never base progress reports on features. Always base progress reports on integration.

A Method-based project performs a lot of integration along the project. These integrations are small and doable. As a result, there is the potential for a constant stream of good news coming out of the project, building trust and avoiding cancellation.

Architect, Not Architects

The architect will need to spend time gathering and analyzing the requirements, identifying the core use cases and the areas of volatility, and producing the system and the project design. While the design itself is not time-consuming (the architect can usually design both the system and the project in a week or two), it may take several months to get to the point that the architect can design the system and the project.

Most managers will recoil both at spending some three or four months on design and at skipping design entirely. They may wish to accelerate the design effort by having more architects participate. However, requirements analysis and architecture are contemplative, time-consuming activities. Assigning more architects to these activities does not expedite them at all, but instead will make matters worse. Architects are typically senior self-confident personnel, used to working independently. Assigning multiple architects only results in them contesting with each other, rather than in system and project design blueprints.

One way of resolving the multiple architects conflict is to appoint a design committee. Unfortunately, the surest way of killing anything is to appoint a committee to oversee it. Another option is to carve up the system and assign each architect a specific area to design. With this option, the system is likely to end as a Chimera—a mythological Greek beast that has the head of a lion, the wings of a dragon, the front legs of an ox, and the hind legs of a goat. While each part of the Chimera is well designed and even highly optimized, the Chimera is inferior at anything it attempts: It does not fly as well as a dragon, run as fast as a lion, pull as much as an ox, or climb as well as a goat. The Chimera lacks design integrity—and the same is true when multiple architects design the system, each responsible for their part.

A single architect is absolutely crucial for design integrity. You can extend this observation to the general rule that the only way to allow for design integrity is to have a single architect own the design. The opposite is also true: If no single person owns the design and can visualize it cover-to-cover, the system will not have design integrity.

Additionally, with multiple architects no one owns the in-betweens, the cross-subsystem or even cross-services design aspects. As a result, no one is accountable for the system design as a whole. When no one is accountable for something, it never gets done, or at best is done poorly.

With a single architect in charge, that architect is accountable for the system design. Ultimately, being accountable is the only way to earn the respect and trust of management. Respect always emerges out of accountability. When no one is accountable, as is the case with a group of architects, management intuitively has often nothing but scorn for the architects and their design effort.

The Core Team

As vital as the architect is to the project, the architect cannot work in isolation. On day 1, the project must have a core team in place. The core team consists of three roles: project manager, product manager, and architect. These are logical roles and may or may not map to three individuals. When they do not, you may see the same person as both the architect and the project manager, or a project with several product managers.

Most organizations and teams have these roles, but the job titles they use may be different. I define these roles as follows:

• The project manager. The job of the project manager is to shield the team from the organization. Most organizations, even small ones, create too much noise. If that noise makes its way into the development team, it can paralyze the team. A good project manager is like a firewall, blocking the noise, allowing only sanctioned communication through. The project manager tracks progress and reports status to management and other project managers, negotiates terms, and deals with cross-organization constraints. Internally, the project manager assigns work items to developers, schedules activities, and keeps the project on schedule, on budget, and on quality. No one in the organization other than the project manager should assign work activity or ask for status from developers.

• The product manager. The product manager should encapsulate the customers. Customers are also a constant source of noise. The product manager acts as a proxy for the customers. For example, when the architect needs to clarify the required behaviors, the architect should not chase customers; instead, the product manager should provide the answers. The product manager also resolves conflicts between customers (often expressed as mutually exclusive requirements), negotiates requirements, defines priorities, and communicates expectations about what is feasible and on what terms.

• The architect. The architect is the technical manager, acting as the design lead, the process lead, and the technical lead of the project. The architect not only designs the system, but also sees it through development. The architect needs to work with the product manager to produce the system design and with the project manager to produce the project design. While the collaboration with both the product manager and the project manager is essential, the architect is held responsible for both of these design efforts. As a process lead, the architect has to ensure the team builds the system incrementally, following the system and the project design with a relentless commitment to quality. As a technical lead, the architect often has to decide on the best way of accomplishing technical tasks (the what-to-do) while leaving the details (the how-to-do) to developers. This requires continuous hands-on mentoring, training, and reviews.

Perhaps the most glaring omission from this definition of the core team are developers. Developers (and testers) are transient resources that come and go across projects—a very important point that this chapter revisits as part of the discussion of scheduling activities and resource assignment.

Unlike developers, the core team stays throughout the project since the project needs all three roles from beginning to end. However, what these roles do in the project changes over time. For example, the project manager shifts from negotiating with stakeholders to providing status reports, and the product manager shifts from gathering requirements to performing demos. The architect shifts from designing the system and the project to providing ongoing technical and process leadership, such as conducting design and code reviews at the service level and resolving technical conflicts.

Plans, Not Plan

The result of project design is a set of plans, not a single plan. As described in the previous chapter, the project plan is not a single coordinate of time and cost. There are always multiple possible ways of building any system, and only one option will offer the right combination of time, cost, and risk. The architect may be tempted to simply ask management what the design parameters of the project are and just design that single option. The problem is that managers often do not say what they mean or mean what they say.

For example, consider a 10-man-year project—that is, a project where the sum of effort across all activities is 10 man-years. Suppose management asks for the least costly way of building the system. Such a project would have one person working for 10 years, but management is unlikely to be willing to wait 10 years. Now suppose that management asks for the quickest possible way to build the system. Imagine it is possible to build the same system by engaging 3650 people for 1 day (or even 365 people for 10 days). Management is unlikely to hire so many people for such short durations. Similarly, management will never ask for the safest way of building the system (because anything worth doing requires risk, and safe projects are not worth doing) or knowingly go for the riskiest way of doing the project.

Software Development Plan Review

The only way to resolve the ambiguity about what management really wants is to present a buffet of good options from which to choose, with each option being a viable combination of time, cost, and risk. You present these options to management in a dedicated meeting unofficially called the Feed Me/Kill Me meeting. As the name implies, the purpose of this meeting is for management to choose one of the project design options and commit the required resources (the “Feed Me” route). One of the options is always that of not doing the project (the “Kill Me” route). Officially, the name of the meeting should be the Software Development Plan Review, or SDP review. It makes no difference if your process does not have an SDP review point: Just call a meeting (no manager can refuse a meeting request whose subject line is “Software Development Plan Review”).

Once the desired option is identified, management must literally sign off on the SDP document. This document now becomes your project’s life insurance policy because, as long as you do not deviate from the plan’s parameters, there is no reason to cancel your project. This does require proper tracking (as described in Appendix A) and project management.

If no option is palatable, then you need to drive the right decision—in this case, killing the project. A doomed project, a project that from inception did not receive adequate time and resources, will do no one any good. The project will eventually run out of time or money or both, and the organization will have wasted not just the funds and time but the opportunity cost of devoting these resources to another doable project. It is also detrimental to the careers of the core team members to be on a project that never has a chance. Since you have only a few years to make your mark and move ahead, every project must count and be a feather in your cap. Spending a year or two on a sideways move that failed will limit your career prospects. Killing such a project before development starts is beneficial for all involved.

Design and Team Efficiency

When using 1:1 assignments of services to developers, it follows that the interaction between the services is isomorphic to the interaction between the developers. Consider Figure 7-1.

The relationship between the services, their interactions and communication, dictates the relationships and interactions between the developers. When using 1:1 assignment, the design of the system is the design of the team.

Next, consider Figure 7-2. While the number of services and their size has not changed from Figure 7-1, no one could claim it is a good design.

A good system design strives to reduce the number of interactions between the modules to their bare minimum—the exact opposite of what happens in Figure 7-2. A loosely coupled system design such as that in Figure 7-1 has minimized the number of interactions to the point that removing one interaction makes the system inoperable.

The design in Figure 7-2 is clearly tightly coupled, and it also describes the way the team operates. Compare the teams from Figure 7-1 and Figure 7-2. Which team would you rather join? The team in Figure 7-2 is a high-stress, fragile team. The team members are likely territorial and resist change because every change has ripple effects that disrupt their work and the work of everybody else. They spend an inordinate amount of time in meetings to resolve their issues. In contrast, the team in Figure 7-1 can address issues locally and contain them. Each team member is almost independent from the others and does not need to spend much time coordinating work. Simply put, the team in Figure 7-1 is far more efficient than the team in Figure 7-2. As a result, the team with the better system design has far better prospects of meeting an aggressive deadline.

This last observation is paramount: Most managers just pay lip service to system design because the benefits of architecture (maintainability, extensibility, and reusability) are down-the-road benefits. Future benefits do not help a manager who is facing the harsh reality of scant resources and a tight schedule. If anything, it behooves the manager to reduce the scope of work as much as possible to meet the deadline. Since system design is supposedly not helping with the current objectives, the manager will throw overboard any meaningful investment in design. Sadly, by doing so, the manager loses all chance of meeting the commitments, because the only way to meet an aggressive deadline is with a world-class design that yields the most efficient team. When striving to get management support for your design effort, show how design helps with the immediate objective. The long-term benefits will flow out of that.

Task Continuity

When assigning services (or activities such as UI development), try to maintain task continuity, a logical continuation between tasks assigned to each person. Often, such task assignments follow the service dependency graph. If service A depends on service B, then assign A to the developer of B. One advantage is that the A developer who is already familiar with B needs less ramp-up time. An important, yet often overlooked advantage of maintaining task continuity is that the project and the developer’s win criteria are aligned. The developer is motivated to do an adequate job on B to avoid suffering when it is time to do A. Perfect task continuity is hardly ever possible, but it should be the goal.

Finally, take the developers’ personal technical proclivities into account when making assignments. For example, it will likely not work well to have the security expert design the UI, to have the database expert implement the business logic, or to have junior developers implement the utilities such as message bus or diagnostics.

Classic Mistakes

As just mentioned, people tend to overestimate and underestimate in an attempt to compensate for uncertainty. Both of these are deadly when it comes to project success.

Overestimation never works because of Parkinson’s law.² For example, if you give a developer three weeks to perform a two-week activity, the developer will simply not work on it for two weeks and then be idle for a week. Instead, the developer will work on the activity for three weeks. Since the actual work consumed only two of those three weeks, in the extra week the developer will engage in gold plating—adding bells and whistles, aspects, and capabilities that no one needs or wants, and that were not part of the design. This gold plating significantly increases the complexity of the task, and the increased complexity drastically reduces the probability of success. Consequently, the developer labors for four or six weeks to finish the original task. Other developers in the project, who expect to receive the code after three weeks, are now delayed, too. Furthermore, the team now owns, perhaps for years and across multiple versions, a code module that is needlessly more complex than what it should have been in the first place.

2. Cyril N. Parkinson, “Parkinson’s Law,” The Economist (November 19, 1955).

Underestimation guarantees failure just as well. Undoubtedly, giving a developer two days to perform a two-week coding activity will preclude any gold plating. The problem is that the developer will try to do the activity quick-and-dirty, cutting corners and disregarding all known best practices. This is as sensible as asking a surgeon to operate on you quick-and-dirty or a contractor to build a house quick-and-dirty.

Sadly, there is no quick-and-dirty with any intricate task. Instead, the two options are quick-and-clean and dirty-and-slow. Because the developer is missing all the best practices in software development, from testing to detailed design to documentation, the developer is now trying to perform the task in the worst possible way. Consequently, the developer will not work on the activity for the nominal two weeks it could have taken, assuming the work was performed correctly, but will work on it for four or six (or more) weeks due to the low quality and increased complexity. As with overestimation, other developers in the project who expected the code after the scheduled two days are much delayed. Furthermore, the team now has to own, perhaps for years and across multiple versions, a code module that is done the worst possible way.

Probability of Success

While these conclusions may make common sense, what many miss is the magnitude of these classic mistakes. Figure 7-3 plots in a qualitative manner the probability of success as a function of the estimation. For example, consider a 1-year project. With proper architecture and project design, the project’s normal estimation is 1 year, indicated by point N in Figure 7-3. What would be the probability of success if you give this project a day? A week? A month? Clearly, with sufficiently aggressive estimations, the probability of success is zero. How about 6 months? While the probability of a 1-year project completing in 6 months is extremely low, it is not zero because maybe a miracle will happen. The probability of success if you estimate at 11 months and 3 weeks is actually very high, and it is also fairly high for 11 months. However, it is unlikely the project can complete in 9 months. Therefore, to the left of the normal estimation is a tipping point where the probability of success drastically improves in a nonlinear way. Similarly, this 1-year project could last 13 months, and even 14 months is reasonable. But if you give this project 18 or 24 months, you will surely kill it because Parkinson’s law will kick in: Work will expand to fill the allotted time, and the project will fail due to the increased complexity. Therefore, another tipping point exists to the right of the normal estimation, where the probability of success again collapses in a nonlinear way.

Figure 7-3 illustrates the paramount importance of good nominal estimations because they maximize the probability of success, in a nonlinear way. In the past, you were likely to hurt yourself and others when you both underestimated and overestimated. These are not just common, classic mistakes—they are cardinal mistakes.

Estimation Techniques

The poor track record with estimations in the software industry persists even though a decent set of effective estimation techniques have been available for decades and across multiple other industries. I have yet to see a team that has practiced estimations correctly and was also off the mark with their project design and commitments. Instead of trying to review all of these techniques, this section highlights some of the ideas and techniques I have found over the years to be the most simple and effective.

Overall Project Estimation

Estimating the project as a whole is useful primarily for project design validation, but can also be beneficial when initiating project design. When you finish the detailed project design, compare it to the overall project estimation. The two need not match perfectly but should be congruent and validate each other. For example, if the detailed project design was 13 months and the overall project estimation was 11 months, then the detailed project design is valid. But if the overall estimation was 18 months, then at least one of these numbers is wrong, and you must investigate the source of the discrepancy. You can also utilize the overall project estimation when dealing with a project with very few up-front constraints. Such a clean canvas project has a great deal of unknowns, making it difficult to design. You can use the overall project estimation to work backward to box in certain activities as a way of initiating the project design process.

Activity Estimations

You start the project design with the estimated duration of the individual activities in the project. Before you estimate individual activities, you must prepare a meticulous list of all activities in the project, both coding and noncoding activities alike. In a way, even that list of activities is an estimation of the actual set of activities, so the same rationale about reducing uncertainties holds true here. Avoid the temptation to focus on the structural coding activities indicated by the system architecture, and actively look below the waterline at the full extent of the iceberg. Invest time in looking for activities, and ask other people to compile that list so you could compare it with your own list. Have colleagues review, critique, and challenge your list of activities. You may be surprised by what you actually missed.

Since accuracy is superior to precision, a best practice is to always use a quantum of 5 days in any activity estimation. Activities that take 1 or 2 days should not be part of the plan. Activities that are 3 or 4 days are always estimated at 5 days. Activities are either 5, 10, 15, 20, 25, 30, or 35 days long. Activities estimated at 40 or more days may be good candidates to break down into smaller activities to reduce the uncertainty. Using 5 days for each activity aligns the project nicely on week boundaries and reduces waste of parts of weeks before or after an activity. This practice also matches real life—no activity has ever started on a Friday.

The reduction in uncertainty benefits even regular-size activities. Force yourself and others to break down each activity into tasks in addition to coding, such as learning curves, test clients, installation, integration points, peer reviews, and documentation. Again, by avoiding focusing on coding and examining the full scope of the work ahead, you greatly reduce the uncertainty of individual activity estimations.

Project Network

You can graphically arrange the activities in the project into a network diagram. The network diagram shows all activities in the project and their dependencies. You first derive the activity dependencies from the way the call chains propagate through the system. For each of the use cases you have validated, you should have a call chain or sequence diagram showing how some interaction between the system’s building blocks supports each use case. If one diagram has Client A calling Manager A and a second diagram has Client A calling Manager B, then Client A depends on both Manager A and Manager B. In this way, you systematically discover the dependencies between the components of the architecture. Figure 7-4 shows the dependency chart of the code modules in a sample Method-based architecture.

The dependency chart shown in Figure 7-4 has several problems. First, it is highly structural and is missing all the nonstructural coding and noncoding activities. Second, it is graphically bulky and with larger projects would become visually too crowded and unmanageable. Third, you should avoid grouping activities together, as is the case with the Utilities in the figure.

You should turn the diagram in Figure 7-4 into the detailed abstract chart shown in Figure 7-5. That chart now contains all activities, coding and noncoding alike, such as architecture and system testing. You may want to also add a side legend identifying the activities for easy review.

Activity Times

The effort estimation for an activity alone does not determine when that activity will complete: Dependencies on other activities also come into play. Therefore, the time to finish each activity is the product of the effort estimation for that activity plus the time it takes to get to that activity in the project network. The time to get to an activity, or the time it takes to be ready to start working on the activity, is the maximum of time of all network paths leading to that activity. In a more formal manner, you calculate the time for completing activity i in the project with this recursive formula:

Ti=Ei+Max(Ti−1,Ti−2,...,Ti−n)

where:

• Ti is the time for completing activity i.

• Ei is the effort estimation for activity i.

• n is the number of activities leading directly to activity i.

The time for each of the preceding activities is resolved the same way. Using regression, you can start with the last activity in the project and find the completion time for each activity in the network. For example, consider the activity network in Figure 7-6.

In the diagram in Figure 7-6, activity 5 is the last activity. Thus, the set of regression expressions that define the time to finish activity 5 are:

T5=E5+Max(T3,T6)T6=E6T3=E3+Max(T2,T4)T4=E4T2=E2+T1T1=E1

Note that the time to finish activity 5 depends on the effort estimation of the previous activities as much as it depends on the network topology. For example, if all the activities in Figure 7-6 are of equal duration, then:

T5=E1+E2+E3+E5

However, if all activities except activity 6 are estimated at 5 days, and activity 6 is estimated at 20 days, then:

T5=E6+E5

While you could manually calculate the activity times for small networks such as Figure 7-6, this calculation quickly gets out of hand with large networks. Computers excel at regression problems, so you should use tools (such as Microsoft Project or a spreadsheet) to calculate activity times.

The Critical Path

By calculating the activity times, you can identify the longest possible path in the network of activities. In this context, the longest path means the path with greatest duration, not necessarily the one with the greatest number of activities. For example, the project network in Figure 7-7 has 17 activities, each of different estimated duration (the numbers in Figure 7-7 are just the activity IDs; durations are not shown).

Based on the effort estimation for each activity and the dependencies, using the formula given earlier and starting from activity 17, the longest path in the network is shown in bold. That longest path in the network is called the critical path. You should highlight the critical path in your network diagrams using a different color or bold lines. Calculating the critical path is the only way to answer the question of how long it will take to build the system.

Because the critical path is the longest path in the network, it also represents the shortest possible project duration. Any delay on the critical path delays the entire project and jeopardizes your commitments.

No project can ever be accelerated beyond its critical path. Put another way, you must build the system along its critical path to build the system the quickest possible way. This is true in any project, regardless of technology, architecture, development methodology, development process, management style, and team size.

In any project with multiple activities on which multiple people are working, you will have a network of activities with a critical path. The critical path does not care if you acknowledge it or not; it is just there. Without critical path analysis, the likelihood of developers building the system along the critical path is nearly zero. Working this way is likely to be substantially slower.

Assigning Resources

During project design, the architect assigns abstract resources (such as Developer 1) to each of the project design options. Only after the decision makers have chosen a particular project design option can the project manager assign actual resources. Since any delay in the critical path will delay the project, the project manager should always assign resources to the critical path first. You should take matters a step further by always assigning your best resources to the critical path. By “best,” I mean the most reliable and trustworthy developers, the ones who will not fail to deliver. Avoid the classic mistake of first assigning developers to high-visibility but noncritical activities, or to activities that the customer or management care the most about. Assigning development resources first to noncritical activities does nothing to accelerate the project. Slowing down the critical path absolutely slows down the project.

Staffing Distribution

The required staffing for your project is not constant with time. At the beginning, you need only the core team. Once management selects a project design option and approves the project, you can add resources such as developers and testers.

Not all resources are needed all at once due to the dependencies and the critical path. Much the same way, not all resources are retired uniformly. The core team is required throughout, but developers should not be needed through the last day of the project. Ideally, you should phase in developers at the beginning of the project as more and more activities become possible, and phase out the developers toward the end of the project.

This approach of phasing in and phasing out resources has two significant advantages. First, it avoids the feast-or-famine cycles experienced by many software projects. Even if you have the required average level of staffing for the project, you could be understaffed in one part of the project and overstaffed in another part. These cycles of idleness or intense overtime are demoralizing and very inefficient. Second (and more importantly), phasing resources offers the possibility of realizing economy of scale. If you have several projects in the organization, then you could arrange them such that developers are always phasing out of one project while phasing into another. Working this way yields a hundreds of percent increase in productivity, the classic “doing much more with less.”

Staffing Distribution Chart

Figure 7-8 depicts the typical staffing distribution chart of a well-designed and properly staffed project. At the start of the project is the front end, during which the core team is working on the system and project design; this phase ends with the SDP review. If the project is terminated at that point, the staffing goes to zero and the core team is available for other projects. If the project is approved, an initial ramp-up in staffing occurs in which developers and other resources are working on the lowest-level activities in the project that enable other activities. When those activities become available, the project can absorb additional staff. At some point you have phased in all the resources the project ever needs, reaching peak staffing. For a while, the project is fully staffed. The system tends to appear at the end of this phase. Now the project can phase out resources, and those left are working on the most dependent activities. The project concludes with the level of staffing required for system testing and release.

Figure 7-9 shows a staffing distribution chart that demonstrates the behavior of Figure 7-8. You produce a chart such as Figure 7-9 by first staffing the project, then listing all the dates of interest (unique dates when activities start and end) in chronological order. You then count how many resources are required for each category of resources in each time period between dates of interest. Do not forget to include in the staffing distribution resources that do not have specific activities but are nonetheless required, such as the core team, quality control, and developers between coding activities. This sort of stacking bar diagram is trivial to do in a spreadsheet. The files accompanying this book contain several example projects and templates for these charts.

Since the dates of interest may not be regularly spaced, the bars in the staffing distribution chart may vary in time resolution. However, in most decent-size projects with enough activities, the overall shape of the chart should follow that of Figure 7-8. By examining the staffing distribution chart, you get a quick and valuable feedback on the quality of your project design.

Staffing Mistakes

Several common project staffing mistakes may be evident in the staffing distribution chart. If the chart looks rectangular, it implies constant staffing—a practice against which I have already cautioned.

A staffing distribution with a huge peak in the middle of the chart (as shown in Figure 7-10) is also a red flag: Such a peak always indicates waste.

Consider the effort expended in hiring people and training them on the domain, architecture, and technology when you use them for only a short period of time. A peak is usually caused by not consuming enough float in the project, resulting in a spike in resource demand. If the project were to trade some float for resources, the curve would be smoother. Figure 7-11 depicts a sample project with a peak in staffing.

A flat line in the staffing distribution chart (as shown in Figure 7-12) is yet another classic mistake. The flat line indicates the absence of the high plateau of Figure 7-8. The project is likely subcritical and is missing the resources to staff the noncritical activities of the original plan.

Figure 7-13 shows the staffing distribution for a sample subcritical project. This project goes subcritical at a level of 11 or 12 resources. It is not just missing the plateau, but has a valley instead.

Erratic staffing distributions (as in Figure 7-14) are yet another distress signal. Projects that are designed with this kind of elasticity in mind are due for a disappointment (see Figure 7-15) because staffing can never be that elastic. Most projects cannot conjure people out of thin air, have them be instantly productive, and then dispose of them a moment later. In addition, when people constantly come and go from a project, training (or retraining) them is very expensive. It is difficult to hold people accountable or retain their knowledge under such circumstances.

Figure 7-16 illustrates another staffing distribution to avoid, the high ramp-up coming into the project. While this figure does not include any numbers, the chart clearly indicates wishful thinking. No team can instantly go from zero to peak staffing and have everyone add value and deliver high-quality, production-worthy code. Even if the project initially has that much parallel work, and even if you have the resources, the network downstream throttles how many resources the project can actually absorb beyond that, and the required staffing fizzles out.

Figure 7-17 demonstrates such a project. This plan expects instantaneously to get to 11 people, and shortly afterward deflates to around six people until the end of the project. It is improbable that any team can ramp up this way, and the available resources are used inefficiently due to the oversized team.

Project Cost

Plotting the staffing distribution chart for each project design option is a great validation aid in reflecting on the option and seeing if it makes sense. With project design, if something does not feel right, more often than not, something is indeed wrong.

Drawing the staffing distribution chart offers another distinct benefit: It is how you figure out the cost of the project. Unlike physical construction projects, software projects do not have a cost of goods or raw materials. The cost of software is overwhelmingly in labor. This labor includes all team members, from the core team to the developers and testers. Labor cost is simply the staffing level multiplied by time:

Cost=Staffing*Time

Multiplying staffing by time is actually the area under the staffing distribution chart. To calculate the cost, you need to calculate that area.

The staffing distribution chart is a discrete model of the project that has vertical bars (the staffing level) in each time period between dates of interest. You calculate the area under the staffing distribution chart by multiplying the height of each vertical bar (the number of people) by the duration of the time period between its dates of interest (Figure 7-18). You then sum the results of these multiplications.

The formula for the calculation of the area under the staffing chart is:

Cost=Σi=1n(Si*(Ti−Ti−1))

where:

• Si is the staffing level at date of interest i.

• Ti is the date of interest i (T0 is the start date).

• n is the number of dates of interest in the project.

Finding the area under the staffing distribution chart is the only way to answer the question how much the project will cost.

If you use a spreadsheet to produce the staffing distribution chart, you just need to add another column with a running sum to calculate the area under the chart (in essence, a numerical integration). The support files accompanying this book contain several examples of this calculation.

Since cost is defined as staffing multiplied by time, the units of cost should be effort and time, such as man-month or man-year. It is better to use these units as opposed to currency to neutralize differences in salary, local currencies and budgets. It then becomes possible to objectively compare the cost of different project design options.

Given the architecture, the initial work breakdown, and the effort estimation, it is a matter of a few hours to a day at the most to answer the questions of how long it will take and how much it will cost to build the system. Sadly, most software projects are running blind. This is as sensible as playing poker without ever looking at the cards—except, instead of chips, you have your project, your career prospects, or even the company’s future on the line.

Project Efficiency

Once the project cost is known, you can calculate the project efficiency. The efficiency of a project is the ratio between the sum of effort across all activities (assuming perfect utilization of people) and the actual project cost. For example, if the sum of effort across all activities is 10 man-months (assuming 30 workdays in a month), and the project cost is 50 man-months (of regular workdays), then the project efficiency is 20%.

The project efficiency is a great indicator of the quality and sanity of the project’s design. The expected efficiency of a well-designed system, along with a properly designed and staffed project, ranges between 15% and 25%.

These efficiency rates may seem appallingly low, but higher efficiency is actually a strong indicator of an unrealistic project plan. No process in nature can ever even approach 100% efficiency. No project is free from constraints, and these constraints prevent you from leveraging your resources in the most efficient way. By the time you add the cost of the core team, the testers, the Build and DevOps, and all the other resources associated with your project, the portion of the effort devoted to just writing code is greatly diminished. Projects with high efficiency such as 40% are simply impossible to build.

Even 25% efficiency is on the high side and is predicated on having a correct system architecture that will provide the project with the most efficient team (see Figure 7-1) and a correct project design that uses the smallest level of resources and assigns them based on floats. Additional factors required for delivering on high efficiency expectations include a small, experienced team whose members are accustomed to working together, and a project manager who is committed to quality and can handle the complexity of the project.

Efficiency also correlates with staffing elasticity. If staffing were truly elastic (i.e., you could always get resources just when you need them and let them go at the precise moment when you no longer need them), the efficiency would be high. Of course, staffing is never that elastic, so sometimes resources will be idle while still assigned to the project, driving the efficiency down. This is especially the case when utilizing resources outside the critical path. If a single person is working on all critical activities, that person is actually at peak efficiency because the person works on activities back-to-back, and the cost of that effort approaches the sum of the cost of the critical activities. With noncritical activities, there is always float. Since staffing is never truly elastic, the resources outside the critical path can never be utilized at very high efficiency.

If the project design option has a high expected efficiency, you must investigate the root cause. Perhaps you assumed too liberal and elastic staffing or the project network is too critical. After all, if most network paths are either critical or near-critical (most activities have low float), then you would get a high efficiency ratio. However, such a project is obviously at high risk of not meeting its commitments.

Classic Mistakes

The realization that you can gauge the expected throughput of the team from the earned value chart enables you to quickly discern mistakes in your project plan. For example, consider the planned earned value chart in Figure 7-20. No team in the world could ever deliver on such a plan. For much of the project, the expected throughput was shallow. What kind of miracle of productivity would deliver the rocket launch of earned value toward the end of the project?

Such unrealistic, overly optimistic plans are usually the result of back-scheduling. The plan may even start with the best of intentions, progressing along the critical path. Unfortunately, you find that someone has already committed the project on a specific date with no regard for a project design or the actual capabilities of the team. You then take the remaining activities and cram them against the deadline, basically back-scheduling from that. Only by plotting the planned earned value will you be able to call attention to the impracticality of this plan and try to avert failure. Figure 7-21 depicts a project with such behavior.

In much the same way, you can detect unrealistically pessimistic plans such as that shown in Figure 7-22. This project starts out well, but then productivity is expected to suddenly diminish—or more likely, the project was given much more time than was required. The project in Figure 7-22 will fail because it allows for gold plating and complexity to raise their heads. You can even extrapolate from the healthy part of the curve when the project should have finished (somewhere above the knee in the curve).

The Shallow S Curve

A project utilizing a fixed-size team would always results in a straight line on the planned earned value chart. As mentioned already, you should not keep the team size fixed. A properly staffed and well-designed project always results in a shallow S curve for the earned value chart, as shown in Figure 7-23.

The shape of the planned earned value curve is related to the planned staffing distribution. At the beginning of the project, only the core team is available, so not much measurable value is added at the front end, and the pitch of the earned value curve is almost flat. After the SDP review, the project can start adding people. As you increase the size of the team, you will also increase its throughput, so the earned value curve gets steeper and steeper. At some point you reach peak staffing. For a while the team size is mostly fixed, so there is a straight line at maximum throughput in the center of the curve. Once you start phasing out resources, the earned value curve levels off until the project completes. Figure 7-24 shows a sample shallow S curve.

The Logistic Function

The shallow S curve of the planned earned value is a special case of the logistic function.^a The generic form of the logistic function can take any S shape (S, mirror S, inverted S, ascending or descending), can span any range of values, and can even be asymmetric.

Every process that involves change can be modeled using a logistic function. For example, the temperature in a room rises and falls according to a logistic function, as does your body weight, the market share of a company, radioactive decay, the risk of burning your skin as a function of distance from a flame, statistical distributions, population growth, effectiveness of design, the intelligence of neural networks, and pretty much everything else. The logistic function is the single most important function known to mankind because it enables us to quantify and model the world—a world that is highly dynamic.

The standard logistic function is defined by this expression:

F(x)=11+e−x

Figure 7-25 plots the standard logistic function. The standard logistic function approaches 0 and 1 asymptotically, crossing the y-axis when x = 0 and y = 0.5.

Subsequent chapters will refer to the logistic function in a qualitative manner when putting it to good use for risk and complexity modeling.

a. https://en.wikipedia.org/wiki/Logistic_function

The earned value curve is a simple and easy way to answer the question: “Does the plan make sense?” If the planned earned value is a straight line, or it exhibits the issues of Figure 7-20 or Figure 7-22, the project is in danger. If it looks like a shallow S, then at least you have hope that the plan is sound and sensible.