“The problem with communication ... is the illusion that it has been accomplished.” | ||
| --George Bernard Shaw | ||
In Chapter 1, “Eating Elephants Is Difficult,” we introduced IT environmental complexity as the major roadblock for the successful delivery of big IT projects. Not only are the projects themselves complex, but the environments into which they deliver their new systems are, too.
Why is complexity such an inhibitor for success? It’s not the actual complexity that causes the problem—it is understanding and communicating that complexity. Consider the parable of the blind men and the elephant—a story so old, its origin is unclear:
Once upon a time, not so long ago, three blind men were happily sitting together on the park bench that they frequented every day. From that bench, they talked with each other and heard the world go by.
One day, they heard the crunch of huge footprints. Not knowing what approached, they became alarmed.
A passerby saw their alarm and reassured them, “The circus is in town. What you hear are their elephants strolling through the park.” None of the men had ever been near an elephant—at the zoo, they were on the other side of a big ditch—so the kind passerby led them to the nearest beast.
One man was led to the side of its head, the next to its side, and the last to its trunk. Each raced his hands over the amazing animal. The men filled their heads with their experience of what an elephant was really like.
After they returned to their bench, they compared notes. “It was beautiful and delicate, just like a huge sail,” said the first man, who had held its ear. “No, no!” said the second. “It was more like a huge wall of flesh.” “Rubbish!” said the third, who had held the trunk. “An elephant is clearly some kind of snake.”
An argument ensued, but each man knew he was right and refused to concede. The bench became a less pleasant place to be.
The parable provides an important lesson in holism and ensuring that you understand the whole problem. In many regards, this parable is at the heart of one of the key messages in this book—but in our problem, there are not three blind men, but more than a hundred people, each with his or her own disconnected view of the elephant and an absolute belief that his or her view of the problem is the right one.
By definition, any large project is executed by a large number of people with different roles and responsibilities. Each person needs to coordinate his or her activities and outputs with those of the other people working on the project. This occurs through varying degrees of formality, including documents, conversations, plans, notice boards, e-mails, methods, and even instant messages. As the project gets larger and more complex, more people are needed to handle that complexity, and the amount of communication grows.
Fred Brooks identified this problem in 1975 in his seminal work on software engineering, The Mythical Man Month. While delivering the hugely complex OS/360 operating system[1] that ran IBM’s new generation of mainframes, Brooks noticed that adding more people to a project increased the communication overhead and actually reduced productivity. Adding people to a late software project to speed it up actually makes it later. This observation has become known as Brooks’s Law.
Of course, Brooks had it easy. All he had to do was control communication within his own local team. Today’s environments are more complex: We rarely have the opportunity to build a complete, self-contained system on a Greenfield site with a local team. Whatever we build must satisfy the new capabilities required and also use the information and facilities that are present in the existing IT environment. For example, for any new system that must communicate with other existing systems, existing users must be able to operate it, and the existing maintenance staff must be able to support it. Each of those areas (business analysts looking at new functions, data modelers and database administrators looking at existing data and other system owners, and so on) requires us to find, talk to, and establish a working relationship with yet another group of people. This is increasingly difficult when groups are located in separate time zones.
With considerable communication difficulties arising between groups within projects, and between projects and their surrounding environments, how can we improve these areas? To solve these kinds of problems, we need to better understand the issues of complex communication. In the next section, we analyze why communication fails and what we can do to improve it.
Elephantine projects, by definition, involve a lot of people. Clearly, not everyone on the project can understand everything that is going on. Each person tends to concentrate on a particular view of the project.
For example, a project director who is responsible for the entire project’s success might have a very broad and shallow view, focused on the major commercial aspects and major deadlines of the project. A programmer who is responsible for writing a component of the overall solution might have a small and contained view, but this person would understand this area better than anyone else on the project. The chief architect might need to understand a little bit of the commercial and resourcing view, but would also need to know how that programmer’s component fits into the whole solution; this person’s view would be exceptionally broad but necessarily superficial.
These perspectives are central to the Brownfield approach that we outline in this book. Throughout the book, we call them Views. Views are formal, bounded perspectives of an overall project or problem. Views are typically maintained by a single person or a like-minded, similarly skilled team. They can range from plans, estimates, or designs, to code or definitions of business processes. Views are the components from which projects are built.
Views can vary drastically in breadth and depth, but they typically contain a similar amount of information. We believe that Views generally become widely adopted and successful if they limit their scope to the amount of information that the average human brain can realistically and reliably store and process.
Looking across multiple projects, heuristic rules suggest that we can express the size of a View more precisely in terms of the function points introduced in Chapter 1. We believe that a successful View typically contains no more than 2,000 function points.[2] To determine this, we have used a number of estimating techniques, including these:
Dividing the number of function points in a variety of large projects by the number of people who, as a group, are meant to understand all of it
Asking individuals to remember the largest project in which they knew all the relevant details
When a large project might deliver tens of thousands of function points in an environment that contains 500,000 function points, the implications of this limitation should be obvious.
Using this rationale, it should also be clear that the size of the project and the output environment can create communication problems. To understand the real environment (or Brownfield site) that you are delivering into at the level of detail that will allow your integration to work, you need access to many Views. Few, if any, of the people on your project will maintain or understand those environmental Views.
Unfortunately, as the responsibilities on the project are divided among people, the “2,000 function point” Views they choose to do their jobs can never be entirely independent. The definition of the hardware that runs the project is inextricably linked to the design of the business application and the supporting software that has been chosen. Each of the Views partially overlaps the others. These three overlapping Views of hardware, application design, and supporting software are all linked by a performance model View that describes how the system will behave when users execute their business tasks on the system. These business tasks are described in a business process model View, which, in turn, is linked... and so on.
This tendency to segment the whole into a complex network of intersecting and overlapping Views causes three types of communication difficulties:
Inconsistency
Ambiguity
Parochialism
Inconsistency arises because the Views overlap. A well-engineered project aims to remove duplicate information across Views, but some duplication is inevitable. For example, duplication is required so that we can trace information from one document to another. (Some common information must exist to make the link.) In other cases, we can summarize or decompose information between documents.
Documenting the requirements and solution of a big project can produce large amounts of information. As you saw in Chapter 1, the requirements themselves (including functional, nonfunctional, and constraints) can easily be more than 10,000 pages. Most projects involve writing requirements documents first, followed by architecture and design documents; then the solution is coded and tested. A network of ancillary documents supports each of these documents—plans, estimating models, and status reports that support the overall management of the project. Clearly, if the requirements change significantly when the coding is being performed, it produces a cumulative effect on all the intervening and ancillary documentation.
For example, if a programmer notices during the project’s coding stage that the system under construction needs to talk to another system, many previously written documents will need to be updated, including plans, architectures, designs, and estimation models. In addition, many people will need to know about those changes so that they can act on them. Whenever the shared information is updated in one place, it must be updated in all the other places it is found. Unfortunately, this requires everyone to know where the information is duplicated and to follow a rigorous process.
As the project and its environment grow more complex, these effects become more pronounced and more difficult to efficiently administer. As complexity grows, the number of Views that are required to encompass the complexity grows. As the number of Views grows, the number of potential connections between them grows. The more connections exist, the greater the burden of keeping everything in step and the reduced likelihood that it actually will happen. This growth in connections is exponential instead of linear in nature because every new View could conceivably be connected to every previous View. In reality, each new View is connected to only a small proportion of the other Views, but even in such a model, the communication overhead grows faster than the size of the project.
Ultimately, either the manual process of maintaining these tens of thousands of pages of paper documents breaks, or the overhead it causes slows the project to a snail’s pace. In either case, the ability to manage the project’s and the surrounding environment’s complexity is lost. The project is undermined by the inability to efficiently manage the complexity and communicate it.
Therefore, despite the dire warning of The Mythical Man Month, productivity on such large projects is almost always overestimated. As a result, project managers still tend to throw people at the problem when things go slower than expected, which can result in a downward spiral and project failure. To solve this problem, any Brownfield-oriented approach needs to remove the inconsistency issue between Views.
The English language is imprecise and conducive to miscommunication. Much English-language humor is based on its capability to be ambiguous. As Groucho Marx famously said, “I shot an elephant in my pajamas. What he was doing in my pajamas, I’ll never know.”
A quick look in a thesaurus confirms that many individual words are ambiguous. If something is hard, does that mean that it is difficult or that it would hurt if you ran into it?
In addition, our sentence structures can be ambiguous. Does complex project development mean “the task of development is complex on the project” or “the development of a complex project”?
Projects can have very long chains of communication, and any degree of ambiguity can be amplified as the message flows through the project. In “bad news diode” cultures such as the ones described in Chapter 1, this can be exploited. Indeed, we observed one example in which the message issued to the project was diametrically opposed to the one issued by the project sponsor. Just two layers of management exploited a degree of unintentional ambiguity in the original message.
With its lack of gender and noun/verb agreements, the English language almost encourages miscommunication. Anyone who has ever played the whisper game—in which a message is passed between a group, with each participant repeating the original phrase—knows that, after a few exchanges, the meaning of the phrase can change completely. Such information decay is not the result of participants intentionally exploiting the language’s ambiguity or deliberate lack of precision; it is just an everyday fact of communication failure. Probably the most famous example of this comes from English military history in World War I. The original message issued was “Send reinforcements—we’re going to advance.” After being relayed a number of times, the message received was “Send three and four pence[3]—we’re going to a dance.” Not surprisingly, the reinforcements did not arrive in time for the push forward.
To ensure that messages are understood, we often need to supplement our words with tone of voice and body language. Good face-to-face communicators also test common understanding during a conversation. Unfortunately, face-to-face communication is not always possible on large projects. On such projects, most conversation is asynchronous, in that information is issued without expecting any immediate acknowledgment. Whether e-mail based or via lengthy documents, the communication is essentially “fire and forget.” Yes, you might have an opportunity to review the document, or you might be able to question the e-mail, but these acts are the exception, not the rule. After a document or instruction is issued, it is used and reinterpreted without question.
This removal of real-time, nonverbal feedback often occurs when teams are geographically separated. The loss of body language and signals can cause misunderstandings because words and sentence structures can be ambiguous. Of course, this is becoming an ever-increasing problem on IT projects whose resources are spread across the globe.
Parochialism is a natural outcome of dividing a project into multiple areas of responsibility and asking each part to ensure the success of its own area. As a result, people consider only the elements of the problem or solution that are directly relevant to their View, and they consider themselves to be only a small part of the overall problem.
This kind of behavior is often endemic in large organizations. When Lou Gerstner became chairman of IBM, he was horrified by the parochial management style. He immediately changed the incentive scheme of his senior management team so that managers were rewarded by overall corporation success instead of primarily by the success of their part of the organization.
Parochialism kills projects because the success of only one View means nothing—only the success of the conglomeration of Views can actually deliver success.
Parochialism is not caused only by dividing responsibilities into specific Views. It is even more likely to arise when teams are geographically separated. In Peopleware,[4] Tom DeMarco and Timothy Lister list physical separation as a key technique for achieving teamicide—their term for the ability to prevent a successful team from forming.
The separation need not be large. On one project we observed, almost no communication occurred between the application architects and the middleware architects. This lack of coordination led to misunderstandings and overlaps and underlaps of Views, resulting in high levels of inconsistency. Ultimately, the project became a dysfunctional and highly fractious one that failed to deliver. Unbelievably, the teams who never spoke were in the same building, on separate floors. The physical distance between them wasn’t more than 12 feet, but it was enough to derail the entire project.
Splitting teams across buildings occurs often these days. Advances in worldwide communication and access to inexpensive but highly skilled labor across multiple continents means projects many times are split across the globe.
If communication between floors is difficult, then communication via long-distance phone calls, between cultures and time zones, is almost certain to cause communication difficulties. Projects are sometimes deliberately geographically split so that they can achieve around-the-clock effort—that is, a team somewhere in the world is always working on the project. These projects experience all kinds of communication issues. For example, it is impossible to schedule a convenient time for a joint teleconference. Such problems lead to intense parochialism, as the bond to the local team becomes far stronger than the bond to the project.
One relatively small project that I recently reviewed had its project manager in New Zealand, its application management in India, and its design team in New York—and it reported to a business based in London. The systems were located in a number of data centers around the globe. Each was maintained by a different company. Not everything ran as smoothly as it could. At the same customer, I worked with another team that was truly based around the globe. Some people were in Texas, others in the U.K., one or two in Asia, and the leader of the group was in Australia. The team could meet in person only once a year, and it was pretty clear that most of the individuals had a stronger network and team spirit built around their previous local roles than they did with their geographically dispersed team. | ||
| --R.H. | ||
The ultimate form of parochialism is the creation of a private language that only your own area of the project can understand.
Even when people are in the same room, the IT industry is so full of acronyms and specialist terms that people who have slightly different perspectives or backgrounds often can’t understand what is being said. This trend toward language divergence is apparent in all aspects of human life.
Another such group of like-minded individuals is stock brokers. When stock brokers talk, they speak a different language than the rest of the world. Most of the words seem familiar, but they are put together in strange ways that are difficult to understand for an outsider. This also occurs with businessmen, IT personnel, and even children. All of these groups use words or terms to mean different things, or they use acronyms as words. This is done to provide clarity and simplify communication within the group, but it creates a barrier to those outside. For example, teenagers’ text messaging is an entirely new experience:
f u tnk dat theres NP n transl8N thN l%k @ yr kids txt msgs. Itz a diFrent wrld n lang.
Translation: If you think that there is no problem in translation, then look at your kid’s text messages. It is a different world in language.
Looking at the original text message, you might be able to deduce most of the words, but this was a very simple example. It doesn’t contain any of the code words that you couldn’t really deduce, such as “CD9” for “Code 9,” which translates to “Parents Are Around,” or “MOS” for “Mom Over Shoulder.”
Linguists have a theory known as mosaic language zones[5] that explains this proliferation of languages. The mosaic theory suggests that in conditions of strong competition, languages tend to mutate and speciate. This divergence into many different tongues is thought to be a way to detect outsiders and enforce a sense of security. The language or dialect becomes a badge of group membership. Private languages are endemic in the IT industry, and although they might offer a small advantage in terms of communication efficiency within the same immediate group, they can cause many problems across a large project.
Private languages are a natural outcome of having highly specialized jobs in a highly competitive industry. It is unlikely that we can remove this blocker for better communication—and human nature might make it impossible. If this is the case, the Brownfield approach must embrace the need for private languages without impairing our ability to communicate between groups.
Perhaps the most disturbing and worrisome manifestation of parochialism is that business and IT do not share the same language. As a result, business and IT frequently misunderstand each other. Project requirements are lost in translation, and the distance between the business and IT often makes it difficult to resolve these misunderstandings. To resolve this, it must be possible to describe both the problem that the project will address and the problem’s proposed solution in a way that is meaningful to both the business and IT worlds.
Unfortunately, during the past 20 years, business and IT have moved farther apart. A decision made in the boardroom is conveyed to the IT department as a request to be fulfilled. No real dialogue exists. Confirming the requirements is difficult when virtually no two-way communication occurs.
Compounding this situation is the lack of success in delivering large projects to enable the accelerated business change that is necessary to be competitive. Little common language exists between business and IT, so the two have precious little to talk about other than high-level requirements, deadlines, milestones, and budgets. These terms form a very limited common ground.
Service Oriented Architecture (SOA) is beginning to soften this standoff. SOA breaks down business software into granular, business function-oriented chunks that organizations can flexibly combine to create new business and IT capabilities. When applied correctly, SOA aligns the services that the business wants to provide to its customers with the IT services that underpin them. In addition, the IT services are expressed in business terms instead of in terms of the underlying application or technology. This alignment provides a common, business-focused but precise language for the business and IT worlds to jointly define problems and solutions.
At the same time as SOA was introduced, some of the software-engineering techniques that have evolved over the past 20 years are beginning to be applied to reengineer the business itself. IBM’s component business modeling is an excellent example that uses the philosophy of designing large systems to conceptually break the business into small, largely self-contained capabilities. The organization then can determine whether duplication of capability exists within the organization. It is also possible to see which areas are unique and core to business success, and which areas are less critical. Those less critical areas could be sold off or outsourced.
This convergence of business modeling and IT modeling provides an opportunity for the business and IT perspectives to align. This clearly can improve communications and, ultimately, result in a better ability to provide more aligned solutions to business problems.
We summarize this principle in a Brownfield Belief. The Brownfield Beliefs are a set of directives that summarize the difference between traditional Greenfield and Brownfield techniques. The first principle is this:
Make Business and IT Indivisible
SOA provides a fantastic end goal for this principle, but many businesses struggle with the migration from their existing, highly complex Brownfield environments to the new, rationalized, and business-oriented services environments.
Therefore, our Brownfield approach aims to remove inconsistency and ambiguity from communication and to enable the establishment of a language that spans both business and IT.
In this chapter, you have seen that inconsistency, ambiguity, and parochialism are unfortunate but natural aspects of how we communicate. In any situation when a complex task has to be divided into a number of different tasks, you are likely to observe such problems, whether you are working in civil engineering or IT engineering. When the project involves the highly abstract concepts of software design (as opposed to bricks and mortar), the problems are exaggerated because the core complexity of the thing you are trying to build is itself expressed in a language (the code that the system runs).
To cope with the accumulated complexity of today’s systems and deliver projects reliably and efficiently, we need an approach that absorbs complexity and copes with the inherent communication problems that people face. This cacophony of communication (or lack of communication) is illustrated by the small subset shown in Figure 2.1. To this small subset, we could add many different types of code, models, requirements, process definitions, methods, and tools. Each one has its own largely private language preventing communication and allowing the gaps and inconsistencies between the Views to be poorly understood.
We have concluded that, when faced with a hugely complex environment and a massive program of change, the only way to cope is to build something to handle the complexity for us. We need a machine to eat our elephant. We need an Elephant Eater. In the next chapter, we consider how such a beast might be built.