3. The Architecture Design Process
In this chapter we provide a detailed discussion of ADD, the design method that is the focus of this book. We begin with an overview of the method and of each one of its steps. This overview is followed by more detailed discussions of different aspects that need to be considered when performing these steps. We suggest different roadmaps that provide guidance on when different types of design concepts can be used depending on which type of system is being designed. We also discuss the identification and selection of design concepts, the production of structures from these design concepts, the definition of interfaces, the production of preliminary documentation, and, finally, a technique to track design progress.
3.1 The Need for a Principled Method
In Chapter 2, we discussed the various concepts associated with design. The question is, how do you actually perform design? Performing design to ensure that the drivers are satisfied requires a principled method. By “principled”, we refer to a method that takes into account all of the relevant aspects that are needed to produce an adequate design. Such a method provides guidance that is necessary to guarantee that your drivers are satisfied. To achieve this goal in a cost-effective, repeatable way, you need a method that guides you in combining and incorporating reusable design concepts.
Performing design adequately is important because architecture design decisions have significant consequences at different points in a project’s lifetime. For example, during a pre-sales phase, an appropriate design will allow for a better estimation of cost, scope, and schedule. During development, an appropriate design will be helpful to avoid later rework and facilitate development and deployment. Finally, a clear understanding of what architectural design involves is necessary to better manage aspects of technical debt.
3.2 Attribute-Driven Design 3.0
Architecture design is performed in a series of rounds across the development of a software project. Each design round may take place within a project increment such as a sprint. Within these rounds, a series of design iterations is performed. Perhaps the most important characteristic of the ADD method is that it provides detailed, step-by-step guidance on the tasks that have to be performed inside the design iterations (see Chapter 7 for a comparison with other design methods). When ADD appeared, it was the first method to focus specifically on quality attributes and their achievement through the selection of different types of structures and their representation through views. Another important contribution of ADD was that it recognized that analysis and documentation are an integral part of the design process. Although ADD was and is a major contribution in the field of software architecture, we believe that its adoption within the practitioner community has been limited by a number of inherent weaknesses, as discussed in Section 1.4.
ADD has been used successfully for more than 15 years. The world of software has changed dramatically since ADD’s introduction, however, and even more since version 2.0 was published in 2006. For this reason, and to address the weaknesses of version 2.0, we have decided to create ADD 3.0. Henceforth, we will simply refer to this method as ADD. Figure 3.1 shows the steps and artifacts associated with ADD and in the following subsections we provide an overview of the activities in each of its steps.
3.2.1 Step 1: Review Inputs
Before starting a design round, you need make sure that the inputs to the design process are available and correct. First, you need to ensure that you are clear about the purpose for the design activities that will ensue. The purpose may be, for example, to produce a design for early estimation, to refine an existing design to build a new increment of the system, or to design and generate a prototype to mitigate certain technical risks (see Section 2.4.1 for a discussion of the design purpose). Also, you need to make sure that the other drivers needed for the design activity are available. These include primary functional requirements, quality attribute scenarios, architectural constraints, and concerns. Finally, if this is not the first design round, or if this is not greenfield development, an additional input that you need to consider is the existing architecture design.
At this point, we assume that primary functionality and quality attribute scenarios have been prioritized, ideally by your most important project stakeholders. (If not, there are techniques that you can employ to elicit and prioritize them, as discussed in Sections 2.4.2 and 2.4.3.) You, as the architect, must now “own” these drivers. You need to check, for example, whether any important stakeholders were overlooked in the original requirements elicitation process, or whether any business conditions have changed since the prioritization was performed. These drivers really do “drive” design, so getting them right and getting their priority right is crucial. We cannot stress this point strongly enough. Software architecture design, like most activities in software engineering, is a “garbage in, garbage out” process. The results of ADD cannot be good if the inputs are poorly formed.
As a rule of thumb, you should be able to start designing if, besides the design purpose, constraints, and initial architectural concerns, you have established the primary use cases and the most important quality attribute scenarios. This, of course, does not mean you will make decisions only about these drivers: You still need to address other quality attribute scenarios, use cases and architectural concerns, but these can be treated later on.
The drivers become part of an architectural design backlog that you should use to perform the different design iterations. We discuss this idea in more depth in Section 3.8.1.
3.2.2 Step 2: Establish the Iteration Goal by Selecting Drivers
A design round represents the architecture design activities performed within a development cycle if an iterative development model is used, or the whole set of architecture design activities if a waterfall model is used. Through one or more rounds, you produce an architecture that suits the established design purpose.
A design round is generally performed in a series of design iterations, where each iteration focuses on achieving a particular goal. Such a goal typically involves designing to satisfy a subset of the drivers. For example, an iteration goal could be to create structures from elements that will support a particular performance scenario, or that will enable a use case to be achieved. For this reason, when performing design, you need to establish a goal before you start a particular design iteration.
As we will discuss in Section 3.3, depending on the type of system whose architecture is being designed, there may be a “best”—or at least strongly suggested—ordering of the iteration goals that need to be addressed. For example, for a greenfield system in a mature domain, your initial goal is typically to identify an overall structure for the system by choosing a reference architecture.
3.2.3 Step 3: Choose One or More Elements of the System to Refine
Satisfying drivers requires you to produce one or more architectural structures. These structures are composed of interrelated elements, and those elements are generally obtained by refining other elements that you previously identified in an earlier iteration. Refinement can mean decomposition into finer-grained elements (top-down approach), combination of elements into coarser-grained elements (bottom-up approach), or improvement of previously identified elements. For greenfield development, you can start by establishing the system context and then selecting the only available element—that is, the system itself—for refinement by decomposition. For existing systems or for later design iterations in greenfield systems, you normally choose to refine elements that were identified in prior iterations.
The elements that you will select are the ones that are involved in the satisfaction of specific drivers. For this reason, when design is performed for an existing system, you need to have a good understanding of the elements that are part of the as-built architecture of the system. This may involve some “detective work”, reverse engineering, or discussions with developers.
We have presented steps 2 and 3 in the order they appear in the method. That is to say, step 2 precedes step 3. However, in some cases you may need to reverse this order. For example, when designing a greenfield system or when fleshing out certain types of reference architectures (as we will show in Chapter 5), you will, at least in the early stages of design, focus on elements of the system and start the iteration by selecting a particular element and then consider the drivers that you want to address.
3.2.4 Step 4: Choose One or More Design Concepts That Satisfy the Selected Drivers
Choosing the design concepts is probably the most difficult decision you will face in the design process, because it requires you to identify alternatives among design concepts that can be used to achieve your iteration goal, and to make a selection from these alternatives. As we saw in Section 2.5, different types of design concepts exist, and, for each type, there may be many options. This can result in a considerable number of alternatives that need to be analyzed to make a choice. In Section 3.4, we discuss the identification and selection of design concepts in more detail.
3.2.5 Step 5: Instantiate Architectural Elements, Allocate Responsibilities, and Define Interfaces
Once you have selected one or more design concepts, you must make another design decision, which involves instantiating elements out of the design concepts that you selected. For example, if you selected the Layers pattern as a design concept, you must decide how many layers will be used, since the pattern itself does not prescribe a specific number. In this example, the layers are the elements that are instantiated. In certain cases, instantiation can mean configuration. For example, you may have dedicated an iteration to selecting technologies and associating them with the elements in your design. In further iterations, you might refine these elements by making finer-grained decisions about how they should be configured to support a particular driver, such as a quality attribute.
After instantiating the elements, you need to allocate responsibilities to each of them. For example, in a typical web-based enterprise system, at least three layers are usually present: the presentation layer, the business layer, and the data layer. The responsibilities of these layers differ: The responsibilities of the presentation layer include managing all of the user interactions, whereas the responsibilities of the data layer include managing the persistence of data.
Instantiating elements is just one of the tasks you need to perform to create structures that satisfy a driver or a concern. The elements that have been instantiated also need to be connected, to allow them to collaborate with one another. This requires the existence of relationships between the elements and the exchange of information through some kind of interface. The interface is a contractual specification of how information should flow between the elements. Section 3.5 provides more details on how the different types of design concepts are instantiated and how structures are created, and Section 3.6 discusses how interfaces can be defined.
3.2.6 Step 6: Sketch Views and Record Design Decisions
At this point, you have finished performing the design activities for the iteration. Nevertheless, you may not have taken any actions to ensure that the views—the representations of the structures you created—are preserved. For instance, if you performed the previous step in a conference room, you probably ended up with a series of diagrams on a whiteboard. This information is essential, and you need to capture it so that you can later analyze and communicate it to other stakeholders.
The views that you have created are almost certainly incomplete, so these diagrams may need to be revisited and refined in a subsequent iteration. This is typically done to accommodate elements resulting from other design decisions that you will make to support additional drivers. This factor explains why we speak of “sketching” the views in ADD—that is, creating a preliminary type of documentation. The more formal, more fully fleshed-out documentation of these views—should you choose to produce them—occurs only after a number of design iterations have been finished (as part of the architectural documentation activity discussed in Section 1.2.2).
In addition to storing the sketches of the views, you should record the significant decisions that are made in the design iteration, and the reasons that led to these decisions (i.e., the rationale), to facilitate later analysis and understanding of the decisions. For example, decisions about important tradeoffs might be recorded at this time. During a design iteration, decisions are primarily made in steps 4 and 5. Section 3.7 provides further information on how to create preliminary documentation during design, including creating sketches, recording design decisions and their rationale.
3.2.7 Step 7: Perform Analysis of Current Design and Review Iteration Goal and Achievement of Design Purpose
By the time you reach step 7, you should have created a partial design that addresses the goal established for the iteration. Making sure that this is actually the case is a good idea, so as to avoid unhappy stakeholders and later rework. You can perform the analysis yourself by reviewing the sketches of the views and design decisions that you recorded, but an even better idea is to have someone else help you review this design. We do this for the same reason that organizations frequently have a separate testing/quality assurance group: Another person will not share your assumptions, and will have a different experience base and a different perspective. Pulling in someone with a different point of view can help you find “bugs”, in both code and architecture. We discuss analysis in more depth in Chapter 8.
Once the design performed in the iteration has been analyzed, you should review the state of your architecture in terms of the established design purpose. This means considering if, at this point, you have performed enough design iterations to satisfy the drivers that are associated with the design round as well as considering whether the design purpose has been achieved or if additional design rounds are needed in future project increments. Section 3.8 describes simple techniques that allow you to keep track of design progress.
3.2.8 Iterate If Necessary
Ideally, you should perform additional iterations and repeat steps 2 to 7 for every driver that was considered as part of the input. More often than not, such iterations are not possible because of time or resource constraints that force you to stop the design activities and move on to the next activities in the development process—typically implementation.
What are the criteria for evaluating if more design iterations are necessary? We let risk be our guide. You should at least have addressed the drivers with the highest priorities. Ideally, you should have assured that critical drivers are satisfied or, at least, that the design is “good enough” to satisfy them. Finally, when performing iterative development, you can choose to perform one design round in every project iteration. The first rounds should focus on addressing the drivers, while subsequent rounds focus on making design decisions for other requirements that were not selected as drivers but that need to be addressed nonetheless.
3.3 Following a Design Roadmap According to System Type
When writing, you might have experienced the much-dreaded “fear of the blank page”. Similarly, when you start designing an architecture, you may face a situation in which you ask yourself, “How do I begin designing?” To answer this question, you need to consider which type of system you are designing.
Design of software systems falls into three broad categories: (1) the design of a greenfield system for a mature (i.e., well-known) domain; (2) the design of a greenfield system for a domain that is novel (i.e., a domain that has a less established infrastructure and knowledge base); and (3) the design for making changes to an existing system (brownfield). Each one of these categories involves a different roadmap in terms of the sequence of goals that you should perform across the design iterations.
3.3.1 Design of Greenfield Systems for Mature Domains
The design of a greenfield system for a mature domain occurs when you are designing an architecture for a system that is built from “scratch” and when this type of system is well known and understood—that is, when there is an established infrastructure of tools and technologies, and an associated knowledge base. Examples of mature domains include the following:
Traditional desktop applications
Interactive applications that run on a mobile device
Enterprise applications accessed from a web browser, which store information in a relational database, and which provide support for partially or fully automating business processes
Since these types of applications are relatively common, some general architectural concerns associated with their design are well known, well supported, and well documented. If you are designing a new system that falls into this category, we recommend the following roadmap (shown in Figure 3.2).
The goal of your initial design iteration(s) should be to address the general architectural concern of establishing an initial overall system structure. Is this to be a three-tier client-server application, a peer-to-peer application, a mobile app connecting to a Big Data back-end, and so on? Each of these options will lead you to different architectural solutions, and these solutions will help you to achieve your drivers. To achieve this iteration goal, you will select some design concepts. Specifically, you will typically choose one or more reference architectures and deployment patterns (see Sections 2.5.1 and 2.5.3). You may also select some externally developed components, such as frameworks. The types of frameworks that are typically chosen in early iterations are either “full-stack” frameworks that are associated with the selected reference architectures, or more specific frameworks that are associated with elements established by the reference architecture (see Section 2.5.5). In this first iteration, you should review all of your drivers to select the design concepts, but you will probably pay more attention to the constraints and to quality attributes that are not associated with specific functionalities and that favor particular reference architectures or require particular deployment configurations. Consider an example: If you select a reference architecture for Big Data systems, you have presumably chosen a quality attribute such as low latency with high data volumes as your most important driver. Of course, you will make many subsequent decisions to flesh out this early choice, but this driver has already exerted a great influence on your design such as the selection of a particular reference architecture.
The goal of your next design iteration(s) should be to identify structures that support the primary functionality. As noted in Section 2.4.3, allocation of functionality (i.e., use cases or user stories) to elements is an important part of architectural design because it has critical downstream implications for modifiability and allocation of work to teams. Furthermore, once functionality has been allocated, the elements that support it can be refined in later iterations to support the quality attributes associated with these functionalities. For example, a performance scenario may be associated with a particular use case. Achieving the performance goal may require making design decisions across all of the elements that participate in the achievement of this use case. To allocate functionality, you usually refine the elements that are associated with the reference architecture by decomposing them. A particular use case may require the identification of multiple elements. For example, if you have selected a web application reference architecture, supporting a use case will probably require you to identify modules across the different layers associated with this reference architecture. Finally, at this point you should also be thinking about allocating functionality—associated with modules—to (teams of) developers.
The goal of your subsequent design iterations should be to refine the structures you have previously created to fully address the remaining drivers. Addressing these drivers, and especially quality attributes, will likely require you to use the three major categories of design concepts—tactics, patterns, and externally developed components such as frameworks—as well as commonly accepted design best practices such as modularity, low coupling, and high cohesion. For example, to (partially) satisfy a performance requirement for the search use case in a web application, you might select the “maintain multiple copies of data” tactic and implement this tactic by configuring a cache in a framework that is used inside an element responsible for persisting data.
This roadmap is appropriate for the initial project iterations, but it is also extremely useful for early project estimation activities (see the discussion about the architecture design process during pre-sales in Section 9.1.1). Why have we created such a roadmap? First, because the process of starting an architectural design is always complex. Second, because many of the steps in this roadmap are frequently overlooked or done in an intuitive and ad hoc way, rather than in a well-considered, reflective way. Third, because different types of design concepts exist, and it is not always clear at which point in the design they should be used. This roadmap encapsulates best practices that we have observed in the most competent architecture organizations. Simply put, the use of a roadmap results in better architectures, particularly for less mature architects.
3.3.2 Design of Greenfield Systems for Novel Domains
In the case of novel domains, it is more challenging to establish a precise roadmap, because reference architectures may not exist and there may be few, if any, externally developed components that you can use. You are, more than likely, working from first principles and creating your own home-grown solutions. Even in this case, however, general-purpose design concepts such as tactics and patterns can guide you, aided by strategic prototyping. In essence, your iteration goals will mostly be to continuously refine previously created structures to fully address the drivers.
Many times, your design goal will focus on the creation of prototypes so that you can explore possible solutions to the challenge that you are facing. In particular, you may need to focus on quality attributes and design challenges oriented toward issues such as performance, scalability, or security. We discuss the creation of prototypes in Section 3.4.2.
Of course, the notion of “novel” is fluid. Mobile application development was a novel domain 10 or 15 years ago, but now it is a well-established field.
3.3.3 Design for an Existing System (Brownfield)
Architecture design for an existing system may occur for different purposes. The most obvious is maintenance—that is, when you need to satisfy new requirements or correct issues, and doing so requires changes to the architecture of an existing system. You may also be making architectural changes to an existing system for the purpose of refactoring. When refactoring, you change the architecture of an existing system, without altering its functions, to reduce technical debt, to introduce technology updates, or to fix quality attribute problems (e.g., the system is too slow, or insecure, or frequently crashes).
To be able to choose elements to decompose as part of the design process (step 3 of ADD), you need to first identify which elements are present in the architecture of the existing system. In this sense, before starting the design iterations, your first goal should be to make sure that you have a clear understanding of the existing architecture of the system.
Once you understand the elements, properties, and relationships that constitute the architecture of the system, and the characteristics of the existing code base, you can perform design similar to what is done for greenfield systems after the initial design iteration. Your design iteration goals here will be to identify and refine structures to satisfy architectural drivers, including new functionality and quality attributes, and to address specific architectural concerns. These design iterations will typically not involve establishing a new overall system structure unless you are dealing with a major refactoring.
It might seem that the preceding discussion of the different contexts of design is rather abstract and perhaps even confusing. In the next three chapters we will be presenting examples of design of a system in a mature domain (Chapter 4), design for a system in a relatively novel domain (Chapter 5), and design to modify an existing system (Chapter 6). These extended examples will make the previously described concepts clearer and more concrete.
3.4 Identifying and Selecting Design Concepts
Freeman Dyson, the English physicist, once said the following: “A good scientist is a person with original ideas. A good engineer is a person who makes a design that works with as few original ideas as possible”. This quotation is particularly relevant in the context of software architecture design: Most of the time you don’t need to, and shouldn’t, reinvent the wheel. Rather, your major design activities are to identify and select design concepts to address the challenges and drivers that you encounter across the design iterations. Design is still an original and creative endeavor, but the creativity resides in the appropriate identification of these existing solutions and then on combining and adapting them to the problem at hand.
3.4.1 Identification of Design Concepts
The identification of design concepts can appear to be daunting, because of the vast number of design concepts that exist. There are likely dozens of design patterns and externally developed components that you could use to address any particular issue. To make things worse, these design concepts are scattered across many different sources: in the popular press, in research literature, in books, and on the Internet. Moreover, in many cases, there is no canonical definition of a concept. Different sites, for example, will define the Broker pattern in different, largely informal, ways. Finally, once you have identified the alternatives that might potentially help you achieve the design goals of the iteration, you need to select among them.
To identify which design concepts you need at a particular point, you should consider what we previously discussed regarding the design roadmap. Different points in the design process usually require different types of design concepts. For example, when you are designing a greenfield system in a mature domain, the types of design concepts that will help you initially structure the system are reference architectures and deployment patterns. As you progress in the design process, you will use all of the categories of design concepts: tactics, architecture and design patterns, and externally developed components. Keep in mind that to address a specific design problem, you can and often will use and combine different types of design concepts. For example, when addressing a security driver, you may employ a security pattern, a security tactic, a security framework, or some combination of these.
Once you have more clarity regarding the types of design concepts that you wish to use, you still need to identify alternatives—that is, design candidates. There are several ways to do so, although you will probably use a combination of these techniques rather than a single one:
Leverage existing best practices. You can identify alternatives for your required design concepts by making use of catalogs that are available in printed or online form. Some design concepts, such as patterns, are extensively documented; others, such as externally developed components, are documented in a less thorough way. The benefits of this approach are that you can identify many alternatives, and that you can leverage the considerable knowledge and experience of others. The downsides are that searching for and studying the information can require a considerable amount of time, the quality of the documented knowledge is often unknown, and the assumptions and biases of the authors are unknown.
Leverage your own knowledge and experience. If the system you are designing is similar to other systems you have designed in the past, you will probably want to begin with some of the design concepts that you have used before. The benefit of this approach is that the identification of alternatives is performed rapidly and confidently. The downside is that you may end up using the same ideas repeatedly, even if they are not the most appropriate for all the design problems that you are facing, and if they have been superseded by newer, better approaches. As the saying goes, “If you give a small child a hammer, all the world looks like a nail”.
Leverage the knowledge and experience of others. As an architect, you have background and knowledge that you have gained through the years. This foundation varies from person to person, especially if the types of design problems they have addressed in the past differ. You can leverage this information by performing the identification and selection of design concepts with some of your peers through brainstorming.
3.4.2 Selection of Design Concepts
Once you have identified a list of alternative design concepts, you need to select which one is the most appropriate to solve the design problem at hand. You can achieve this in a relatively simple way, by creating a table that lists the pros and cons associated with each alternative and selecting one of the alternatives based on those criteria and your drivers. The table can also include other criteria, such as the cost associated with the use of the alternative. Table 3.1 shows an example of such a table used to support the selection of different reference architectures.
You may also need to perform a more in-depth analysis to select the alternative. Methods such as CBAM (cost benefit analysis method) or SWOT (strengths, weaknesses, opportunities, threats) can help you to perform this analysis (see the sidebar “The Cost Benefit Analysis Method”).
The Cost Benefit Analysis Method
The CBAM is a method that guides the selection of design alternatives using a quantitative approach. This method considers that architectural strategies (i.e., combinations of design concepts) affect quality attribute responses, and that the level of each response in turn provides system stakeholders with some benefit called utility. Each architectural strategy provides a different level of utility, but also has a cost and takes time to implement. The idea behind the CBAM is that by studying levels of utility and costs of implementation, particular architectural strategies can be selected based on their associated return on investment (ROI). The CBAM was conceived to be performed after an ATAM (architecture tradeoff analysis method), but it is possible to use the CBAM during design—that is, prior to the moment where the architectural evaluation is performed.
The CBAM takes as its input a collection of prioritized traditional quality attribute scenarios, which are then analyzed and refined with additional information. The addition is to consider several levels of response for each scenario:
The worst-case scenario, which represents the minimum threshold at which a system must perform (utility = 0)
The best-case scenario, which represents the level after which stakeholders foresee no further utility (utility = 100)
The current scenario, which represents the level at which the system is already performing (the utility of the current scenario is estimated by stakeholders)
The desired scenario, which represents the level of response that the stakeholders are hoping to achieve (the utility of the desired scenario is estimated by stakeholders)
Using these data points, we can draw a utility–response curve, as shown in the figure. After the utility–response curve is mapped for each of the different scenarios, a number of contemplated design alternatives may be considered, and their expected response values can be estimated. For example, if we are concerned about mean time to failure, we might consider three different architectural strategies (i.e., redundancy options)—for example, no redundancy, cold spare, and hot spare. For each of these strategies, we could estimate their expected responses (i.e., their expected mean times to failure). In the graph shown here, the “e” represents one such option, placed on the curve based on its expected response measure.
Using these response estimates, the utility values of each architectural strategy can now be determined via interpolation, which provides its expected benefit. The costs of each architectural strategy are also elicited—one would expect hot spare to be the most costly, followed by cold spare and no redundancy.
Given all of this information, architectural strategies can now be selected based on their expected value for cost.
Although the CBAM may seem relatively complex and time-consuming at first, you need to consider that some design decisions can have enormous economic consequences—in terms of their costs, their benefits, and their effects on project schedule. You must decide if you are willing to take the chance of making these decisions solely using a gut-feeling approach versus this more rational and systematic approach.
In case the previous analysis techniques do not guide you to make an appropriate selection, you may need to create throwaway prototypes and collect measurements from them. The creation of early throwaway prototypes is a useful technique to help in the selection of externally developed components. This type of prototype is usually created in a “quick and dirty” fashion without too much consideration for maintainability or reuse. For these reasons, it is important to keep in mind that throwaway prototypes should not be used as a basis for further development.
Although the creation of prototypes can be costly compared to analysis (the ratio of costs is between 10 and 5 to 1, according to our sources), certain scenarios strongly motivate the creation of prototypes. Aspects that you should consider when deciding whether you will create a prototype include the following:
Does the project incorporate emerging technologies?
Is the technology new in the company?
Are there certain drivers, particularly quality attributes, whose satisfaction using the selected technology presents risks (i.e., it is not understood if they can be satisfied)?
Is there a lack of trusted information, internal or external, that provides some degree of certainty that the selected technology will be useful to satisfy the project drivers?
Are there configuration options associated with the technology that need to be tested or understood?
Is it unclear whether the selected technology can be integrated with other technologies that are used in the project?
If most of your answers to these questions are “yes”, then you should strongly consider the creation of a throwaway prototype.
When identifying and selecting design concepts, you need to keep in mind the constraints that are part of the architectural drivers, because some constraints will restrict you from selecting particular alternatives. For example, a constraint might require that all libraries and frameworks in the system do not use the GPL license; thus, even if you have found a framework that could be useful for your needs, you may need to discard it if it has a GPL license. Also, you need to keep in mind that the decisions regarding the selection of design concepts that you have made in previous iterations may restrict the design concepts that you can select in the future because of incompatibilities. For example, if you selected a web application reference architecture for use in an initial iteration, you cannot select a user interface framework intended for local applications in a subsequent iteration.
Finally, you need to remember that even though ADD provides guidance on how to perform the design process, it cannot ensure that you will make appropriate design decisions. Thorough reasoning and considering different alternatives (not just the first thing that comes to mind) are the best means to improve the odds of finding a good solution. We discuss doing “analysis in the design process” in Chapter 8.
3.5 Producing Structures
Design concepts per se won’t help you satisfy your drivers unless you produce structures; that is, you need to identify and connect elements that are derived from the selected design concepts. This process is the instantiation of architectural elements in ADD: creating elements and relationships between them, and associating responsibilities with these elements. It is important to remember that the architecture of a software system is composed of a set of structures, which can be grouped into three major categories:
Module structures: composed of logical and static elements that exist at development time, such as files, modules, and classes
Component and connector (C&C) structures: composed of dynamic elements that exist at runtime, such as processes and threads
Allocation structures: composed of both software elements (from a module or C&C structure) and non-software elements that may exist both at development time and at runtime, such as file systems, hardware, and development teams
When you instantiate a design concept, you may actually produce more than one structure. For example, in a particular iteration you may instantiate the Layers pattern, which will result in a Module structure. As part of instantiating this pattern, you will need to choose the number of layers, their relationships, and the specific responsibilities of each layer. As part of the iteration, you may also study how a scenario is supported by the elements that you have just identified. For example, you could create instances of the logical elements in a C&C structure and model how they exchange messages (see Section 3.6). Finally, you may want to decide who will be responsible for implementing the modules inside each of the layers, which is an allocation decision.
3.5.1 Instantiating Elements
The instantiation of architectural elements depends on the type of design concept that you are working with:
Reference architectures. In the case of reference architectures, instantiation typically means that you perform some sort of customization. As part of this work, you will add or remove elements that are part of the structure that is defined by the reference architecture. For example, if you are designing a web application that needs to communicate with an external application to handle payments, you will probably need an integration layer in addition to the traditional presentation, business, and data layers.
Architectural and design patterns. These patterns provide a generic structure composed of elements, their relationships and their responsibilities. As this structure is generic, you will need to adapt it to your specific problem. Instantiation usually involves transforming the generic structure defined by the pattern into a specific one that is adapted to the needs of the problem that you are solving. For example, consider the Pipe and Filters architectural pattern. It establishes the basic elements of computation—filters—and their relationships—pipes—but does not specify how many filters you should use for your problem or what their relationships should be. You will instantiate this pattern by defining how many pipes and filters are needed to solve your problem, by establishing the specific responsibilities of each of the filters, and by defining their topology.
Deployment patterns. Similar to the case with architectural and design patterns, the instantiation of deployment patterns generally involves the identification and specification of physical elements. If, for example, you are using a Load-Balanced Cluster pattern, instantiation may involve identifying the number of replicas to be included in the cluster, the load-balancing algorithm, and the physical location of the replicas.
Tactics. This design concept does not prescribe a particular structure, so you will need to use other design concepts to instantiate a tactic. For example, you may select a security tactic of authenticating actors and instantiate it by creating a custom-coded ad hoc solution, or by using a security pattern, or by using an externally developed component such as a security framework.
Externally developed components. The instantiation of these components may or may not imply the creation of new elements. For example, in the case of object-oriented frameworks, instantiation may require you to create specific classes that inherit from the base classes defined in the framework. This will result in new elements. Other approaches, which do not involve the creation of new elements, might include choosing a specific technology from a technology family that was identified in a previous iteration, associating a particular framework to elements that were identified in a previous iteration, or specifying configuration options for an element associated with a particular technology (such as a number of threads in a thread pool).
3.5.2 Associating Responsibilities and Identifying Properties
When you are creating elements by instantiating design concepts, you need to consider the responsibilities that are allocated to these elements. For example, if you instantiate the Layers pattern and decide to use the traditional three-layer structure, you might decide that one of the layers will be responsible for managing the interactions with the users (typically known as the presentation layer). When instantiating elements and allocating responsibilities, you should keep in mind the high cohesion/low coupling design principle: Elements should have high cohesion (internally), defined by a narrow set of responsibilities, and low coupling (externally), defined by a lack of knowledge of the implementation details of other elements.
One additional aspect that you need to consider when instantiating design concepts is the properties of the elements. This may involve aspects such as the configuration options, statefulness, resource management, priority, or even hardware characteristics (if the elements that you created are physical nodes) of the chosen technologies. Identifying these properties supports analysis and the documentation of the design rationale.
3.5.3 Establishing Relationships Between the Elements
The creation of structures also requires making decisions with respect to the relationships that exist between the elements and their properties. Once again, consider the Layers pattern. You may decide that two layers are connected, but these layers will eventually be allocated to components that are, in turn, allocated to hardware. In such a case, you need to decide how communication will take place between these layers, as they have been allocated to components: Is the communication synchronous or asynchronous? Does it involve some type of network communication? Which type of protocol is used? How much information is transferred and at what rate? These design decisions can have a significant impact with respect to achieving certain quality attributes such as performance.
3.6 Defining Interfaces
Interfaces are the externally visible properties of elements that establish a contractual specification that allows elements to collaborate and exchange information. There are two categories of interfaces: external and internal.
3.6.1 External Interfaces
External interfaces include interfaces from other systems that are required by the system that you are developing and interfaces that are provided by your system to other systems. Required interfaces are part of the constraints for your system, as you usually cannot influence their specification. Provided interfaces need to be formally defined, which can be performed in a similar way to defining internal interfaces—that is, by considering interactions between the external systems and your system and seeing them as elements of a bigger structure.
Establishing a system context at the beginning of the design process is useful to identify external interfaces. This context can be represented using a system context diagram, as shown in Figure 3.3. Given that external entities and the system under development interact via interfaces, there should be at least one external interface per external system (each relationship in the figure).
3.6.2 Internal Interfaces
Internal interfaces are interfaces between the elements that result from the instantiation of design concepts. To identify the relationships and the interface details, you generally need to understand how the elements exchange information at runtime. You can achieve this with the help of modeling tools such as UML sequence diagrams (Figure 3.4), which allow you to model the information that is exchanged between elements during execution to support use cases or quality attribute scenarios. This type of analysis is also useful for identifying relationships between elements: If two elements need to exchange information directly, then a relationship between these elements must exist. The information that is exchanged becomes part of the specification of the interface. Interfaces typically consist of a set of operations (such as methods) with specified parameters, return values, and possibly, exceptions and pre and post conditions. Some interfaces, however, may involve other information exchange mechanisms, such as a component that writes information to a file or database and another component that then accesses this information. Interfaces may also establish quality of service agreements. For example, the execution of an operation specified in the interface may be time-constrained to satisfy a performance quality attribute scenario.
The following is an initial sequence diagram for Use Case UC-2 (Detect Fault)1 from the FCAPS case study in Chapter 4. This diagram shows the interactions between an actor and the five components that participate in UC-2. In creating this diagram, we identify the information that is exchanged, the methods that are invoked, and the values that are passed and returned.
1. More detail about this example is presented in Chapter 4.
Key: UML
From this interaction, initial methods for the interfaces of the interacting elements can be identified:
Name: TimeServerConnector
FIGURE 3.4 A sequence diagram used to identify interfaces
The identification of interfaces is usually not performed equally across all design iterations. When you are starting the design of a greenfield system, for example, your first iterations will produce only abstract elements such as layers, with these elements then being refined in later iterations. The interfaces of abstract elements such as layers are typically underspecified. For example, in an early iteration you might simply specify that the UI layer sends “commands” to the business logic layer, with the business logic layer sending “results” back. As you advance in the design process and particularly when you create structures to address specific use cases and quality attribute scenarios, you will need to refine the interfaces of the specific elements that participate in the interaction.
In some special cases, identification of interfaces is greatly simplified. For example, in the Big Data case study we present in Chapter 5, interfaces are already defined by the technologies that are selected. The specification of interfaces then becomes a relatively trivial task, as the chosen technologies are designed to interoperate and hence have already “baked in” many interface assumptions and decisions.
Finally, you need to consider that not all of the internal interfaces of the system element will be identified as part of the design process (see the sidebar “Identifying Interfaces in Element Interaction Design”).
3.7 Creating Preliminary Documentation During Design
A software architecture is typically documented as a set of views, which represent the different structures that compose the architecture. The formal documentation of these views is not part of the design process. Structures, however, are produced as part of design. Capturing them, even in an informal manner (i.e., as sketches), along with the design decisions that led you to create these structures, is a task that should be performed as part of normal design activities.
3.7.1 Recording Sketches of the Views
When you produce structures by instantiating the design concepts that you have selected to address a particular design problem, you will typically not produce these structures in your mind, but rather will create some sketches of them. In the simplest case, you will produce these sketches on a whiteboard, a flip-chart, or even a piece of paper. Otherwise, you may use a modeling tool in which you will draw the structures. The sketches that you produce are the initial documentation for your architecture that you should capture and may flesh out later, if necessary. When you create sketches, you don’t need to always use a more formal language such as UML. If you use some informal notation, you should at least be careful in maintaining consistency in the use of symbols. Eventually, you will need to add a legend to your diagrams to provide clarity and avoid ambiguity.
You should develop discipline in writing down the responsibilities that you allocate to the elements as you create the structures. The reasons for this are simple: As you identify an element, you are determining some responsibilities for that element in your mind. Writing it down at that moment ensures that you won’t have to remember it later. Also, it is easier to write down the responsibilities associated with your elements gradually, rather than compiling all of them at a later time.
Creating this preliminary documentation as you design requires some discipline. But the benefits are worth the effort—you will be able to produce the more detailed architecture documentation relatively easily and quickly at a later point. One simple way that you can document responsibilities if you are using a whiteboard, a flip-chart, or a PowerPoint slide is to take a photo of the sketch that you have produced and paste it in a document, along with a table that summarizes the responsibilities of every element depicted in the diagram (Figure 3.5 provides an example). If you are using a computer-aided software engineering (CASE) tool, you can select an element to create and use the text area that usually appears in the properties sheet of the element to document its responsibilities, and then generate the documentation automatically.
This diagram presents a sketch of a module view depicting the overall system structure from the case study in Chapter 5.
The diagram is complemented with a table that describes the element’s responsibilities:
FIGURE 3.5 Sample preliminary documentation
Of course, it is not necessary to document everything. The three purposes of documentation are analysis, construction, and education. At the moment you are designing, you should choose a documentation purpose and then document to fulfill that purpose, based on your risk mitigation concerns. For example, if you have a critical quality attribute scenario that your architecture design needs to satisfy, and if you will need to prove this requirement is met in an analysis, then you must take care to document the information that is relevant for the analysis to be satisfactory. Alternatively, if you anticipate having to train new team members, then you should make a sketch of a C&C view of the system, showing how it operates and how the elements interact at runtime, and perhaps construct a crude module view of the system, showing at least the major layers or subsystems.
Finally, it is a good idea to remember, as you are documenting, that your design may eventually be analyzed. Consequently, you need to think about which information should be documented to support this analysis (see the sidebar “Scenario-Based Documentation”).
An analysis of an architecture design is based on your most important use cases and quality attribute scenarios. Simply put, a scenario is selected and you must explain to reviewers how the architecture supports the scenario, and justify your decisions. To start preparing for the analysis while you design, it is useful to produce and document structures that contain the elements that are involved in the satisfaction of a scenario. This should come naturally given that the design process is guided by scenarios, but keeping this point firmly in mind is always helpful.
During the design process, you should at least try to capture the following elements in a single document:
The primary presentation: the diagram that represents the structure that you produced
The element responsibilities table: it will help you record the responsibilities of the elements that are present in the structure
The relevant design decisions, and their rationales (see Section 3.7.2)
You might also capture two other pieces of information:
A runtime representation of the element’s interaction—for example, a sequence diagram
The initial interface specifications (which can also be captured in a separate document)
As you can see, all of this information needs to be produced as part of the design process. One way or another, you need to decide which elements are present in the system and how they interact. The only question is whether you bother to write this information down, or whether its sole representation is in the code.
If you follow the approach that we advocate here, at the end of the design you will have a set of preliminary views documented, in which each of the views is associated with a particular scenario, and you will have this documentation at little cost. This preliminary documentation can be used “as is” to analyze the design, and particularly through scenario-based evaluations.
3.7.2 Recording Design Decisions
In each design iteration, you make important design decisions to achieve your iteration goal. As we saw previously, these design decisions include the following:
Selecting a design concept from several alternatives
Creating structures by instantiating the selected design concept
Establishing relationships between elements and defining interfaces
Allocating resources (e.g., people, hardware, computation)
Others
When you study a diagram that represents an architecture, you see the end product of a thought process, but it may not be easy to understand the decisions that were made to achieve this result. Recording design decisions beyond the representation of the chosen elements, relationships, and properties is fundamental to help in understanding how you arrived at the result: the design rationale.
When your iteration goal involves satisfying a specific quality attribute scenario, some of the decisions that you make will play significant roles in your ability to achieve the scenario response measure. These are, therefore, the decisions that you should take the greatest care in recording. You should record these decisions because they are essential to facilitate analysis of the design you created; then to facilitate implementation; and, still later, to aid in understanding of the architecture (e.g., during maintenance). Also every design decision is “good enough” but seldom optimal, so you need to justify the decisions made, and possibly revisit the remaining risks later.
You might think that recording design decisions is a tedious task. In reality, depending on the criticality of the system being developed, you can adjust the amount of information that is recorded. For example, to record a minimum of information, you can use a simple table such as the one shown in Table 3.2. If you decide to record more than this minimum, the following information can prove useful:
What evidence was produced to justify decisions?
Who did what?
Why were shortcuts taken?
What assumptions did you make?
And, in the same way that we suggest you record responsibilities as you identify elements, you should record the design decisions as you make them. The reason for this is simple: If you leave it until later, you may not remember why you did things.
3.8 Tracking Design Progress
Even though ADD provides clear guidelines to perform design systematically, it does not provide a mechanism to track design progress. When you are performing design, however, there are several questions that you want to answer:
How much design do we need to do?
How much design has been done so far?
Are we finished?
Agile practices such as the use of backlogs and Kanban boards can help you track the design progress and answer these questions. These techniques are not limited to Agile methods, of course. Any development project using any methodology should track progress.
3.8.1 Use of an Architectural Backlog
The concept of an architecture (or design) backlog has been proposed by several authors (see Section 7.1). This is similar to what is found in Agile development methods such as Scrum. The basic idea is that you need to create a list of the pending actions that still need to be performed as part of the architecture design process.
Initially, you should populate the design backlog with your drivers, but other activities that support the design of the architecture can also be included. For example:
Creation of a prototype to test a particular technology or to address a specific quality attribute risk
Exploration and understanding of existing assets (possibly requiring reverse engineering)
Issues uncovered in a review of the design
Review of a partial design that was performed on a previous iteration
For example, when using Scrum, the sprint backlog and the design backlog are not independent: Some features in the sprint backlog may require architecture design to be performed, so they will generate items that go into the architectural design backlog. These two backlogs can be managed separately, however. The design backlog may even be managed internally, as it contains several items that are typically not discussed or prioritized by the customer (or product owner).
Also, additional architectural concerns may arise as decisions are made. For example, if you choose a reference architecture, you will probably need to add specific architectural concerns, or quality attribute scenarios derived from them, to the architectural design backlog. An example of such a concern is the management of sessions for a web application reference architecture.
3.8.2 Use of a Design Kanban Board
As design is performed in rounds and as a series of iterations within these rounds, you need to have a way of tracking the design’s degree of advancement. You must also decide whether you need to continue making more design decisions (i.e., performing additional iterations). One tool that can be used to facilitate this task is a Kanban board, such as the one shown in Figure 3.6
At the beginning of a design round, the inputs to the design process become entries in the backlog. Initially, that activity occurs in step 1 of ADD; the different entries in your backlog for this design round should be added to the “Not Yet Addressed” column of the board (except if you have some entries that were not concluded in previous design rounds that you wish to address here). When you begin a design iteration, in step 2 of ADD, the backlog entries that correspond to the drivers that you plan to address as part of the design iteration goal should be moved to the “Partially Addressed” column. Finally, once you finish an iteration and the analysis of your design decisions reveals that a particular driver has been addressed (step 7 of ADD), the entry should be moved to the “Completely Addressed” column of the board. It is important to establish clear criteria that will allow a driver to be moved to the “Completely Addressed” column (think of this as the “Definition of Addressed” criteria, similar to the “Definition of Done” criteria used in Scrum). A criterion may be, for example, that the driver has been analyzed or that it has been implemented in a prototype. Also, drivers that are selected for a particular iteration may not be completely addressed in that particular iteration, in which case they should remain in the “Partially Addressed” column and, in preparation for subsequent iterations, you should consider how they can be allocated to the elements that exist at this point.
It can be useful to select a technique that will allow you to differentiate the entries in the board according to their priority. For example, you might use different colors of Post-it notes depending on the priority.
With such a board, it is easy to visually track the advancement of design, as you can quickly see how many of the (most important) drivers are being or have been addressed in the design round. This technique also helps you decide whether you need to perform additional iterations as, ideally, the design round is terminated when a majority of your drivers (or at least the ones with the highest priority) are located under the “Completely Addressed” column.
3.9 Summary
In this chapter, we presented a detailed walk-through of the Attribute-Driven Design method, version 3.0. We also discussed several important aspects that need to be considered in the various steps of the design process. These aspects include the use of a backlog, the various possible design roadmaps (for greenfield, brownfield, and novel contexts), the identification and selection of design concepts and their use in producing structures, the definition of interfaces, and the production of preliminary documentation.
Even though the overall architecture development life cycle includes documenting and analyzing architecture as activities that are separate from design, we have argued that a clean separation of these activities is artificial and harmful. We stress that preliminary documentation and analysis activities need to be regularly performed as integral parts of the design process.
In Chapters 4, 5, and 6, we will instantiate ADD 3.0 in several extended examples, showing how the method works in the real world, in both greenfield and brownfield contexts.
3.10 Further Reading
Some of the concepts of ADD 3.0 were first introduced in an IEEE Software article: H. Cervantes, P. Velasco, and R. Kazman, “A Principled Way of Using Frameworks in Architectural Design”, IEEE Software, 46–53, March/April 2013. Version 2.0 of ADD was first documented in the SEI Technical Report: R. Wojcik, F. Bachmann, L. Bass, P. Clements, P. Merson, R. Nord, and B. Wood, “Attribute-Driven Design (ADD), Version 2.0”, SEI/CMU Technical Report CMU/SEI-2006-TR-023, 2006. An extended example of using ADD 2.0 was documented in W. Wood, “A Practical Example of Applying Attribute-Driven Design (ADD), Version 2.0”, SEI/CMU Technical Report: CMU/SEI-2007-TR-005.
Several alternative methods exist to support the design of software architectures. These are discussed in more detail and referenced in Chapter 7.
The concept of an architecture backlog is discussed in C. Hofmeister, P. Kruchten, R. Nord, H. Obbink, A. Ran, and P. America, “A General Model of Software Architecture Design Derived from Five Industrial Approaches”, Journal of Systems and Software, 80:106–126, 2007.
The ARID method is discussed in P. Clements, R. Kazman, and M. Klein, Evaluating Software Architectures: Methods and Case Studies, Addison-Wesley, 2002.
The CBAM method is presented in L. Bass, P. Clements, and R. Kazman, Software Architecture in Practice, 3rd ed., Addison-Wesley, 2013.
The ways in which an architecture can be documented are covered extensively in P. Clements et al. Documenting Software Architectures: Views and Beyond, 2nd ed., Addison-Wesley, 2011. More Agile approaches to documenting are discussed in books such as S. Brown, Software Architecture for Developers. Lean Publishing, 2015.
The importance and challenges of capturing design rationale are discussed in A. Tang, M. Ali Babar, I. Gorton, and J. Han, “A Survey of Architecture Design Rationale”, Journal of Systems and Software, 79(12):1792–1804, 2007. A minimalistic technique for capturing rationale is discussed in U. Zdun, R. Capilla, H. Tran, and O. Zimmermann, “Sustainable Architectural Design Decisions”, IEEE Software, 30(6):46–53, 2013.