Completeness is both an external and an internal analysis goal. It is external with respect to system requirements. The main goal of assessing an architecture's completeness in this context is to establish whether it adequately captures all of a system's key functional and non-functional requirements. Analyzing an architectural model for external completeness is nontrivial. Software systems for which an architecture-centric development perspective is most useful are often large, complex, long lived, and dynamic. In such settings, both the captured requirements and the modeled architecture may be very large and complex, and may be captured using a multitude of notations of various levels of rigor and formality. Furthermore, both likely will be specified incrementally and will change over time, so that the system's engineers need to carefully select points at which external completeness of the architecture can and should be assessed meaningfully.

Analyzing an architecture for internal completeness establishes whether all of the system's elements have been fully captured, both with respect to the modeling notation and with respect to the system undergoing architectural design.

Establishing the completeness of the model with respect to the modeling notation (recall Chapter 6) ensures that the model includes all the information demanded by the notation's syntactic and semantic rules. For example, the architectural model depicted in Figure 8-2 is captured in the Rapide architecture description language and thus must adhere to Rapide's syntax and semantics. Given this choice of modeling notation, the component instances in the architecture portion of the model must be attached to one another (see the connect statement Figure 8-2) according to the rules of Rapide: A component's out action must be connected to another component's in action; furthermore, connectors are not declared explicitly in Rapide.

Note, however, that fulfilling the modeling requirements of a language such as Rapide does not ensure that the architecture is, in fact, captured completely. Rapide is agnostic as to whether the architect accidentally omitted a major system component or whether a specified component's interface is missing critical services. Establishing the completeness of the architectural model with respect to the system being designed requires checking—often manually, as will be further discussed later in the chapter—whether there are missing components and connectors in the architecture; whether the specified components' and connectors' interfaces and protocols of interaction are fully specified; whether all of the dependencies and interaction paths are captured by the system's architectural configuration; and so on.

In principle, internal completeness is easier to assess than external completeness, and is amenable to automation. A number of software architecture analysis techniques have focused on this very analysis category, as will be elaborated later in this chapter.

Consistency is an internal property of an architectural model, which is intended to ensure that different elements of that model do not contradict one another. The need for consistency derives from the fact that software systems, and thus their architectural models, are complex and multifaceted. As a result, even if no architectural design decisions are invalidated during the architectural design process, capturing the details of those decisions during architecture modeling may result in many inadvertently introduced inconsistencies. Examples of inconsistencies in a model include the following.

ReqInt: getSubQ(Natural first, Natural last, Boolean remove)
        returns FIFOQueue;

ProvInt1: getSubQ(Index first, Index last)
          returns FIFOQueue;

ProvInt2: getSubQ(Natural first, Natural last, Boolean remove)
          returns Queue;

subtract(Integer x, Integer y) returns Integer;

precondition q.size ≥ 0;
postcondition ~q.size=q.size;

precondition q.size ≥ 1;
postcondition ~q.size=q.size - 1;

Interaction Inconsistency

Interaction inconsistencies can occur even if two components' respective provided and required operations have consistent names, interfaces, and behaviors. An interaction inconsistency occurs when a component's provided operations are accessed in a manner that violates certain interaction constraints, such as the order in which the component's operations are to be accessed. Such constraints comprise the component's interaction protocol.

Figure 8-3. Interaction protocol for the QueueServer component. Transitions corresponding to operations such as front and is_empty, typically provided by a queue component, have been elided for clarity. Transition guards are enclosed within brackets. The assumption is that the queue can contain at least two elements.

A component's interaction protocol can be specified using different notations. Frequently, it is modeled via state-transition diagrams (Yellin and Strom 1994). Analyzing the system for interaction consistency in that case consists of ensuring that a given sequence of operation requests matches some sequence of legal state transitions specified by each component's protocol.

An example of such interaction protocol is provided in Figure 8-3. In the figure, the QueueServer component requires that at least one element always be enqueued before an attempt to dequeue an element can be made; furthermore, it assumes that no attempts to enqueue elements onto a full queue will be made. A QueueClient component that does not adhere to these constraints—that is, whose sequence of invocations cannot be executed by the state machine from Figure 8-3—will cause an interaction inconsistency with the QueueServer.

Compatibility is an external property of an architectural model, intended to ensure that the model adheres to the design guidelines and constraints imposed by an architectural style, a reference architecture, or an architectural standard. If the design constraints are captured formally, or at least rigorously, ensuring an architecture's compatibility to them will be relatively straightforward. If an architecture must be compatible with a set of semiformally or informally specified design guidelines, analyzing the architecture for compatibility may be more challenging and the outcome of the analysis process also may be ambiguous at times.

Reference architectures are usually specified formally, using an architecture description language. Therefore, establishing the compatibility of a given system's architecture to the reference architecture may be a precise and automatable process. Since a reference architecture captures a set of properties that must hold true across any number of systems in a given domain, it may be only partially specified or certain parts of it may be at a very high level of abstraction. In such cases, establishing that a product-specific architecture adheres to the reference architecture can be accomplished by ensuring refinement consistency, in the manner discussed above.

However, as the reader will recall from Chapter 4 and will see in later chapters, most architectural styles and many standards provide general, high-level design guidelines, so that establishing an architecture's adherence to them may be more challenging. The difficulty may arise from fuzziness on the part of the style definition or imprecision or incompleteness on the part of the architectural model. For example, Figure 8-5 depicts the Lunar Lander architecture according to the event-based style. This diagram was first shown in Chapter 4, with a slightly different component layout, as were the relatively similar diagrams for the Lunar Lander architectures in the C2 (Figure 4-23) and blackboard (Figure 4-16) styles. Determining the style to which this architecture adheres is a nontrivial task: The depicted configuration of components may, in fact, also adhere to C2's principles, such as substrate independence. Likewise, this particular visual layout of the architecture's topology may be misleading, for the Spacecraft component may in fact play the role of a blackboard in this system. In cases such as this, the architect may need to obtain additional information, rely on tacit knowledge, and use one or more architectural analysis techniques presented later in this chapter.

Figure 8-5. A depiction of the Lunar Lander architecture according to the event-based style.

Correctness is an external property of an architectural model. A system's architecture is said to be correct with respect to an external system specification if the architectural design decisions fully realize those specifications. Furthermore, the system's implementation is correct with respect to the system's architecture if the implementation fully captures and realizes all the principal design decisions comprising the architecture. Correctness is therefore relative : It is the result of architecture to some other artifact, where the artifact is either intended to elaborate and fulfill the architecture or the architecture is intended to elaborate and fulfill the artifact.

An interesting observation arises in the implementation of many large, modern software systems that reuse off-the-shelf functionality, or off-the-shelf architectural solutions such as reference architectures. In such cases, it is very likely that, through the off-the-shelf components, the implemented system will include structural elements or functionality, as well as non-functional properties, that are not specified in the requirements and/or not modeled in the architecture. With a 1970's notion of refinement-based correctness, such systems would not be considered to be correct with respect to the architecture. However, with the notion of correctness built upon fulfillment, as described above, such systems are not only correct, but they are likely efficiently created and form a suitable basis for efficiently fulfilling new system requirements.

In a large system, individual components and connectors may not always be as interesting to an architect as their compositions, but both components and connectors need to provide specific services at specific quality levels. In the case of components, this typically means application-specific functionality; in the case of connectors, this means application-independent interaction services.

The simplest type of component- and connector-level analysis ensures that the given component or connector provides the services expected of it. For example, a component's or connector's interface can be inspected to make sure that no expected services are missing. As an illustration, Figure 8-6 shows Lunar Lander's Data Store component modeled in xADLite. It is trivial to analyze this component, either manually or automatically, to establish that it provides both getValues and storeValues services; furthermore, if a system stakeholder expects or requires the Data Store component to provide further services, it can easily be ascertained that no such services currently exist. Even if the component were much larger and provided many more services, this task would not be significantly harder.

Of course, "checking off" the services a component or connector provides does not ensure that those services are modeled correctly. The described analysis can be thought of as equivalent to establishing only name consistency. A component or connector may provide services with the expected names, but with incorrect interfaces. For example, getValues in the above example may be modeled to expect the values in a wrong format, such as untyped versus typed or string versus integer. Therefore, it is not sufficient to establish that the component or connector provides appropriately named services; the complete interface of the component or connector has to be analyzed. The information provided in Figure 8-6 will thus have to be supplemented with additional details, possibly from other models.

Figure 8-6. Lunar Lander's Data Store component modeled in xADLite. This model is extracted from that provided in Chapter 6, in Figure 6-27.

Figure 8-7. Pipe connector modeled in Wright.

Taking this argument a step further, it is not sufficient to ensure that the component's or connector's services are exported via an appropriate interface. The semantics of those services as modeled (and, eventually, as implemented) may be different from the desired semantics. Thus, for example, the getValues service from Figure 8-6 may not be modeled such that it accesses the Data Store to obtain the needed values, but instead may request those values from a system user. Since the intended usage of the component, implied in its name, is to access a repository, this implementation of getValues, while legitimate in principle, would be wrong for this context.

Similarly, a connector may provide interaction services with semantics that are different from the expected semantics. For example, a connector may be expected to support asynchronous invocation semantics, but is actually modeled for synchronous invocation. Establishing this type of semantic consistency is not trivial. Consider as an illustration a model of a Pipe connector in the Wright ADL discussed in Chapter 6; the model is depicted in Figure 8-7. The Pipe connector plays two roles; that is, it provides two services: Reader and Writer. Their interplay, that is, the connector's overall behavior, is captured by the connector's glue. Even though a pipe is a relatively simple connector, establishing manually that it adheres to its expected behavior is significantly more challenging than with name or interface conformance. For example, is the intended semantics to allow unimpeded writing to the pipe, as in the current specification of the Writer role, or does the pipe have a bounded buffer so that writing a certain amount of data would require that at least some of that data be read before additional data can be written? Assessing more complex connectors for these types interaction properties will be much more difficult and may be especially error-prone.

Even if individual components and connectors have desired properties, no guarantees can be made that their compositions in a given system will behave as expected, or even be legal. The interplay among complex components can itself be very complex. Architects may assess the properties of their compositions at the level of the entire system or incrementally, by focusing on particular subsystems.

The most manageable increment is pair-wise conformance, where only two interacting components are considered at a time, and name, interface, behavior, and interaction conformance are established as discussed above. The next step up is to take a set of components possibly interacting through a single connector, such as those shown in Figure 8-5. Most analysis techniques can easily support both these cases. Ensuring desirable properties at the level of large subsystems and entire systems can be quite challenging, and many analysis techniques have tried to get a handle on this problem.

In certain scenarios it may be obvious that compositions of two or more components that respectively possess properties α, β, γ, and so forth will have some combination of those properties. For example, combining a data encryption component—intended to provide communication security—with a data compression component—intended to provide communication efficiency—can be expected to provide both security and efficiency.

Much more frequent in practice is the situation where the interplay among the components will result in their interference and either enhancement or diminishment of each other's properties. This will be acceptable, and even desirable, in certain cases. For example, a component that provides a critical service very efficiently may be vulnerable to malicious attacks. The system architects may decide that sacrificing some of the efficiency is acceptable in order to enhance the system's security, and may compose this component with one or more components that provide services such as encryption, authentication, or authorization. Likewise, in many real-time systems, a component that computes its results or provides data more quickly and/or frequently than the system requires will be composed with a (simple) component that introduces the necessary delays. Such compositions will be synergistic: The system will ultimately be greater than the sum of its parts.

Such interference among system components will not be desirable in all situations. More importantly, such interference will often not be as obvious as in the above scenarios and will result in unintended composite properties. Examples abound. One such well-known example from software engineering literature involved integrating two components that both assumed that they owned the system's main thread of control, ultimately resulting in an unusable system (Garlan, Allen, and Ockerbloom 1995). Another similar example involved integrating concurrent components implemented in two different programming languages. The resulting system's performance was unacceptable, and a long and painstaking analysis of the system uncovered that the individual components' threading models were incompatible (Maybee, Heimbinger, and Osterweil 1996). The reader can easily envision many other such scenarios, such as when a system comprises interacting memory-efficient components (which use computationally intensive data compression algorithms) and CPU-efficient components (which use techniques such as data caching and prefetching) or, further, when a component introducing a fault-tolerance service (for example, via continuous state replication) is introduced into such a system.

In many large, distributed software systems large amounts of data are processed, exchanged, and stored. Examples of data-intensive systems are numerous and appear in such wide ranging domains as scientific computing, many Web-based applications, e-commerce, and multimedia. In such systems, in addition to the properties of the individual structural architectural elements—components and connectors—and the entire architectural configuration, it is important to ensure that the system's data is properly modeled, implemented, and exchanged among the structural elements. This involves assessing the data elements, including the following.

As a simple example, consider a system consisting of a data-producer component and two data-consumer components, whose architectural configuration is depicted in Figure 8-8. The figure also shows the respective frequencies at which they are able to exchange data. The Producer component sends one megabit per second, and Consumer 1 is able to receive and process that data in a timely manner. In fact, Consumer 1 may wait idly up to 50 percent of the time to receive additional data from the Producer. However, Consumer 2 is able to receive and process the data at a rate that is only one-half of the production rate. This means that Consumer 2 may lose up to one-half of the produced data.

Figure 8-8. A simple system with one data producer and two consumers, with the frequencies at which they are able to send and receive data, respectively.

This problem may be mitigated if the Multicast Connector servicing the three components is able to buffer data, but clearly, if the system is long running, the connector's buffer will eventually overflow. The connector will also have to include additional processing logic to store the data temporarily and also route it to the two components in the order received, which will introduce additional overhead in the system. If the two Consumer components require the data to be in different formats from that generated by the Producer, the connector will also have to act as an adapter, introducing further overhead. Likewise, the connector may have to perform additional tasks based on the architectural requirements, such as encrypting the data for security, ensuring that the data is delivered according to the specified routing policy (for example, best-effort or exactly-once), or compressing the data if necessary.

During the architectural design process, architects frequently address the critical system requirements first, and then both introduce additional elements into the architecture and refine the architecture to include additional details that are necessary for the architecture's realization into the final system. This may involve the addition and further breakdown of architectural elements, as well as the introduction and refinement of existing design decisions.

Consider a simple example. A high-level architectural breakdown of a system may look like the diagram shown in Figure 8-9. For simplicity, the connectors are depicted only as arrows indicating the interacting components as well as the direction of interaction. For example, component C1 initiates the interactions with components C3 and C4. As is the case with all boxes-and-arrows architectural descriptions, many details of this architecture are missing, including the components' interfaces, details of their behaviors and of the data exchanged, as well as the semantics of their interactions, such as that involving components C1, C2, and C3. While important, such information is not critical to this discussion.

Figure 8-9. A high-level architectural configuration.

Figure 8-10. A refined, more detailed architectural configuration.

Figure 8-10 depicts another architectural configuration, which is intended to be a refinement of that shown in Figure 8-9. For convenience, groups of components in the refined architecture which are presumed to comprise the original components are highlighted. For example, components C11, C12, C13, and C14 along with their interconnections in Figure 8-10 comprise the component C1 from Figure 8-9. Furthermore, the connectors are refined, such that it is clear that only C1's subcomponent C14 is engaged in interactions with the subcomponents of C3 and C4.

In the process of refinement, several problems can be introduced, as discussed above in the context of refinement incompatibility. The easiest cases to address are those where an architectural design decision has been clearly invalidated. However, even the addition or modification of architectural decisions may cause a refinement inconsistency depending on the context and on the range of architectural changes that are allowed. For example, the architecture depicted in Figure 8-9 and Figure 8-10 may need to abide by an architectural constraint stating that interactions crossing the boundaries of the system's original four components must have a single target and a single destination. In that case, elaborating the interactions among the architecture's components as shown in Figure 8-10 would violate the constraints, both in the case of component C1's interactions with C3 and C4's interactions with C2.

Clearly, based on the scant information provided about the architectures in Figure 8-9, it is unclear whether the changes to the original architecture depicted in Figure 8-10 should be allowed. This will depend on many factors, and some, though not all, will be apparent from a more detailed and rigorous modeling of the architecture such as discussed in Chapter 6. This will also depend on a refinement policy, which may be explicitly stated. Thus, for example, Mark Moriconi and colleagues define a refinement policy called conservative extension (Moriconi, Qian, and Riemenschneider 1995). Conservative extension essentially precludes an architect from introducing any new features into a system's architecture, and instead restricts him or her to either elaborating or possibly eliminating those features already existing in the more abstract, that is, higher-level, architecture. While a property such as conservative extension may not always be useful in practice, Moriconi and colleagues have shown it to be amenable to formal specification and analysis. Other refinement policies may be much less restrictive, but that very flexibility may make it difficult to establish that they are being adhered to in a concrete situation.

In certain situations it is important for architects to understand the relationship between the architecture they are interested in and a baseline architecture with known properties. One such case, discussed previously in this chapter, is ensuring the compliance of a given system's architecture with a reference architecture. Other cases may include ensuring the architecture's compliance with the design guidelines captured in an architectural style or with a particular architectural pattern. It is, of course, also possible that the baseline architecture is simply one that the architect knows and understands. The architect may have encountered the architecture in literature, it may be the architecture of a related product he or she and colleagues had developed previously, or it could even be an architecture that has been recovered from a system (possibly even competitor's system) that exhibits desired properties.

Such comparisons of two or more architectures involve comparing the processing and data storage capabilities provided by the components, the interactions as embodied in the connectors, the characteristics of the data exchange, the components' and connectors' compositions into the system's configuration, and the sources of the non-functional properties exhibited by the system. It is possible for the architectures in question to be at different levels of abstraction, in which case the techniques discussed in the previous subsection can be employed.

Software inspections and reviews are widely used code-analysis techniques. If unfamiliar with these techniques, the reader is encouraged to consult an introductory software engineering text. Architectural inspections and reviews are conducted by different stakeholders to ensure a variety of properties in an architecture. They involve a set of activities conducted by system stakeholders in which different architectural models are studied for specific properties. These activities often take place in architecture review boards, where several stakeholders define the objective of the analysis—such as ensuring that the architecture satisfies a given non-functional property—and then, as a group, carefully study and critique the architecture or some of its parts.

Figure 8-11. Architectural analysis dimensions discussed in this chapter.

Inspections and reviews are manual analysis techniques, and as such can be expensive. On the other hand, they have the advantage of being useful in the case of informal or partial architectural descriptions. They can also be employed effectively in the case of "soft"' architectural properties, such as scalability or adaptability, which are not precisely understood and amenable to formal definition. Another advantage of inspections and reviews is that they can simultaneously take into account the objectives of multiple system stakeholders and consider multiple desired architectural properties.

Depending on the context, inspections and reviews can have any of the four architectural analysis goals : consistency, correctness, completeness, and compatibility. In terms of consistency, they will typically be well suited to name and interface consistency analysis. Behavior, interaction, and refinement consistency analysis may be conducted by the technical stakeholders—architects and developers—although doing so manually may be a difficult and error-prone task. For example, recall the interaction characteristics embodied in the single Wright Pipe connector from Figure 8-7, and even the comparatively simpler single state-transition interaction protocol of the QueueServer component from Figure 8-3. Dealing with a large number of such models manually would cognitively overload even the most capable architects. Thus, if analyzing for any of the three latter types of consistency is undertaken during an architectural inspection or review, it may be advisable to restrict the analysis to carefully confined subsets of the architecture.

The scope of inspections and reviews can vary. The stakeholders may be interested in individual components and connectors, or their compositions in a specific subsystem or the entire system. The stakeholders may also center on the data exchanged among the specific components and connectors or, globally, across the entire architecture. They may try to assess the compliance of the architecture to a higher-level architecture that served as its starting point. They may also try to assess the architecture's similarity to an existing architecture with known properties.

Similarly, the specific concern of the analysis can vary. The stakeholders may focus on the structural, behavioral, or interaction properties, although as mentioned above, the latter two may be difficult to assess manually. Inspections and reviews may be particularly well suited to establishing certain non-functional properties, especially those that require some interpretation and consensus reaching by the human stakeholders.

In terms of the types of models particularly amenable to inspections and reviews, any level of formality may be suitable in principle. However, highly formal models will not be useful to the nontechnical stakeholders, and even the technical stakeholders may find them difficult to read and understand. At the other end of the spectrum, informal models may be useful if, for example, the objective of the inspection is to develop a common understanding of the architecture's general characteristics. On the other hand, it may not be very meaningful to rely on informal models when inspecting the architecture for concrete properties of interest.

By their nature, the types of analysis for which inspections and reviews are geared are static and scenario-based. Since the stakeholders manually assess the architectural models, they have to focus on the architecture's static properties, such as proper connectivity, interface conformance between interacting components, adherence to desired architectural patterns, and so on. Furthermore, as discussed below in the case of the ATAM analysis technique, the stakeholders may manually run through some critical scenarios to ensure that the architecture will behave as expected.

As already mentioned, in terms of automation level inspections and reviews are manual and very human intensive.

Finally, all system stakeholders, save for perhaps component vendors, may participate in inspections and reviews. Architects and developers will conduct inspections and reviews most frequently, and will periodically be joined by project managers and possibly by customers.

Figure 8-12. The high-level view of the process steps followed in ATAM. The diagram has been adopted from Carnegie Melon University's Software Engineering Institute.

Model-based architectural analysis techniques rely solely on a system's architectural description and manipulate that description to discover properties of the architecture. Model-based techniques involve analysis tools of different levels of sophistication. These tools are frequently guided by architects, who may have to interpret the intermediate analysis results and guide the tool in further analysis.

Because of their tool-driven nature, model-based techniques are much less human intensive, and hence usually less costly, than inspections and reviews. On the other hand, they can only be used to establish "hard" properties of a system's architecture, that is, those properties that can be encoded in the architectural model. They cannot easily account for implicit properties—those that a human might readily infer from the existing information and thus chooses not to model explicitly. Additionally, model-driven analysis techniques cannot typically assess "soft," but very important, aspects of an architecture, such as design intent and rationale. Model-based techniques usually also focus on a single, specific facet of a system's architecture, such as syntactic correctness, deadlock freedom, adherence to a given style, and so forth.

Another concern with model-driven analysis techniques is their scalability: More sophisticated techniques are required to keep track of a very large number of the modeled system's elements and properties. The usual trade-off encountered in many static analysis tools, of which model-based architectural analysis tools are an instance, is between scalability on the one hand and precision or confidence on the other. In other words, architects may be able to arrive at highly precise analysis results with a high degree of confidence in those results for smaller systems, but would have to sacrifice that precision and confidence for larger systems. Because of all these reasons, the results of model-driven analyses are usually partial, and multiple such techniques are used in tandem in any given architecture-driven software development project. Even then, model-based analysis usually does not provide all the needed answers to an architect, and is coupled with techniques from the other two categories—inspections and reviews and simulation-based analysis.

The goals of model-based architectural analysis techniques are usually consistency, compatibility, and internal completeness. The goals can also include certain aspects of external completeness and correctness. Recall that both external completeness and correctness assess the adherence of a system's architectural model to the requirements, and of the system's implementation to the architectural model. Certain facets of external completeness and correctness, such as structural completeness or correctness, can be established from the architectural model. Furthermore, the output of the analysis technique may be automatic generation of (partial) system implementation from the architectural model, which would ensure external completeness and correctness by construction. If the analysis involves system requirements, then to some extent the requirements will need to be formalized.

The scope of model-based architectural analysis can span individual components and connectors, their compositions in a specific subsystem or the entire system, as well as the data exchanged among the specific components and connectors or, globally, across the entire architecture. Model-based techniques may also try to assess the compliance of the model to a higher-level model from which it has been derived. Analysis techniques in this category may also try to assess the architecture's similarity to an existing architecture with known properties. This presumes that both architectures are modeled in one of the notations supported by the analysis technique.

Similarly, the specific concern of model-based analysis can vary. The techniques may focus on the structural, behavioral, or interaction properties. Behavioral and interaction properties may be difficult to assess completely using model-based techniques alone, and these techniques are usually coupled with simulation-based approaches. Model-based techniques can be used to analyze an architecture's non-functional properties, but usually require the use of specific formalisms.

In terms of the types of models particularly amenable to this type of analysis, the general rule of thumb is that more formality yields more meaningful and precise results. Thus, for example, a Rapide specification, such as that shown in Figure 8-2, will be much more amenable to manipulation by an analysis tool than an informal diagram such as that shown in Figure 8-1. Usually the architectural models to which sophisticated analysis tools are applied have formally specified syntax as well as semantics.

By their nature, the type of analysis for which model-based techniques are well suited is static analysis. These techniques are well suited to assessing properties such as proper connectivity, type conformance, definition-use analysis of architectural services, interface and behavioral conformance between interacting components, structural adherence to desired architectural patterns, deadlock freedom, and so on.

As already mentioned, with regard to automation level model-based techniques are typically at least partially automated, and are often fully automated, requiring no human intervention.

Finally, model-driven architecture analysis techniques are usually targeted at the technical stakeholders, namely architects and developers. Other stakeholders may be interested in summaries of analysis results, but typically these techniques require a certain degree of mastery and provide low-level insights into the architecture.

The remainder of this section will survey the spectrum of model-based analysis techniques developed for different architecture description languages and discuss how architectural models can be leveraged to enable analysis of a representative non-functional property—reliability.

Simulation requires producing a dynamic, executable model of a given system, or of a part of the system that is of particular interest, possibly from a source model that is otherwise not executable. For example, the model of QueueServer component's behavior from Section 8.1, specified in terms of its operations' pre-and post-conditions, is not executable. On the other hand, its interaction protocol from Figure 8-3 can be simulated by selecting a possible sequence of component invocations, which can also be referred to as a sequence of system events. This event sequence would be used to execute QueueServer's interaction protocol state machine. For instance, if we assume that the Empty state is the start state in the state machine, the following will be valid sequences of events

<enqueue, enqueue>
<enqueue, dequeue, enqueue>
<enqueue, enqueue, dequeue, enqueue, dequeue>

while the single-event sequence

<dequeue>

is invalid, as is any sequence that starts with the dequeue event or has more dequeue than enqueue events. Note that the validity of a sequence such as

<enqueue, enqueue, enqueue>

cannot be established without introducing additional information about the system, namely, the size of the queue.

Simulation need not produce identical results to the system's execution: The source model, such as an architectural model, may elide many details of the system. For example, the QueueServer component's model says nothing about the frequency with which the component will be invoked, the sizes of elements that will be placed on the queue, the processing time required to store or access and return the queue elements, and so on. Because of this, the output of simulation might be observed only for event sequences, general trends, or ranges of values rather than specific results. Note that a system's implementation can be thought of as a very faithful executable model. Likewise, running that implementation model can be thought of as a highly precise simulation.

Clearly not all architectural models will be amenable to simulation—recall as a simple example the informal model from Figure 8-1. Even those architectural models that are amenable to simulation may need to be augmented with an external formalism in order to enable their execution. For example, models of event-based architectures may not provide event generation, processing, and response frequencies. To include that information, the architectural model may be mapped, for example, to a discrete event system simulation formalism or a queueing network, with possible ranges of such frequencies specified, and then simulated.

Of course, every time additional information such as event frequency is introduced, architects run the risk of injecting imprecision into the architectural model, and hence into the analysis results. On the other hand, since simulation is tool supported, architects can supply many different values for the model parameters, representing different possible system usage scenarios. They would thereby obtain more precise results than would be possible with one-size-fits-all model-based analysis, and do so much more easily and inexpensively than via inspections and reviews.

Figure 8-13. The conceptual relationship between architectural and simulation models.

Several software simulation platforms exist, and different ADLs discussed in Chapter 6 and in the previous section have made use of them. The conceptual relationship between ADLs and simulation models and platforms is depicted in Figure 8-13: Certain architectural models (such as UML's state-transition–based models or Rapide's partially ordered event-based models) can be simulated directly; other models may require a mapping to the simulation substrate. The required mapping from an architectural model to a simulation model may be partial or complete and of various degrees of complexity, depending on the semantic closeness between the ADL and the simulation platform. In turn, the simulation platform can be executed on top of a specific run time platform.

The goals of simulation-based architectural analysis techniques can be any of the four C s: completeness, consistency, compatibility, and correctness. However, similarly to software testing, because of the nature of simulation these properties can be established only with a limited degree of confidence and possibly only for a particular subsystem or system property.

The scope of simulation-based architectural analysis usually is the entire system or a particular subsystem, as well as the dataflow in the system. However, it is possible to isolate individual components and connectors and simulate their behaviors as well. Simulation-based techniques can be particularly effective in assessing the compliance of the given architectural model to a higher-level model from which it has been derived, as well as the model's similarity to an existing architecture with known properties. Because such models, and thus the results of their simulations, can be expected to differ, additional know-how may be required to determine the exact relationship between the architectural models under consideration.

Since simulation-based analysis techniques allow "running" an architecture and observing the outcome, the concern of simulation-based analysis spans behavior, interaction, and non-functional properties. For example, the architect can target a set of components in the architecture to ensure that they will be able to interact in the desired manner. Again, the degree of architects' confidence in the analysis results will vary depending on the amount of detail provided in the architectural model. An architect's confidence in the simulation results will also depend upon his or her level of confidence that the selected operational scenarios (that is, system inputs) will match the implemented system's eventual usage. As a very simple example, consider the QueueServer component from Figure 8-3: If the architect can be certain that the first invocation of the component in its actual execution environment will never be to dequeue an element, he can trust the results of simulations even if they do not start with a dequeue event.

Simulation-based analysis demands that the architectural models be formal. Informal models cannot be mapped to the necessary simulation substrate unless the system's architects make many impromptu, possibly arbitrary and incorrect, design decisions in the course of the mapping.

The types of analysis for which simulation-based techniques are well suited are dynamic and, in particular, scenario-based analysis. Simulation-based techniques are well suited to assessing the system's run time behavior, interaction characteristics, and non-functional qualities. If known system usage scenarios are postulated, the architects no longer need to be concerned with the completeness of the analysis results and whether the results will be representative of the system's actual use.

As already discussed, with regard to automation level simulation-based techniques are typically fully automated, requiring no human intervention other than supplying system inputs. However, the process of mapping an architectural model to a simulation model may require significant human involvement. This, in turn, can render the process error-prone.

Finally, simulation-driven architecture analysis techniques can be useful to all system stakeholders. Setting up and running simulations may require a high degree of technical expertise and familiarity with the architectural model, the architecture modeling notation, as well as the simulation substrate.

XTEAM

The eXtensible Tool-chain for Evaluation of Architectural Models (XTEAM) is a model-driven architectural description and simulation environment for mobile and resource-constrained software systems (Edwards, Malek, and Medvidović 2007). XTEAM composes two existing, general-purpose ADLs, enhancing them to capture important features of mobile systems. It implements model generators that take advantage of the ADL extensions to produce simulation models.

For modeling architectural concerns, XTEAM builds an architectural meta-model. This meta-model leverages xADL's Core, which, as the reader will recall from Chapter 6, defines architectural structures and types common to most other ADLs. This meta-model is augmented with finite state processes, or FSP, which allows the specification of component behaviors (Magee and Kramer 2006). The combination of xADL and FSP allows the creation of architectural representations that can be simulated. XTEAM masks many of the details of the two notations via a visual language depicted in Figure 8-14.

Figure 8-14. An example XTEAM model of a system comprising three hardware hosts and eight software components. The behavior of one of the components has been highlighted. This behavior represents XTEAM's syntactic overlay on top of FSP. XTEAM allows behaviors to be abstracted in the form of the ProcessReference construct.

Models conformant to the composed ADL contain sufficient information to implement a semantic mapping into low-level simulation constructs, which can be executed by an off-the-shelf discrete event simulation engine, such as adevs (Nutaro). This semantic mapping is implemented by XTEAM's model interpreter framework. When invoked by an architect, the XTEAM interpreter framework traverses the architectural model, building up a discrete event simulation model in the process. The interpreter framework maps components and connectors to discrete event constructs, such as atomic models and static digraphs. The FSP-based behavioral specifications encoded in XTEAM, such as the model shown in the bottom diagram of Figure 8-14, are translated into the state-transition functions employed by the discrete event simulation engine. The interpreter framework also creates discrete event entities that represent various system resources, such as threads. Hook methods provided by the framework allow an architect to generate the code needed to realize a wide variety of dynamic analyses.

XTEAM implements simulation-based analysis capabilities for end-to-end latency, memory utilization, component reliability, and energy consumption. Each analysis type requires the implementation of the needed XTEAM ADL extensions and a model interpreter. For example, XTEAM implements the component reliability modeling extension and analysis technique using the HMM-based approach discussed in Section 8.8.2. This reliability estimation approach relies on the definition of component failure types, the probabilities of those failures at different times during component execution, and the probability of, and time required for, recovery from failure. In order to support this type of analysis, XTEAM's FSP-based behavior language had to be extended to include failure and recovery events and probabilities. XTEAM was then augmented with an analysis technique that determines if and when failures occur as the components in the system progress through different tasks and states.

Figure 8-15. End-to-end latency, energy consumption, and reliability profiles of different elements of the architecture from Figure 8-14, obtained from adevs simulations. The simulations were generated automatically from the XTEAM model.

Another analysis supported by XTEAM is that of the modeled mobile system's energy consumption rate. The selected energy consumption model by Seo (Seo, Malek, and Medvidović 2007) was implemented as another XTEAM ADL extension and model interpreter, which incorporated the necessary elements of the energy cost model. Then, another adevs simulation was generated by XTEAM to show the system's likely energy consumption over time, under certain usage assumptions. The results of three different simulations for the architecture depicted in Figure 8-14 are shown in Figure 8-15.

XTEAM: Analysis Summary

Goals	Consistency
	Compatibility
	Correctness
Scope	Component- and connector-level
	Subsystem- and system-level
	Data exchange
Concern	Structural
	Behavioral
	Interaction
	Non-functional
Models	Formal
Type	Dynamic
	Scenario-based
Automation Level	Automated
Stakeholders	Architects
	Developers
	Managers
	Customers
	Vendors