The Technical Integration Architecture represents the enterprise building codes for all integration projects. It is the specification that all projects will reference when choosing integration technology for their particular implementation. The architecture includes guidance and design restrictions on how applications should be developed.
Therefore, the specification must be both thorough to define all aspects of the integration architecture, and easily accessible, so that the information can be easily found and understood. While in many cases detailed descriptions are necessary and appropriate, we also recommend the use of summary charts and tables for presenting information. Each of the solution architectures presented in Part III of this book is based on this architecture specification, and is a subset of this specification. Creating an Integration Architecture Specification will guide many IT implementation solutions to ensure interoperability and leverage reuse. For example, the State of Florida has created a set of guidelines for their integration architecture, described in Case Study 6.1 (State of Florida State Technology Office 2003).
The Technical Integration Architecture should be driven by business requirements. Over time, a large organization may need one of everything. While current business needs should drive infrastructure requirements and implementations, architecture decisions should take future requirements and adaptability into account. It should define the following:
Common, reusable business-domain services that can support multiple applications
Common, standardized technical services that can support any style of integration such as service, information, or process oriented
Service levels that must be supported
A comprehensive security framework based on a clearly articulated enterprise-wide security policy
Focus on the ability to leverage existing (legacy) information systems and commercial packaged system products to provide a significant portion of application functionality
In some cases, the technical architecture effort will focus on reducing the number of redundant technologies. The Current Integration Architecture Assessment (Chapter 5) provides a great deal of information that will drive architecture decisions.
As stated above, the Technical Integration Architecture provides the building codes for the integration infrastructure. Project-level adherence to this architecture ensures that there is consistency, reusability, and economic benefit to the organization for investments in integration technology. This adherence can be accomplished through design reviews, as explained in the Architecture Governance section of Chapter 4. See Appendix D for the complete specification template.
This specification represents the enterprise technical integration architecture specification. This document will be used to guide all decisions and designs related to integration in the organization. It defines the scope of the integration architecture, preferred vendors and technologies, and enterprise standards. When creating the introduction, outline all enterprise-wide decisions readers of the document should be aware of, and call attention to appendixes, such as references and governance rules.
Define the scope of the integration architecture. It should address whether it is enterprise-wide or limited to a certain organization or class of applications. Other areas to address include types of integration (data, application, or process), any limitations and reasons for the limitations. The scope must also describe what types of external applications are covered, including whether an application outside the scope of the enterprise is a candidate for connecting to enterprise applications. This will be the case if the organization has any supply chain or e-commerce initiatives planned.
Define the audience and major stakeholders. The audience should include all members of the IT organization; however, it should explicitly list specific roles or titles that are to apply the integration in the normal execution of their jobs. The major stakeholders should include the IT executives and those responsible for maintaining the document.
This section relies on the business requirements captured in Chapter 2 as well as the current integration assessment. The Integration Architecture Requirements section includes requirements for the types of integration services and technologies that will be part of the infrastructure and it defines what services should be utilized for different types of applications, the applications that need to be integrated with each other, and the types or styles of integration that will be used across the enterprise.
The organization needs to begin this specification by identifying the types of the integration that need to be supported (see Figure 6-1). The data from the business strategy and requirements gathered in Chapter 2 and Chapter 3 along with the current assessment described in Chapter 5 guides this activity. It helps to define known requirements for this type of integration to determine scope of investment. For example, if there are a number of applications that require process integration, then the organization should consider an enterprise license for a BPM solution.
As previously noted, the integration architecture is comprised of a number of different integration services, and these services can be implemented with different technologies. Rather than letting product selection drive architecture, the architecture should be based upon a framework that encompasses all aspects of integration necessary for that organization. The architecture is then constructed using existing or new products. Furthermore, the architecture is constructed on the principles of the organization and not of the products selected. For example, companies embarking on the SOA path can define their architecture as a series of services. Figure 6-2 depicts the different types of integration services, and the technologies that can be used to implement the service. As noted below, there may be a number of technologies used to implement a service because different technologies are suitable for different types of applications. Different technologies implementing the same service doesn't always mean redundancy if the technologies deliver the same service to different types of applications.
Figure 5-3 (page 81), which was constructed during the assessment of the current architecture and shows existing products in the organization, is used as the basis for determining whether the preferred vendor or technology is currently installed.
The Architecture Description contains two different views: the conceptual view and the development view. The conceptual view provides the big picture of the enterprise integration infrastructure and the types of services that will be provided. The development view contains information relevant to developers who will utilize the architecture. In Part III of this book we will describe specific integration patterns and how they utilize the services of the Technical Integration Architecture.
The conceptual architecture is intended to give the big picture of the integration architecture. There is no right way or one way to develop this diagram. It is a conceptual drawing. It needs to convey all of the components of the infrastructure, how they interrelate, and how they relate to the other components of the enterprise. In fact, there may be multiple conceptual views to illustrate a variety of points on the architecture.
The conceptual architecture should include the types of applications or systems that will connect using the integration architecture, what technologies are used for integration, how the technical architecture will be used by portals and on the corporate network and external connectivity as well as how users interact with the resulting applications. The conceptual architecture should be a diagram that can be used to explain the architecture to both management and staff. It will not be satisfying to the technically deep personnel, but it can be used to explain to the business users how the infrastructure is used.
Part III of the book contains patterns and architecture reference diagrams for different integration solutions. However, large companies are likely to have a combination of integration requirements. Below are two examples of diagrams. Figure 6-3 represents a simplified view of the layering of integration services offered. Figure 6-4 (page 99) represents an alternative view of all the integration services that can be part of the Technical Integration Architecture.
The diagram should be accompanied by an overall description of the conceptual architecture, descriptions of each component and the relationships between each.
The development view is a description of how and when each of the different tools and interfaces is used to guide the development team utilizing the integration architecture. An integration architecture is put in place to support developers in the rapid development of new applications that require heavy integration. Many different tools and approaches might be employed by developers to use the architecture. For each and every aspect of the integration architecture there must be a description of how a developer may utilize the integration services in an application. This would include the languages supported and the manner in which services and capabilities are accessed, tools for developing any integrations, and tools for configuration and administration. Also standard interfaces available for use should be defined. (See Figure 6-5, page 100.)
This section specifies all standards that have been adopted by the organization that are relevant to the integration architecture. The full specification should also include a governance policy that defines how compliance to standards will be managed, and the process and guidelines for approving solutions that do not comply with standards. Most of these standards are related to interfaces, formats, or communications mechanisms. Architectural standards are beginning to appear that may have a larger impact in the future on an enterprise integration architecture. One key standard to watch is the Model Driven Architecture (MDA) standard from the Object Management Group. Case Study 6.2 describes the activities of MDA (Soley n.d.).
Types of standards to be addressed in the specification are listed in Figure 6-6, (page 100).
Service-level requirements include availability, integrity and delivery service, scalability, maintainability, manageability, usability, and performance. Transaction, persistence, and directory services may also be required to support the necessary level of service. These requirements can be derived from the application requirements section or they may be imposed by the organization based upon needs of the business.
Each section will most likely need to break down the requirements by application as well as type or span of integration. These requirements are intended to be a guide for the design and implementation of the integration architecture. Many of these requirements will be at a high level and not at a detailed level that will occur with the application design. Specific application requirements may necessitate adjustments to the high-level specification. It is important that the organization treat the Enterprise Integration Architecture as an ongoing process rather than a finished product.
This section captures the availability requirements, such as when the integration will take place (real time or batch), expectations on the access to the service, and any specific metrics that the integration architecture needs to meet. There are two types of metrics to be defined regarding availability: system availability and availability of integration. Typical system availability measurements are working hour availability, usually defined as 8 hours per day, 5 days per week (8 × 5), or full time availability, defined as 24 hours per day, 7 days per week (24 × 7). Availability of integration can be defined as real time or other, such as periodic or batch. This metric defines when the information that has been integrated is available for use.
The integrity of integrated information rests on the integrity of the transmission as well as the integrity of the information being processed. Transmission integrity is ensured by transmission services such as guaranteed delivery, once and only once delivery, and persistent message stores. The integrity of the information processes is dependent upon the validity of the translation and transformation process, and the processing of the information by the target system. This metric can be measured in error rates, and relates to both the quality and cost of the system.
Scalability is a large factor in capacity planning and purchasing. The scalability requirements must be defined for the expected needs of the organization in both the short term and the longer term. Scalability requirements can be defined by the following parameters:
These requirements determine the type of architecture as well as the technologies selected for implementation.
Maintainability and manageability refer to the operational characteristics of the architecture. This part of the specification defines the specific services required. Also, define any requirements to integrate with the existing operational environment. Lastly, identify all related maintainability constraints, such as applications that are migrating to different platforms, or are being “sunsetted.”
Maintainability and management requirements can be defined by the following services:
Monitoring and alerting
Startup, shutdown, and restart
Troubleshooting and level of support
Maintainability of code and use of tools
Installation and managing release of updates and ability to rollback
Scheduling
Integration with existing tools
After determining requirements, we recommend summarizing them for the purposes of enterprise planning. Assigning a manageability requirement rating to each application or project can do this. This rating provides a summary view of all manageability requirements. The following rating can be used:
Level 1. Startup, shutdown and restart, troubleshooting, scheduling remote installation
Level 2. Level 1 plus updates and rollbacks, integrated application repository
Level 3. Level 2 plus real-time monitoring and alerts, full integration of development and management tools
Usability refers to how easily each type of user will use the system. Defining all types of system users, along with the type of access and usability they require, helps determine tools and application requirements. Figure 6-7 (page 104) provides a template for determining usability requirements. This table can be modified or expanded as needed.
Performance requirements define the level of service the infrastructure needs to provide to support business users, processes, and transactions. Performance requirements are also used in the capacity planning view (see Figure 6-8).
A number of different types of measurements may be included in performance requirements. Response time is the expected or acceptable time for users or applications to wait for a response from the system. It can be measured in sub-seconds or seconds (real time), minutes, hours, or days. Throughput is the amount of information that can be sent through the system within a certain amount of time. It can be measured in number of transactions or volume of data. Turnaround time is the amount of time it takes for the entire process to complete. It can be measured in seconds, minutes, hours, or days. Number of simultaneous users determines the number of live connections or sessions the system must support. Number of connected applications refers to the number of integrated applications that could send or receive information through the system.
Transaction services include distributed transaction support and XA standard transaction compliance. This information determines how transactions will be managed and how transactional integrity will be maintained. This section also defines requirements for supporting industry and regulatory standards such as RosettaNet, HIPAA, or other industry-standard transactions.
Often it is necessary to persist or store data for future use during an integration process. Persistence is required for improved reliability when recovering from a failure. Being able to restart a failed system without losing any in process integrations is the most basic use of a persistence service. However, there are numerous other uses for this type of service. Other types of uses for persisted data include the ability to rollback any actions, perform audits of activity, or use the collected data to analyze activity on the infrastructure. This section defines the requirements to provide storage of the integration data and state information during and after any use of the integration infrastructure.
It has become a best practice in modern distributed systems to provide the ability for directory services. Directories provide several fundamental capabilities for the infrastructure. They can provide location transparency by allowing applications to “find” other applications for integration. This reduces the need to hard code location information into the application, and increases adaptability because a location change would not require changes in other applications. In addition, directories can be used to store configuration information on resources or users that can be used by any application or integration process. Finally, a directory can be used to store security information. This use will be examined in closer detail in the section on security.
In this section, define the requirements for directory services. This includes the ability to register any “component” of the system including servers, interfaces, service, schemas, or other types of information.
Figure 6-9 is an example of a simple setup for a directory that might exist. The mandatory fields are the name and location. The type and description are helpful in an operational system. Other fields might be added for specific components.
The Service Level Summary Table (Figure 6-10) is useful for displaying an aggregate view of service-level requirements.
Security is a type of service-level requirement, but it is such an important topic and a highly specialized topic that it is dealt with separately. The specification should start by summarizing the top-level security requirements by the categories or types of applications that will be utilizing the architecture. This can be done in a general manner as shown in Figure 6-11 (page 108), but is more effective if it can be specifically defined.
Authentication services confirm the identity of a user. A detailed specification of authorization service requirements includes the following:
List of user types. User types should correlate to the types of applications or services a group would access. Examples include: designers, programmers, managers, line of business users, customers, and partners.
Level of authentication for each type of user or role. Levels of authentication may include: password, password with public/private key encryption, digital certificate, and biometrics.
Whether unitary login will be supported. Unitary logic defines whether authentication can be performed once for all applications and services. This requires a centralized directory for all services.
Definition of how user accounts will be managed. User accounts must be constantly created and updated based on the changes that occur in the business. It is important to have a formal process defined on how this information will be kept synchronized.
Authorization levels determine what operations a user or process is authorized to perform on a set of data or within an application. This section defines categories for authorization, based on application and/or sensitivity of data (see Figure 6-12). Authorization is usually defined in a CRUD matrix that defines rights to Create, Read, Update, or Delete information.
This section should address how the integration architecture will work with perimeter security and the types or categories of integration that will be required to use the perimeter security features. Perimeter security is the combination of firewalls, encryption, authentication services, and architecture used to protect the enterprise from the outside world. The configuration of the perimeter security will dictate the design of the integration architecture as it relates to external usage.
This section defines categories for auditing based on the type of application and the sensitivity of the data being processed. Basic categories of auditing are:
Level 0. Maintain no information
In cases where there is no worry about the interactions because of other factors related to trust, Level 0 can be used, and no auditing would be performed.
Level 1. Maintain information on type of interaction and participants
In cases where the details are not required and only the knowledge that an interaction has taken place is required, Level 1 would be applicable. This would be used in instances where a rollback is not feasible or necessary, but only the fact that an interaction took place is required.
Level 2. Maintain only instructions for each interaction
Level 2 is used to examine the types of interactions that have occurred and look for odd behavior or verify that an interaction occurred. This may be used to verify that an employee has performed an unauthorized operation on the system and have the information to rollback the action.
Level 3. Maintain a complete set of information on every interaction
Level 3 is used in cases where the interactions are extremely sensitive and either proof of the interaction or the need to fully audit every interaction is required. Full audit may be required in cases of significant financial transactions, for example.
Performance and resource requirements are the tradeoffs in making a distinction between each level. Otherwise, if performance and resources were free, then level four would always be applied. In many instances, this may not be feasible.
Confidentiality refers to the level of privacy that a transmission requires. Confidentiality usually applies to the level of encryption that is applied. However, it also could be reflected in the communications path that is used. For example, if a high degree of confidentiality is required, then the interaction could be directed onto a higher cost dedicated line rather than following a path that uses an Internet connection. Generally speaking, when using encryption, the higher the level of confidentiality, the slower the response time. However, when considering communications channels, the higher degree of confidentiality, the more expensive the communications. Performance, cost, and security are often tradeoffs.
Nonrepudiation is extremely important for B2B transactions. It ensures that a request or an order cannot be repudiated later on. Nonrepudiation services are required to ensure the validity and enforceability of electronic contracts. The specification should define the level of nonrepudiation service required, and which types and categories of applications require it (Figure 6-13).
Types of nonrepudiation include:
Nonrepudiated communications sessions. The endpoints in the communication session, such as an SSL session, exchange tokens that uniquely identify them. This type of nonrepudiation validates that a session took place, but does not validate that specific information was exchanged in the session, as it does not permanently bind the session contents to the originator or the recipient.
Nonrepudiated middleware services. Interactions between middleware services, include a token that validates the service's authenticity. Interactions are securely time-stamped and logged. This type of nonrepudiation validates that an interaction took place, but not that specific information was exchanged in the interaction.
Nonrepudiated transactions. The transaction is accompanied with a token that validates its authenticity and the transaction is time-stamped and logged. This type of nonrepudiation validates that a transaction took place, but not what specific data was processed in the transaction.
Nonrepudiated application actions consisting of multiple transactions. The end-user's intent to take the action is recorded, the application actions are uniquely and irrefutably traceable to the user, and the actions are securely time-stamped and securely logged. This validates that the participants intended to engage in the action, irrefutably validates their identities, irrefutably associates the time of the action with this information, and provides nonrepudiation that the whole process was completed.
This section (Figure 6-14, page 112) specifies the design approaches to achieving application requirements defined in the Service Level section. The goal is to define how all service-level requirements will be met including technologies, policies, and procedures.
All constraints and specific guidance for architects, designers, and developers should be defined at this point. This is an open topic area that is unbounded. However, some areas to consider in the setting of constraints and guidance are
Known performance limitations
Formatting guidelines for data
Constraints on metadata definitions and registration
Preference on use of different types of interfaces
Special cases of security implementations
Deviations allowed from the integration architecture
This section will most likely be very limited in the beginning, but as the use of the architecture leads to better understanding and knowledge of what does and does not work, it will grow over time.
The final section of the Integration Architecture Specification summarizes any particular issues or decisions regarding the integration architecture. These can include unresolved solutions to specific requirements. This might be a good place for executive IT management to provide guidance on the expectations of the integration architecture, how it will impact the organization, and what is expected from the staff. Lastly, it might include discussion on where the architecture is going in the future.
Make the architecture specification a living document. It should be consulted for each new integration project and revised periodically, or whenever required.
Don't boil the ocean the first time out. Scope the initial integration architecture definition project to last no more than two to three months.
Make sure all stakeholders have input to the definition and review the architecture specification. Otherwise, the architecture may be sabotaged.
Plan globally, implement locally.
Design for reuse.
Measure and management reuse.
Implement quality metrics to justify infrastructure investments.
The Technical Integration Architecture provides the framework for choosing infrastructure technologies for the solutions discussed in Part III of the book. Those looking to implement tactical solutions will be tempted to jump there immediately. However, companies wishing to maximize business agility, reuse, and return on investment, will wish to complete the Enterprise Integration Architecture by defining the Service Integration Architecture (Chapter 7), Information Integration Architecture (Chapter 8), and the Process Integration Architecture (Chapter 9).
