Information technology has fundamentally altered the business landscape. In the 1960s and 1970s businesses automated accounting, billing, and other back-end functions. This was the era of the mainframe and the database. In the 1980s information technology provided tools that enhanced the capabilities of workers to be more productive, whether through spreadsheets, word processors, or graphics applications. This was the era of the personal computer and desktop applications. The 1990s opened up communications inside an organization as well as with customers and suppliers. This was the era of the network and the Internet. We are moving into an era driven by the need to improve business processes and to provide increasingly sophisticated capabilities to customers and suppliers to improve how we conduct business. Processing transactions without human intervention, gathering data from multiple sources, and integrating it together or reengineering business processes are the basic requirements for success in today's environment.
The implementation of these applications and systems will be enabled through enterprise integration. In the same way that organizations have mastered mainframes, databases, back-office applications, personal computers, desktop applications, networks, and the Internet, it will be necessary for them to master enterprise integration. How ever, integration has always been addressed as a technology and infrastructure topic. In today's business environment the enterprise integration process begins with business problems. Business groups within organizations are rethinking how they interact with customers, partners, and suppliers; manufacture goods; and operate, organize, and manage the business. Business and IT success are dependent upon integration solutions to enable this business change.
As business problems have different requirements, there are literally hundreds of integration solutions available today. The problem is not whether the technology exists; it's which technology to use to solve the business problem. We designed this book to help companies solve pressing business needs while creating an infrastructure that will enable future business solutions as well.
The need for enterprise integration is being driven by the changing needs of business. Sometime around 500 B.C. Heraclitus said, “Nothing endures but change.” Although change may have been a constant from time immemorial, the rate of change is accelerating far faster than ever before, and this is having a profound effect on business. Business cycles are rapidly shrinking. The way business was conducted even a decade ago is no longer acceptable if a business intends to remain competitive. The changes include how the business interacts with customers, how it manufactures goods, and how it is organized and managed. The changes in business are fundamental and pervasive.
In the past, customers interacted with companies from defined access points such as stores, over the telephone, or through sales reps, during defined business hours. Now customers may place an order over the Web, ask questions about the order over a telephone, and then exchange merchandise at a physical store. The goal and challenge is to have all customer access and interactions managed consistently across the enterprise, even if the customer uses multiple channels for a single transaction. This requires comprehensive integration of customer information across all possible channels. Barnes & Noble effectively dealt with the threat of Amazon.com by providing seamless integration of an online presence with vast brick-and-mortar operations, turning what had been considered a weakness at one time into a real competitive advantage.
It costs many times more to acquire a new customer than to retain an existing customer. Providing an integrated view of the customer brings together knowledge of the customer that is spread across multiple systems in the organization.
Improving customer knowledge enables the company to maximize the value of each customer.
The manufacturing process is also changing. Whereas companies used to produce goods to stock with low levels of customization, the trend and competitive advantage is to manufacture products to order with mass customization. Dell Inc. provides a good example of how this capability propelled it to become the number one computer manufacturer and changed the way people buy computers. Whereas retail customers used to purchase personal computers in stores and then customize configurations after the fact, Dell introduced the capability to purchase computers over the Web or by telephone, customized with the amount of memory, disk storage, and other features they may require such as specialized graphics cards or a math coprocessor. It also delivered the customized system to the customers' door faster and cheaper. To compete effectively, the other computer manufacturers needed to do the same to retain customers.
Integration of the design, planning, and manufacturing processes is critical to managing margins, ensuring that the right amount of materials are on hand, and improving inventory turns. Over the last several years this has been the focus of General Motors Corporation's information systems organization. As a result of reducing redundant systems and integrating across the diverse set of systems that support design and engineering, it was able to reduce the design time of a new car from 48 months down to 18 months.
Competitive advantage requires making the end-to-end process across the value chain—from requisition to payment—as fast and efficient as possible. The concepts of “zero latency” and the “real-time enterprise” are gaining traction as companies seek to accelerate business processes and reduce business cycle times. The term “real time” refers to being able to view, manage, and control business processes in business time, rather than responding to end-of-month or quarter reports after the fact. This includes integrating, monitoring, managing, and optimizing the end-to-end process across applications, business units, and the entire value chain. Optimization may be based on different metrics, such as time or cost (which in some cases may be conflicting goals). Automating business processes can also go a long way towards reducing business cycle times. Cisco Systems has applied this concept to its finance operations to achieve dramatic increases in performance (Powell 2003). Its story is described in Case Study 1.1.
One of the reasons companies are facing the challenges of integration today is the way they were organized in the past. Business operations were organized in functional stovepipes such as sales, order processing, manufacturing, finance, and so on. The back-office computer systems used to manage these organizational stovepipes reflected the specific views and needs of the department, and were not designed to interact with other departmental systems. Each system defined business entities, such as customers and products, without regard to how the other systems represented the same entities. However, the end-to-end business process of the entire interaction with a customer or a business transaction was likely to be supported by multiple business systems.
Lack of integration between systems requires additional manual steps such as rekeying information, increasing the possibility of introducing errors. The cost of fixing errors is not trivial. For example, one high-tech manufacturer integrated the order entry and fulfillment systems, eliminating the need to manually rekey orders, and reduced errors by 40%. Several companies have reported 100% return on investment (ROI) from an integration project within the first year, merely through reducing errors. Automating the flow of information as it crosses stovepipe systems greatly reduces the latency in business processes, reducing business cycle times and enabling the real-time enterprise. FedEx has been a leader in this area. It continues to be on the cutting edge of operational improvement with a focus on field force automation. The details on its latest endeavor can be found in Case Study 1.2 (Brewin 2002).
The globalization of business has fueled the move from centralized to decentralized organizations. Decentralized organizations require access to shared information from multiple locations and systems. This has driven the rise of enterprise portals, which provide front-end, role-based integration. From a browser-based interface, employees, partners, suppliers, or even customers can access the information they require through a single easy-to-use interface that includes information and functionality from multiple back-end source systems.
Moving towards real-time business, or business on-demand, requires real-time management. Previously, defined planning cycles based on historical analysis might have sufficed. However, the company that can recognize and exploit competitive opportunities and react to changes in the market faster than the competition gains competitive advantage. For example, General Electric Company monitors its mission-critical operations across the company's 13 different businesses around the world from “digital cockpits.” The cockpits enable GE to respond faster to change, reduce cycle times, and improve risk management. Rather than waiting for end-of-month or quarter reports, the cockpits provide real-time data to GE.
Management practices based on monitoring key performance indicators and real-time metrics tied to the fundamental business objectives, such as reducing business cycle times, will provide competitive advantage to the real-time enterprise. Methods such as balanced scorecard and Six Sigma have shown that management by metrics improves the overall efficiency, quality, and profitability of the company.
The ultimate goal of the real-time enterprise is business agility—the ability to rapidly adapt to change. Business is changing and the rate of change is accelerating. It is becoming clear that maintaining the status quo is no longer adequate. Companies must adapt to change or perish. The real-time enterprise is becoming a strategic initiative because it enables companies to respond rapidly to rapidly changing business conditions. In an audience poll of an ebizQ Web cast (Gold-Bernstein and White 2003), “Critical Success Factors for the Real-Time Enterprise,” 76% responded that becoming a real-time enterprise was a current strategic business initiative, showing that real time is becoming an essential goal for organizations.
One company that has been successful as a real-time enterprise is Wal-Mart. As the world's largest business, Wal-Mart has done extremely well in operational efficiency, managing growth, and scaling its business. The Chief Information Officer (CIO) established three philosophies in the development of its systems that embody its approach to the real-time enterprise. Details are given in Case Study 1.3 (Lundberg, 2002).
Real-time response to business change requires an integrated infrastructure that enables rapid deployment of new solutions while leveraging existing IT investments. An agile business can't afford to rip and replace business systems. It requires real-time connectivity between people, systems, and business entities. Accelerating business processes requires process automation. Although some processes will require human intervention at certain points, especially authorizations and exception handling, increasing the velocity of business requires automating as much as possible; even after automating processes, companies need to monitor and manage them in real time to continually improve and optimize the processes.
Fortunately, advances in technology are enabling business agility. Integration technology, including messaging, application integration, workflow, business process modeling, automation and management, mobile integration, enterprise portals, business-to-business (B2B) integration, Web services, and eXtensible Markup Language (XML) are all helping organizations tie their systems together and manage the processes across them.
Enterprise integration is the underlying enabler of business agility. It provides the infrastructure that supports rapid change. Without enterprise integration, companies will continue to be constrained by the inability to communicate and manage the flow of information and business processes across the enterprise. The real-time enterprise is not possible without enterprise integration.
Companies that have invested in an integrated IT environment have seen dramatic results in business performance. Case Study 1.4 is a compilation of the impact that can result when an organization is able to harness the power of an integrated information system. However, many decision makers require even more data before they agree to pursue a project.
Even if we grant the fact that real-time business requires comprehensive integration, often managers allocating budgets need more tangible paybacks before they will fund an integration project. In fact, there are clearly definable areas where integration projects have been shown to deliver an ROI. Enterprise integration reduces personnel, IT, and business costs and increases revenues. Two other important benefits that companies gain from integration are increasing customer satisfaction and improving overall quality and efficiency through optimizing business processes. An ROI can be calculated in any of these areas to justify a project.
Personnel costs can be reduced through automating business processes and reducing head count; providing self-service interfaces for customers, thereby reducing customer support costs; and making information easily and readily available, thereby reducing training costs. For example, through enterprise integration a logistics service company was able to triple its size in five years, going from $80 million to $250 million in revenue, without adding additional head count. It standardized on its product offering, which enabled it to implement new solutions faster, with fewer and less-skilled employees. While saving money, it also increased customer satisfaction and reduced risk.
Through process automation, a Canadian transportation company eliminated 70,000 to 80,000 hand-processed transactions, reducing cost per order from $25 to $5, saving $1.6 million per year. A high-tech parts supplier saved over $166,000 annually by replacing transactions that have gone through Value-Added Networks (VANs) with RosettaNet transactions. It reduced the cost of setting up new partners by 50%. Less-specialized personnel can now perform setups previously done by senior-level system engineers. Employees no longer have to learn some of their partners' proprietary applications to conduct business. Lastly, it reduced the time and costs related to checking inventory from five minutes over the phone to six seconds electronically.
IT costs can be reduced by reducing error rates and the cost of fixing errors, eliminating rekeying of information, and reducing system support costs through integration. For example, a high-tech manufacturer using RosettaNet reported an 85% reduction in error rates and achieved a 100% ROI in the first year. Reducing errors cropped up frequently in ROI studies, for good reason: The cost of processing disputed transactions is orders of magnitude higher than nondisputed transactions.
Enterprise integration can reduce business costs by reducing the cost of implementing change and optimizing business processes. Integration provides the infrastructure that enables companies to rapidly implement new business solutions or change existing ones to meet new requirements. Rather than having to hand code connectivity to applications and information for each new business solution, the underlying integration architecture provides the necessary connectivity, translation and transformation, intelligent routing, and process management. With a fully integrated infrastructure and process, management business analysts can implement many business process changes without IT intervention, enabling them to implement change at the speed of business.
Enterprise integration can increase revenue by increasing market share and creating new market opportunities. Increased market share is the result of retaining and increasing revenue from existing customers by increasing customer satisfaction and attracting new customers by bringing new products and services to market rapidly in response to emerging opportunities.
New opportunities are created by integrating with partners and suppliers, bringing products and services to market more quickly, and creating new sales or distribution channels through on-line capabilities. For example, a high-tech electronics supplier achieved a 40% increase in sales through implementing real-time order capabilities. In 1999, 81% of Cisco's orders were placed online, representing $11.7 billion annually, with 98% to 99% accuracy. Because of an integrated supply chain, Dell was able to deliver customized PCs in days rather than weeks, propelling the company to market leadership.
Customer satisfaction is increasingly becoming an important area of focus and spending for many organizations, so we will address it separately here. A major insurance company reported that 1% of retention can equate to nearly $25 million to the bottom line. According to a study by the Insight Technology Group (Dickie 1999), improving knowledge of the customer and customer relationship management has the following benefits:
20% increase in customer satisfaction ratings
42% increase in revenue
35% decrease in sales costs
25% decrease in sales cycles
The potential ROI for increasing customer satisfaction is tremendous. Enterprise integration can increase customer satisfaction by making information easily available and responding to customer requests and complaints more quickly. For example, online customer self-service systems enable customers to view their account balances on demand, track orders, and change information. The customer is more satisfied because needs are met immediately, and the company saves money on personnel costs. Integrating customer information from disparate back-end systems enables improving customer interactions at every stage of a transaction, through every channel used for a customer interaction. Companies investing in Customer Relationship Management (CRM) systems are finding that integration is a large part of the implementation process.
Business process optimization holds the greatest potential for ROI. Through process optimization companies can do things better, faster, and cheaper. For example, a large apparel manufacturer reduced inventory costs by 10%, saving millions of dollars through the ability to plan plant capacity better and optimize the manufacturing process. A logistics service company reported that its customers have been able to shrink inventory floor space by 50%, yielding significant savings in inventory carrying costs through enhanced visibility into its logistics supply chain. A high-tech manufacturer projects a 230% ROI over five years through improved business processes provided by implementing RosettaNet Process Interchange Protocols (specifications for automated B2B transactions in the high-tech industry). Business process optimization reduces the latency in business and improves efficiency, quality, and bottom-line profitability.
Business drivers and potential benefits are compelling. Integration is essential for moving towards real-time business. Surveys show that companies are focusing IT investment on integration. Morgan Stanley's survey of 225 CIOs consistently ranks integration as either the number one priority in IT spending, or in the top three (Phillips and Rathman 2002). Gartner puts application development and middleware in the top three for annual growth rates (Correia 2002). Companies are increasingly recognizing the need for enterprise integration.
Unfortunately, integration is inherently a complex problem. Providing integration, visibility, and management across hardware platforms, operating systems, programming languages, database structures, applications, and business entities is probably one of the most complex tasks in IT. Integration doesn't come in a box. You can't just install and configure it. There are multiple technologies involved, and different technologies are most appropriate for different types of business requirements. What companies need to avoid is the necessity of integrating the integration solutions.
Enterprise integration is really more of a journey than a solution. Companies will need to continually evolve and optimize their infrastructures to provide better response time to their customers and changes in business conditions. This process involves many decisions along the way.
Defining the business drivers, requirements, and metrics driving integration. What is the business initiative launching the integration project? What are the business requirements? What are the business metrics that will measure the success of the solution?
Determining the purpose and scope of the integration. Is it enterprise-wide or linked to a strategic business initiative, or both? Or is it tactical, such as implementing a new application or Web service?
Deciding whether to make an enterprise commitment to a service-oriented architecture.
Choosing the type of integration. Integration scenarios include application integration, data integration, process integration, legacy integration or extension, B2B integration, and portals.
Identifying the technologies necessary for the type of integration. This could include one, some, or all of the following: messaging, data mapping, translation and transformation, intelligent routing, business process management, Web services, XML, B2B integration, and portals.
This book will guide business managers and IT professionals through the complex decisions surrounding enterprise integration. The ultimate goal is to select the right solution to solve the business problem at hand while increasing business agility. This book is not a rigid methodology. Instead, it is intended to provide guidelines based on best practices for implementing successful integration solutions. We have included templates that we have found to be useful for designing and documenting the integration solution.
The book is designed to be flexible, so it can be adapted based on your project needs. Although the authors truly believe that an enterprise integration strategy and architecture ultimately produces the most agile infrastructure (see Chapter 3), we recognize that tight IT budgets require focus on tactical solutions. Therefore, we offer you guidelines for making tactical decisions while maximizing long-term agility.
Figure 1-1 is a road map to this book. It is our sincere intent and hope that it is a helpful guide on your next integration project.
As the map shows, the first step in the process is to define business drivers and requirements (Chapter 2). This needs to be done regardless of the integration project at hand. This step will determine the scope of your integration initiative, after which you will have some choices of what to do next. However, even if you are implementing a tactical solution, it is important to have an enterprise road map to avoid costly mistakes down the line. Therefore, we strongly recommend that everyone read Chapter 3 on creating an integration strategy, and Chapter 4 through Chapter 9 on defining an integration architecture.
For project work, readers can use the chapters and templates that are most relevant to the solution being implemented.