The purpose of integration is almost always to support improving a business process to increase business efficiency. Process-level integration defines the interactions among systems through business workflow definitions. The in structions for the integration are defined at the business-process level rather than the technical-interface level. Although a technical integration infrastructure is still a necessity to implement process integration, the processes themselves are technology independent. This enables change to be made quickly and easily, increasing business agility.
The role of the Process Integration Architecture is to create process models and definitions as managed entities that can be easily viewed and changed in response to changes in a business. The Process Integration Architecture defines end-to-end business processes, which are then automated across existing systems on disparate platforms.
The key benefit of the Business Integration Architecture is the business level view it provides of the end-to-end process. Process integration technology includes dashboards that enable business managers to track key performance indicators in near real time. Process simulation and analytics provide feedback to help companies optimize business processes, reduce cost, and achieve competitive advantage through better processes. Case Study 9.1 describes how General Electric has achieved significant business process improvement with amazing operational results (Lindorff 2002).
The Process Integration Architecture also improves alignment between IT and business. It starts with business process models that business people can understand and review. These models depict the shared understanding between the business and IT staff of the end-to-end business process. The process can then be automated directly from the model. This significantly reduces the chance of misinterpreting the business requirement or getting the process wrong. As the process can span multiple business units and managers, there may not be one manager who owns or even understands the complete end-to-end process. The process models represent a shared understanding of how business is conducted across the enterprise, and should be managed as valuable assets.
This chapter appears last in the Enterprise Integration Architecture section because most companies implement it last (if at all). Companies have not been historically strong on process management and improvement—although those that focus on process tend to be the market leaders in their industry. The next section defines why it is so important for companies to pay attention to process.
Companies that do implement a process integration architecture and infrastructure will start their integration projects by modeling the business process and drill down from there, defining which applications, services, and information is required to complete the process. This tends to be a faster and more efficient approach than a bottom-up integration connectivity approach, and is far more responsive to change.
The benefits of business process management are not a new revelation. It has been known for over half of this century that optimizing business processes yields measurable results. In the 1950s Dr. W. Edward Deming, a mathematician and statistician, proved that improvements in managing business processes lead to improvements in overall business and significant competitive advantage. As a result of Dr. Deming's work, Japanese manufacturing companies were able to produce higher quality products at a lower cost and gain competitive advantage. For his efforts, in May 1960, the Emperor of Japan decorated him with the Second Order Medal of the Sacred Treasure and Japanese manufacturers created the annual Deming Prize in his honor.
Deming taught that variation is created at every step in a production process, and variations have the potential to cause defects. Therefore, the causes of variation need to be identified and reduced and companies need to practice continuous product and process improvement to improve quality and productivity, and decrease costs. Deming proposed a continuous feedback and measurement cycle to reduce defects and improve product quality, usually referred to as the Plan-Do-Check-Act cycle.
PLAN. Design or revise business process components to improve results
DO. Implement the plan and measure its performance
CHECK. Assess the measurements and report the results to decision makers
ACT. Decide on changes needed to improve the process
The Plan-Do-Check-Act cycle is a continuous cycle. To achieve measurable results and business benefit, companies must practice continuous process improvement. Wal-Mart, the world's largest company, is able to require its suppliers to support its process improvement in a manner remarkably similar to the Deming cycle. Case Study 9.2 details the Levi Strauss & Co. experience (Girard 2003).
Dr. Deming's work gave rise to a number of initiatives to manage and measure the efficiency and effectiveness of business, including Total Quality Management (TQM), balanced scorecard, Six Sigma, and ISO 9001. These initiatives can help companies understand how to define key performance indicators to measure their business (see sidebar—How to Measure Success, page 164). These metrics can then be used on a management dashboard.
Despite half a century of research and proven success, few companies have optimized their business processes. In the early 1990s, based on Dr. Michael Hammer's best-selling book Reengineering the Corporation, corporations began systematic efforts to improve their business processes. Unfortunately, those initiatives were often less than successful. A key inhibitor to the success of business process reengineering (BPR) was a lack of technology to support the focus on radical reengineering of business processes. The logic for business processes was either buried in the code for each application or was manually performed, requiring significant custom coding and integration efforts to achieve results. This led to great organizational upheaval. In contrast, emerging process integration technology enables companies to leverage existing IT investments and automate new processes that use them.
Process integration supports the Plan-Do-Check-Act cycle to enable process optimization. The business process integration tools provide process modeling, to “Plan” the process, automation and integration to implement or “Do” the process, monitoring tools and dashboards to “Check” processes, and simulation and analytical tools to help companies “Act” to improve processes.
Just as there are many Eskimo words for snow, there are multiple terms that describe different types of process-based tools. Each of these is best suited to a particular type of process solution. This section discusses the different types of process-integration technologies, including Business Process Management (BPM), Business Process Integration (BPI), Business Process Automation (BPA), Workflow Automation (WA), Business Activity Monitoring (BAM), and Web Service Orchestration (WSO).
We view BPM as the broadest implementation of process management functionality. It includes support for all aspects of designing, implementing, monitoring, and managing both automated and manual business processes. This includes process modeling, process automation and integration, workflow integration (for manual processes), real-time monitoring and alerts, operational and business dashboards, analytics including key performance indicators, and process simulation for enabling process optimization.
BPM solutions are recommended for companies looking to implement business dashboards, practice proactive process management, and work towards process optimization. Alternatively, companies looking for compliance or industry solutions may find the solutions are implemented in (and come with) BPM technology. The company then has the infrastructure necessary for continuous process improvement. General Motors Corporation has mastered the art of BPM with results that are nothing less than incredible (Bacheldor 2003). Case Study 9.3 describes its results.
BPI refers to process-level integration, as opposed to message-level or point-to-point integration. BPI includes the ability to design and model the process, integrate it with the underlying applications, and monitor the process on an end-to-end basis. The monitoring and management capabilities, however, tend to be on the operational level, not the business management level. BPI does not provide analytics and simulation. In other words, it provides process integration without process management.
BPI is appropriate when your management has little interest in process management and improvement and just wants to get the job done as quickly as possible, but the process is changeable, differs by location or business unit, or the underlying applications are changing. The major benefit of a process abstraction layer provided by BPI is that it adds agility for change.
BPA refers to automating business processes. These tools tend to be transaction oriented and do not necessarily support long-lived transactions or manual processes. BPA tools generally include process modeling, application integration, and processing rules, and may include a development tool for new development, process monitoring, and operational-level management. Companies looking to automate transaction-based processes that do not involve humans and complete in seconds or nano-seconds rather than hours, days or weeks, should consider a BPA tool.
Workflow automation focuses on reducing the lag time in manual business processes. It includes a workflow engine that controls the flow of work across workers and departments according to a set of business rules. Managers can assign work to different types of workers. The workflow engine provides load balancing across all workers, accounting for backlogs, vacations, etc. Therefore, work can be automatically and efficiently assigned. Workflow tools provide work portals where employees can see all assignments and report when they're done, to trigger the next part of the processes.
Most business processes include some manual steps. Therefore, most BPM vendors include some support for workflow, but the extent of workflow features differs widely. If your processes have just a few manual steps, such as exceptions and approvals, then minimal workflow features will be sufficient. However if your processes have a significant manual component, then fully investigate the workflow capabilities of any tool you choose, including robust work management features.
BAM is a term coined by Gartner Group and currently in the cycle Gartner refers to as market push or the start of media infatuation. However, it is more accurately at the stage of vendor infatuation, with each vendor rushing to promote its BAM capability, because Gartner asserted that by 2004 BAM would be one of the four top initiatives driving IT investment, affecting every industry. Although monitoring and analytics are an essential part of BPM, Gartner defined additional BAM capabilities for combining real-time alerts with business intelligence, trend analysis, and data mining. This is the marriage between business integration and business intelligence.
The widespread industry acceptance of BAM is due in part to vendors eagerly looking to define themselves with the next big thing. However, although they all declare they have BAM capabilities, they're at different levels of BAM. Some provide monitoring dashboards that provide notifications, then a person needs to go and do something to react to the alert. The system does not guide the reaction—it only provides the alert. The next stage of BAM integrates business intelligence with near real-time analytics, so when the manager is alerted, he or she also has the information to make a better decision about what to do next to respond to the alert. The next stage of BAM uses business rules based on analytical results and automatically takes action instead of just alerting a person to take action.
This wide variance of capability means there are many types of BAM vendors. The BPM vendors all offer BAM solutions, the EAI vendors have added BAM capabilities or just relabeled what they already had as BAM. There are business intelligence vendors, system management vendors, and pure-play BAM vendors. The BAM market is still in the very early stages.
WSO provides process-level development and coordination among Web services. WSO is used to define and implement the flow of control between the service components of a composite application. Web services can reside on multiple platforms and technologies. They can be components in an application server or legacy applications on a mainframe, and both can participate in a single composite application. The WSO tool includes process modeling, a process engine, and operational-level monitoring and management. Many application server vendors are providing basic WSO tools to enable composite application assembly of application server applications.
WSO is a good fit for composite applications consisting of all Web services. However, if the composite application requires complex integration with multiple platforms and technologies and end-to-end real time process monitoring and management, a BPM solution might be a better fit.
The process integration market is expanding, and buyers will find solutions being offered from a variety of different vendors, including the integration vendors, application-server vendors, and pure-play BPM vendors, as well as workflow and document management vendors, rules engine vendors, and soon, business intelligence vendors. Moreover, companies looking for compliance and industry solutions will find that BPM vendors are offering them, and the technology may come into the enterprise through a packaged solution.
By providing end-to-end, real-time visibility into business processes these tools help companies increase operational efficiency and decrease costs. Because process integration provides a layer of business process abstraction that is physically separated from the applications and implementation technology, it enables rapid business change.
Although every process tool on the market offers some kind of modeling capability, most are proprietary tools. They may do a fine job modeling your process, but offer no portability or long-term protection of the models. A process model represents valuable corporate knowledge and a significant corporate investment. Portions of the end-to-end process may be known and managed by different people in different parts of the organization. The process model captures the knowledge from across the organization and enables the company to optimize at an enterprise level to maximize ROI. For this reason, process models should be managed over time as valuable corporate assets.
Process modeling standards help companies protect their process investment by enabling portability across tools. This prevents technology lock-in. A standard process execution notation that can be exported and imported by all tools effectively solves the most important problem of technology lock-in. The process modeling standards companies will want to consider include BPEL, UML, and IDEF.
Often known as BPEL, Business Process Execution Language for Web Services is the emerging leader for standard process representation. BPEL is an amalgamation of XLANG from Microsoft and WSFL from IBM, and supersedes both approaches. Microsoft, IBM, and BEA, along with others, joined together to submit the BPEL specification to the international standards body OASIS.
BPEL is a standard execution language, not a standard notation. This means that tools will implement their own styles of models and produce BPEL code from the models. A number of vendors of existing modeling tools have announced support for BPEL. The BPEL standard enables portability of models among tools. Although BPEL is becoming an important process-modeling standard to pay attention to, there are also some others companies may want to consider.
The Object Management Group's (OMG) UML is also a relevant process modeling standard, especially for companies focused on new development, and who already use UML. Furthermore, the OMG's Model Driven Architecture (MDA) is a generic approach that can be applied to a wide variety of business problems, including process management. MDA creates a metamodel in UML that abstracts the definition from the implementation. Using this approach, the same model can be used to generate code for multiple platforms. MDA provides the framework to integrate different standards, languages, technologies, and platforms.
Companies implementing composite applications that include new development will be interested in a UML-based approach to process modeling. Developers are usually the ones to define the flow of control in the composite application, and UML has wide acceptance for systems development and among system developers.
BPMI and WfMC are two other noteworthy process organizations. BPMI (http://www.bpmi.org/index.esp) was established to create specifications to “standardize the management of business processes that span multiple applications, corporate departments, and business partners, behind the firewall and over the Internet.” BPMI is working on notation standards as well as process query language. WfMC is an organization focusing on the human component of business processes, including processes for assigning work, sharing documents, and managing task lists. Both BPMI and WfMC support and are contributing to the BPEL standard.
IDEF is based on the Structured Analysis and Design Technique (SADT) introduced by Douglas T. Ross in the early 1970s. In 1981, the US Air Force Program for Integrated Computer-Aided Manufacturing (ICAM) standardized and adopted it as a Federal Information Processing Standard (FIPS), and made public a subset of SADT, called IDEF0. This standard is still in widespread use today, and is an extremely important modeling standard for organizations doing business with the federal government.
Companies should pay attention to the evolution of process standards to ensure the portability of their business models and maximize their process investment over time.
See Appendix G for the complete Process Integration Architecture Specification.
The Process Specification provides guidance for applying a process-driven approach to integration. This document is a guide to creating the process specification for the composite applications or process-driven business solutions.
The scope of a Process Specification is usually limited to a set of business processes associated with an integration project. The document should define the business processes and underlying applications to be integrated. We recommend not boiling the ocean. Start with a few business processes that will deliver measurable impact to the business.
This section identifies all stakeholders in the business process(es) being integrated, including business managers who control all or part of the process, system designers and architect(s), and the development team who will execute the implementation. Because the end-to-end process may cross multiple business domains, it may be difficult initially to define all business stakeholders. The process design, review, and verification may, therefore, be an iterative process, with new stakeholders and roles identified along the way.
This section describes the business processes that have been identified in the requirements. To identify the business processes that need to be defined as part of this specification, start with the Statement of Purpose and the scope of responsibilities defined in the Business Strategies and Initiatives Specification. (Figure 9-1). An alternative way to identify business processes is a bottom up approach (see the process flow models and sequence diagram, section 9.5.5).
The Process Description Table can be used to identify each process that will be modeled and implemented (Figure 9-2). A name and a business owner identify each business process. A description of the process is provided along with the event that kicks off the process, the business services that are part of the process and the outcome expected which ends the process. If no services have been defined, then define the functions to be performed as part of the process.
A process flow model is a combination of the event(s) that start the process, actors involved throughout the process, services provided by software components, the messages passed between and among services, and the business rules controlling the flow of the process. Each process flow model should have a description of each of these entities as well as a process diagram.
The best way to keep models up-to-date is to do it in software. Paper documentation requires changes to be made and distributed in print. It makes keeping the specification up-to-date with the real process far more difficult. It is easier, cheaper, faster, and better to manage models in software. Therefore, this part of the specification may actually have a reference to the process model stored in software. If your modeling tool provides browser-based access to models, then the URL would be part of the process-specification document. Copies of the model may be included in this specification to distribute printed documentation to be used in the verification process. However, it must be recognized that these models can quickly become out of date as the processes change. Therefore, it is important to include the date and version number of all printed process models.
The process model, including the diagram and all associated specifications, should include the following information:
The kickoff event(s) of the process. There could be more than one starting point, depending on the purpose and the operation of the process. If a process contains more than one starting point, include all of them.
All tasks/functions/services to be performed during the execution of the process.
The order in which the tasks/functions/services should be accomplished, including any tasks that may be performed in parallel.
All decision points, both those having to do with choosing a path through the process and those that determine whether or not the process should continue.
The business rules that determine the path of the process.
Any points at which the process path may split or merge.
The messages input to and output from each task/function/service.
The completion point of the process. As a process may have multiple starting points, it can also have multiple completion points.
In order to bridge the gap between IT and business and have a seamless handoff between process design and implementation it is important to hand off a business-process model to programmers in a manner they will recognize and use. As noted in section 9.4, there are a number of different modeling methods and systems of notation. The evolving process standards are likely to impact future adoption of a particular modeling method (see Figure 9-3). In the meantime, each tool either supports its own proprietary notation or it supports one of the more widely used methods, such as activity diagrams (usually modified) (Figure 9-4), data flow diagrams (DFD) (Figure 9-5), or IDEF (Figure 9-6). UML is relevant because it has become the most widely applied standard design notation for programmers. The choice of which type of modeling method to use will depend on which industry you're in as well as which tool you use.
Copyright © The Business Process Management Initiative [BPMI.org] (2001–2004). All Rights Reserved.
Figure 9-3. BPMI Process Diagram
Figure from OMG Unified Modeling Language Specifications, v1.5. Reprinted with permission. Object Management Group, Inc. © OMG 2003. http://www.omg.org.
Figure 9-4. UML Activity Diagram with Swim Lines
Figure from OMG Unified Modeling Language Specifications, v1.5. Reprinted with permission. Object Management Group, Inc. © OMG 2003. http://www.omg.org.
Figure 9-5. UML Sequence Diagram
Several vendors are now providing different perspectives for different roles in the process. Some are using Eclipse, an open source framework for tools integration, to enable different models for different design perspectives.
Process design reviews are critical to the overall success and agility of the system. The design reviews should include all relevant stakeholders, defined in the Key Participants section. All parts of the model need to be reviewed and verified. Participants need to verify the portions of the process they are responsible for, including all tasks, functions, and/or services, inputs to and outputs from each service, decision points along the process and business rules that determine the process flow of control, as well as all exception and compensation rules. The overall process should be reviewed for opportunities to decrease time and cost, and increase business flexibility and advantage.
Plan-Do-Check-Act. Model and verify business processes with all stakeholders. Automate and integrate processes. Monitor processes on a real-time basis with business and operational dashboards, and practice continuous process improvement.
Manage process models as a corporate asset. The model contains knowledge of the end-to-end business process, which multiple business managers from across the company may have participated in creating. This represents a significant investment. Manage the investment in modeling business processes so that it provides a payoff down the road.
Standardize. Once again, companies should consider the importance and benefits of standards for long-term viability, reuse, and management of business processes. When a company standardizes on a single modeling method, it facilitates communication and verification of process models across different groups and lowers training costs.
Case Study 9.4 describes how Costco applied process integration and these best practices to improve their decision making process and business results (2003).
The Enterprise Integration Architecture provides guidelines and standards for all integration projects. Although the Process Integration Architecture is often last to be defined in the integration adoption cycle, once the infrastructure is in place it should become the starting place for integration solutions. Part III of the book describes technologies and reference architectures for specific types of integration solutions (see Figure 9-7). It must be noted that some integration projects require a combination of integration styles. Chapter 14 provides a reference architecture for a completely integrated enterprise that combines all styles of integration.
