Businesses run on processes. Processes develop out of necessity and become part of a company's culture, or a way of doing things, and define its day-to-day operations. How those processes work determines a company's success. Efficient processes enable the business to thrive. Inefficient processes drag the business down and can ultimately ruin it if the competition is more efficient.
Everything from making phone calls, to sending e-mails and instant messages, to filling out electronic and even paper forms, most actions that employees take are a part of a larger process. Whether that process is bringing a new employee onboard or paying a supplier, from start to finish the process can be described and documented. Often the details of a particular process are known by a handful of people, and sometimes by just a single person. In large processes involving many people, the full details of the entire process may not be known by any one person, which makes it more difficult to document. In fact, determining the exact nature of a process in large organizations is not easy. But writing the process down pays big dividends, because once the process is documented it can be optimized and improved, which helps the business become more efficient. Parts of the process may remain a manual task for an employee, but many parts can be automated, which can take the efficiency of the business to another level.
What if your business has other needs? Maybe efficiency is not the most important issue facing your company today. Maybe you need to streamline operations, improve responsiveness, make better decisions, increase the visibility of process operations, reduce IT costs, minimize risk, or enforce governance. These are all critical aspects of your business that, if resolved, would make your business run more efficiently. You would be meeting your business requirements — the requirements to run your business — more effectively and more efficiently.
The K2 platform enables businesses to be more efficient. This may sound like a bold statement, but think about how your own company operates. You know that there are big efficiencies to be gained if you could identify all the processes in your business and start automating the most critical pieces of those processes. For example, to be able to optimize a process to avoid bottlenecks or to identify how an employee's absence affects the functioning of the business is something that many companies believe is beyond their reach. But through process discovery, automation and the right tools, getting the business to operate more efficiently is no longer beyond the reach of most, if not all, companies.
To begin your look at process management and K2, this chapter covers the following topics:
Defining workflow
Exploring the history of K2
Examining the K2 platform, including the anatomy of K2 blackpearl and its key features
In terms of workflow, making the build versus buy decision
Evolving workflow
There are many terms that describe the methodology and technology around processes and process management. Some equate the term workflow with business process management (BPM). Since Microsoft introduced the Workflow Foundation (WF) in .NET 3.0, more companies are trying to figure out how workflow can enhance their business. There is also a buzz in the marketplace around business process automation, process-driven applications, and workflow-enabled applications — all of which, at the end of the day, mean the same thing. Analysts predict strong growth for technology solutions and add-on applications that integrate with traditional applications to provide process automation, but how can you make sense of what is real and what is just buzz? It helps to understand the history of business applications.
In the recent past, line-of-business (LOB) application development and implementation happened in the IT shop, where applications were developed from the ground up or custom development occurred on an application from a vendor. The initial request would come from the business or be pushed down from upper management. After application requirements were gathered, IT developers would architect the solution and, through a handful of builds and revisions of the application, deliver it back to the business. This process could take anywhere from 6 months to many years. Maintenance of the application became a much lower priority for IT as other business requests became a priority, and changes to the initial application became very difficult, if not impossible, to make. Over time, these changes would occur, and eventually the price to pay, either in a direct charge to the business or from a resource- and time-investment perspective, became too high for the business. The application became a "legacy" application that IT no longer had to modify, although they still had to spend resources to keep it running.
As custom and off-the-shelf LOB systems became more feature-rich and information technology grew more sophisticated, a new form of business need emerged, namely workflow. Arguably the need was always there. People were responsible for understanding and maintaining complex relationships and processes in the business. Often those processes were documented so that the organization wouldn't forget. Often that documentation wasn't read. The advent of systems helped codify some things, but usually this happened in silos. Software was needed to help bridge the silos of people and information and to help organizations move processes from paper and tribal knowledge to something codified, enforceable, and trackable.
If it wasn't workflow by name, it involved the integration of multiple applications with processes and contracts for how they worked together. It was around this time that Enterprise Application Integration (EAI) and Service-Oriented Architecture (SOA) became popular concepts. By this time, many businesses had moved away from paper-based documents. Electronic versions of documents and business forms came to be used to capture information because this was cheaper and the technology was readily available. This exposed the need to capture the remaining human interactions with these electronic business entities. Whether a document and data were stored haphazardly on a file share or checked in to a line-of-business document management system, the need to manage the chain of human interaction with the data was the next challenge.
Even a decade ago, automation of a human-to-human or human-to-system workflow was virtually impossible for most companies, except for those with a lot of time and money to spend. But in the recent past, many workflow solutions have entered the marketplace that allow small to medium-sized companies, as well as large organizations, to benefit from process applications. The latest versions of SharePoint and Microsoft Dynamics CRM (Customer Relationship Management) include the ability to create workflows out of the box. Creating and running workflows has become commonplace, at least for those companies who adopt some of the latest technology.
The question that hasn't been answered by these solutions, however, is how do you create and automate processes to transform a business? Sure, you may be able to automate the gathering of feedback or the sending of an e-mail based on some action a user takes, but is that really creating efficiencies in the business? The need to discover, optimize, automate, monitor, and improve business processes, or workflows, is known as business process management (BPM), and this can mean many things to many people. At its core, it has nothing to do with technology, as business processes can be optimized and remain completely paper- and human-based. For most businesses, however, technology plays a central role in the way that processes are identified, created, and delivered to those involved in the processes, that is, "the business," and a solution is needed for that. Business process management system (BPMS) is the term for technology platforms used to implement BPM.
The term workflow can mean different things to different people. You may see the term workflow in graphics- and video-editing applications, in SharePoint, or in a description of an assembly line. Workflow, as it relates to K2, BPM, and process-driven applications means one thing: the automation of all or part of a business process involving human actions and/or system functions.
Some would argue that workflows involving humans can't be automated, but capturing that interaction of the user within the business process is a critical piece of a workflow. A workflow is not a task that begins and ends with a single user and contains information that nobody else will ever use. This type of workflow is a matter of personal productivity, not business productivity. However, a workflow that integrates disparate applications to improve the business process and increases the transparency of that process is a workflow that is interesting to the business, even when there is no human interaction with that process. An example of this is an event that creates a file in a location on the network, which in turn creates an instance of a workflow to extract data from the file and update two different LOB systems. No human interaction was necessary for that process to happen. System (or system-only) workflows do increase business efficiency, and the data captured can be used by someone to make better decisions about the business. More common than system workflows is the need to capture human-system interactions, where an action taken by a user creates an instance of a workflow, which in turn creates tasks for other users and records data for reports and other business systems.
K2 blackpearl, the core of the K2 platform, enables businesses to build process-driven applications that improve business efficiency. Started in 2001 in South Africa, the first version of K2 was a visionary product that enabled businesses to leverage information and resources for increasing business efficiencies and making better decisions about the business. K2.net 2003 followed and offered strong integration with Microsoft business applications such as BizTalk, SharePoint, Exchange, and the Office 2003 product line, as well as SQL Server and Active Directory. Along with an extensive API to extend the platform, K2.net 2003 became a market leader in building business applications.
K2.net 2003 included a business process design canvas, K2 Studio, where designers could build processes and modify the code behind each process element, otherwise known as an activity. Those familiar with K2.net 2003 know that this ability to modify and extend the code was necessary in many cases to tailor the process to the business requirements. While powerful, this model also led to longer development times and greater maintenance of the application once deployed to production. K2.net 2003 did not feature out-of-the-box reporting capabilities and encouraged a data model that was tightly coupled to the process itself. K2 blackpearl, released in 2007, addressed many of these concerns and extended the vision of enabling business to be more efficient and make better decisions.
As Adriaan Van Wyk, CEO of SourceCode Technology Holdings, said in response to K2 blackpearl being named a finalist for the 2008 American Business Award (otherwise known as the Stevie Awards), "When we set out to build K2 blackpearl, we set out to create something simple, so that nontechnical people can contribute to and participate in the application-building process; and flexible, because business needs constantly shift; and fast, because time is money. We feel like we accomplished all of that and that it is a unique product in the industry."
K2 blackpearl comprises three key features:
By building on .NET 3.0 technologies, the most important of which is the Workflow Foundation (WF), K2 blackpearl uses a declarative model for building processes. The declarative model, based on WF constructs, allows more flexibility and power when designing the process, while at the same time reducing the amount of custom code required. While modifying the workflow activities and code behind the WF constructs is supported and is similar to what you find in K2.net 2003, the need to use code is minimized to the extent that you can build and deploy even complex processes without a single line of code.
K2 blackpearl includes many reports for common process-related reporting scenarios, including a View Flow report that allows real-time visualization of the state of a process instance. Based on Microsoft SQL Server Reporting Services, all reports are built using the Report Definition Language (RDL) file format and, thus, can be modified and extended using common report design tools. Custom reports can also be imported into the K2 reporting environment following deployment to the SQL Server Reporting Services server. More details of the reporting features are covered in Chapter 20
Data, a key aspect of all business applications, is managed by K2 blackpearl in an innovative way. Data is stored in the process and can be managed in a similar manner to the way that it was managed in K2.net 2003. However, K2 blackpearl includes new data-handling features that allow the important data to be abstracted from the process and, if it is data from a LOB system, to remain in the LOB system. This is useful in many scenarios, including reporting, process management, process design, and providing access to process data to other applications or LOB systems. The key features that enable these scenarios are SmartObjects and SmartObject Services, details of which are covered in Chapter 7.
While those three key features give you a high-level overview of what K2 blackpearl offers, there are other important concepts to understand about the K2 platform. We're going to refer to it as the anatomy of the K2 platform because it's not an architecture per se but rather a view of the entire technology stack that K2 is built on and extends.
On the server, the K2 foundational technologies leverage standard Microsoft server technology, such as Internet Information Services (IIS) and Microsoft Message Queuing (MSMQ), and .NET 3.0. At the heart of K2 is the server, sometimes referred to as the Host Server, which hosts not only the .NET 3.0 WF run time but many K2 services and servers.
More detail of these servers and services is covered in Chapter 5.
Chief among these servers and services are the Workflow server, the SmartObject server, the core servers such as the Dependency server and the User Role server, and the Communication and Discovery services. The Host Server allows all hosted servers and services to communicate using a common framework for sharing data, user credentials, and Single Sign-On (SSO) security tokens. This common framework also provides a mechanism for discovering other Host Servers on the network.
You can also develop custom servers and services to extend the platform.[1] You can substitute custom security providers, logging mechanisms, SmartObject services, and thread management for the out-of-the-box services that the K2 platform provides. This modular, extensible, pluggable, and scalable design of the K2 platform enables partners and customers to customize and tailor the K2 installation to meet almost any network architecture or security requirements. Moreover, since the core features of the server are built on the common Microsoft framework of Windows server technologies and .NET 3.0, the K2 platform benefits from the scalable and secure framework that those components provide.
The key features of the K2 platform include:
A highly-scalable and robust process engine: The engine provides the core business application processing and logic and is built to handle large volumes of processes on a single server. With a distributed topology, the platform is capable of handling an enterprise-class level of processes on a daily basis.
A common communication and messaging platform: Built on common protocols, such as HTTP, DTC, and RPC, the K2 platform can be installed in the most demanding environments by using common protocols for server communications.
Deep integration with Microsoft products and technologies: This includes SQL Server, Reporting Services, Exchange, Windows SharePoint Services 3.0, and Microsoft Office SharePoint Server (MOSS) 2007, the Microsoft Office 2007 System, Visio 2007, BizTalk 2006, and Visual Studio 2005 and 2008. This also extends to core Microsoft technologies such as Microsoft Message Queuing (MSMQ), Internet Information Services (IIS), SMTP, Network Load Balancing and Clustering, Kerberos, Active Directory, and SQL user manager and the .NET 3.0 and 3.5 technologies such as AJAX, WF, Windows Communication Foundation (WCF), Windows Presentation Foundation (WPF), and Silverlight.
Integration with line-of-business systems: This includes both the out-of-the-box capabilities and the additional products like K2 connect for specific systems, such as SAP, Siebel, and Oracle.
Real-time data flow: By default, data in the K2 platform isn't cached, which provides the most up-to-date view of business information regardless of the source of that information. The robustness of this aspect of the K2 platform not only allows LOB information to be used in processes but also enables that LOB information to be updated according to business rules and back-end data constraints, which are controlled by the data engine. The same engine also allows snapshots of the data to be captured for compliance or reporting purposes.
A highly secure platform: The platform, built on Microsoft technologies, provides out-of-the-box support for fine-grained permissions and role-based task assignment, and a method for handling primary and secondary identifications for each user, all of which are configurable by the administrator.
Forms and UI: A dynamic forms capability based on a number of technologies allows custom forms to be rapidly developed for capturing and validating data entered by users.
Business Involvement: Involving business users who know more about their job than anyone in the IT department is a key aspect of the K2 platform. Enabling business owners to model, design, and improve their own processes may require some support from the administrators of the technology itself, but it is a vast improvement upon the sometimes antagonistic relationship that business owners have with the owners of the technology development and management staff.
A continuous Return on Investment: Through continued quality improvements, additional functionality and entirely new components are incorporated into the K2 platform for the benefit of new and existing customers. Two examples of this include the Out of Office functionality and the updated K2 Designer for SharePoint.
The K2 platform is built on proven technologies and is a reliable, secure platform to build on and extend. Even the most demanding environments can employ K2 to automate and optimize their business processes, and tailor the platform to meet the current and future needs of the business.
By now, you should have a good idea of how workflow and BPM fit together, the basic technical underpinnings of the K2 platform, and a general idea of how businesses can benefit from process automation. In subsequent chapters, you will learn how to identify and build processes that transform your business. For the purposes of this discussion, it is necessary to identify where BPM is today and look forward to where it is going.
By whatever name you give it, applications that are "process-aware" don't look or function like other applications. To the end user involved at a particular step in a process, it may look very much like a traditional application, such as a Web page, a form in InfoPath, or even an instant message that reminds them they have work to do. As such, a process-driven application can act and look like one or more standard applications.
What sets a process-driven application apart from other applications is that it is aware of business rules and the next step in the process. For example, a user may create a Word document in a SharePoint library and fill in some data about that document but have no idea that the information contained within that document, coupled with the data they filled in about that document, allows the process engine to determine what happens next in the process. Process-driven applications can be very dynamic in this way, but of course, that depends on how they were originally designed.
K2 blackpearl provides all of the components to build simple to very complex and dynamic process-driven applications. It also provides the tools for business owners to see where an instance of a process is at any given time and to generate a report of all instances of a single process; all instances of multiple processes for a given department, user, or time of day; or all instances of every process in the organization. Based on that report, instances can be redirected to another user or delegated to a group of users depending on the business need. If the design of a process needs to be updated based on changes in the business, those changes can be made to the process. New instances of the process will reflect the changes in the process, and business owners may also choose to alter existing instances of the process with the new changes (often referred to as "in-flight" changes).
Realizing this vision of operational efficiency and agility is the future of business. BPM and process-driven applications is the road to that future. We contend that it is a future that requires the use of technology, simply because managing business processes and getting the level of transparency that is necessary to make sound business decisions can only be enabled by technology, even in small companies.
The following is a broad sketch of the landscape of the future of process-driven applications. Some of these capabilities are available today in applications that are not process-based, while others define a crucial aspect of process-driven applications that you will learn more about in the next chapter.
Applications that leverage existing technologies and employee skill sets to enhance business efficiency
Declarative process models that allow in-flight changes and business agility, decoupled from strict code-based models
Processes that expose business information in context with a user's task and enable corporate data to be retrieved and updated without the need to switch applications
Better insight and visualization of day-to-day business operations for decision makers
A robust, scalable, reliable, and secure platform that is easier for IT to manage because it is built using standard technologies
Loosely coupled integration with third-party servers and services to enable events, rules, and policy changes to affect processes without needing to alter the process itself
Rapid application design and testing with robust simulation capabilities that allows a business to model processes and estimate a return on investment before deploying the solution
Unlike many books in the Wrox Professional series, not all of the chapters in this book dive into code. This book is not meant for a developer-only audience, although developers will find a lot of useful information in this book, including code samples and discussions of the K2 platform API. However, because of the nature of the K2 platform and the fact that for many people the Business Process Discovery phase of implementing a BPM solution may be new, this book attempts to orient and educate both business users and developers about what it means to employ a BPM solution, in particular K2 blackpearl, successfully. Transforming a business to operate more efficiently and enabling better decision making are the key goals of this book. It is our hope that gaining confidence and knowledge of the K2 platform will allow a business process methodology to transform your business, and this book intends to address the full spectrum of ideas, methods, and information to make achieving those key goals a reality.
[1] At the time of this writing, the Host Server can load only servers and services signed by SourceCode Technology Holdings.
[2] "How does Windows Workflow Foundation (WF) compare to product X?" by Paul Andrew, Microsoft Corporation, 2007. http://blogs.msdn.com/pandrew/archive/2007/01/16/how-does-windows-workflow-foundation-wf-compare-to-product-x.aspx