I think there is a world market for maybe five computers. | ||
| --Thomas John Watson, Sr. (1874–1956), President, IBM | ||
User interface-level EAI, the most primitive of all EAI levels, is also the most necessary. Other EAI levels might be more technologically appealing and more efficient, but the simple reality is that, in many applications, the user is the only available mechanism for accessing logic and data. In spite of its inefficiency in "getting into" an application, the user interface has an advantage in that it does not require any changes to the source or target application.
In the context of user interface-level EAI, the user interface is the EAI interface. In a process known as "screen scraping," or accessing screen information through a programmatic mechanism, middleware drives a user interface (e.g., 3270 user interface) in order to access both processes and data. So, why does such a simple topic need an entire chapter? Simply put, many application integration projects will have no other choice but to leverage user interfaces to access application data and processes. But few are willing to admit it. Moreover, user interface-level EAI is very different from application interface- and method-level EAI, which are generally much more complex.
There really isn't much to user interface-level EAI. It is just one of many techniques and technologies that can be used to access, or place, information in an application. The technology has been around for a number of years. As a consequence, there is very little risk using it. There are, however, problems that need to be overcome. In user interface-level EAI, a user interface that was never designed to serve up data is being used for precisely that purpose. It should go without saying that the data-gathering performance of user screens leaves a lot to be desired. In addition, this type of solution can't scale, so it is unable to handle more than a few screen interfaces at any given time. Finally, if these mechanisms are not set up carefully by the EAI architect and developer, they may prove unstable. Controller and server bouncing are common problems.
There are, of course, a number of tricks for sidestepping these limitations, but they bring additional risk to the project. Still, with so many closed and proprietary applications out there, the EAI architect has little choice. The name of the game is accessing information—by any means possible. Ultimately, going directly to the user interface to get at the data is like many other types of technology—not necessarily pretty, but it gets the job done.
For the reasons noted previously, user interface-level EAI should be the "last ditch effort" for accessing information residing in ordered systems. It should be turned to only when there is no well-defined application interface (e.g., BAPI), such as those provided by many ERP applications, or when it doesn't make sense to leverage data-level EAI. Having said that, we can also state that user interface-level EAI needn't be avoided. In most cases, it will prove to be successful in extracting information from existing applications and as a mechanism to invoke application logic. Moreover, if performed correctly, there will be virtually no difference between using the user interface or a true application interface.
As in other levels of EAI, the architect or the developer has the responsibility to understand the application, the application architecture, and the database information in great detail. At the user-level interface, this may prove difficult. Remember, the decision to leverage user interface-level EAI was made specifically to bypass the restrictions of closed proprietary systems, or because it just makes good business sense not to augment an existing system to create an interface, or because other EAI levels are not applicable. In spite of the difficulty, this information is necessary for two reasons: one, the need to understand all the information that an application is producing, and two, to understand how all the data and methods residing within an application relate to all the data and methods existing within the enterprise. In short, it is necessary to know how the application data and methods fit within the enterprise metadata model and the enterprise object models, and map the source system into this logical and physical layer. Using the user interface as a point of integration does not free us from this requirement.
In order to implement user interface-level EAI, it is necessary to understand the application. This requires understanding the underlying data storage schema, much of the application logic, and, most importantly, how the information is presented to the user interface. In order to understand how the data elements are represented on the screen, it is necessary to understand how they exist within the application. Unlike other interface levels, information presented to a user interface may not map back to a database. Most of the data elements on the screen, such as calculated fields (e.g., an order total), are created by the application logic and do not come directly from the database. This being the case, it is evident that an understanding of the application logic is also desirable. This requires reading the documentation so that the mechanisms of the application—as well as the logic—can be understood. Unfortunately, as in other contexts, the applications are not always well documented. If this proves to be the case, the source code itself will have to be read. Ultimately, regardless of how it is achieved, the goal is to be able to trace all data that appears on the screen to the application logic and database schema information.
Reaching this goal will often require "breaking the data out" from its representation on the screen, a task accomplished with simple, mathematical formulas. There are times when it will be necessary to break out six separate data elements from a single number presented on the user interface. This, in and of itself, is a strong statement for the EAI architect or developer to understand the application logic in detail. There are many cases of user interface EAI projects where a failure to understand the application has led to a misreading of the user interfaces and, consequently, erroneous data being fed into target applications.
Once the database schema and application logic contained within the source or target application is understood, a catalog of all the information that appears on the user interfaces must be created. This task requires capturing each screen image and then cataloging each data element that appears on each screen. For example, a data element that appears next to the "Accounts Receivable" label would have to be cataloged by screen, label, and description. In addition to simply cataloging these data elements, the logic that creates them (e.g., mathematical equations, on the fly from user input) would also have to be documented, preferably from the application documentation, or source code.
Screens are typically cataloged by screen name, data element, description, database reference, and any applicable business logic (see Figure 5.1). This format allows for the efficient location of appropriate information from any source or target system that exists in the screen catalog. In addition, this information will facilitate an update of the enterprise metadata model.
In addition to the creation of a screen catalog of the information found in the user interface, it is necessary to map the positions of that same information. This process—screen mapping—determines the location of each data element on each screen, the number of positions it holds, and any other relevant information. This will better enable screen access technology to locate the appropriate information. For example, in order to find accounts receivable information, it would be necessary to know both its exact position on the screen where it appears, and how to process that information according to the information contained in the screen catalog.
The same piece of information can often be found at several locations in the application screens. For example, customer information may appear on every screen pertaining to updating a customer account. In screen mapping, the place that a particular data element is accessed is not important, but consistency is.
Having said all that, we should be quick to point out that extracting information from screens is not as straightforward as it may seem at first glance. There are two basic techniques to extract information from screens: static and dynamic. Each has advantages and limitations. Knowing how each works can optimize user interface EAI.
Static screen extraction means that, whichever technology is being used, the information is being accessed from a static point on the screen (e.g., Column 10, Row 12, Position 10). Consequently, there is no need to search the screen or employ a conditional logic to find the information.
The advantage of static extraction lies in its simplicity. Without the need to create a logic for each screen extraction, acquiring the required information simply means going to the predetermined location and grabbing the information. Simple. But the downside is profound. If the application interface changes for any reason (e.g., by adding more information), then the position where the data element once resided may not continue to be the position where it resides. As a result, the EAI screen extraction process may extract no information, or worse, the wrong information. Clearly, this method of extraction should only be used for applications that never, or rarely, change.
Dynamic extraction is a more sophisticated process than static extraction. It gives EAI architects and developers the ability to look at a screen using conditional logic. For example, rather than going directly to a fixed position when looking for accounts receivable information, this technique would search for the "accounts receivable" label and would use that as a reference point.
The obvious advantage to dynamic extraction is the ability to automatically adapt to changes within an application and react to changes in the structure of the screen. Another advantage of dynamic extraction is the ability to place specific logic sequences in the extraction process, such as a logic to flag errors if a particular label is not found, or one that determines if a data element does not make sense.
Both the dynamic and the static extraction techniques must go through a rudimentary error-checking routine in order to ensure that only valid data is extracted. This routine takes place after the data has been extracted from the screen. It looks for easy-to-spot problems, such as character information in a numeric data element.
The goal of error processing is to alert the appropriate person to a problem, so that it can be corrected before bad data is extracted. Error-processing routines are vital to user interface-level EAI.
There are two approaches for getting information from application interfaces: using screens as raw data and using screens as objects. Both extract the information by using the interface as if the program was an actual user, and reading the information to the virtual interface. Naturally, it is impossible to get all necessary information from a single user interface. Most user interface-level EAI efforts will require as many as a thousand instances of a user interface in order to obtain the correct information in the correct volume (see Figure 5.2).
The screen-as-data scenario involves looking at a screen as a simple stream of text. The information is extracted and then incorporated into the program as a string of text. Once there, the information is parsed, identified, converted, and processed within the program that is responsible for processing the user interface information (e.g., the message broker itself, or an adapter).
The advantage to screens-as-data is simplicity. Information never really changes states and therefore remains easy to comprehend. In addition, the screen information is never tied directly to methods. Unfortunately, the very advantage of this method—its simplicity—is also its disadvantage. Because only information is being tracked—and not the methods that act upon that information—only half the story is being told. The other half requires the screens-as-objects scenario.
The screens-as-objects scenario is much more sophisticated than the screens-as-data scenario. It may even be too sophisticated for most user interface-level EAI projects. Using screens-as-objects requires translating the information gathered from a user interface into an application object, either a Java object, a CORBA, or a COM ORB. Translating the information into objects requires adding the methods needed to interact on the data (see Figure 5.3).
Figure 5.2. Using user interface-level EAI may require running hundreds of instances of the source, or target application's, user interface.
The advantage of using screen objects is the ease with which they can be placed into other environments that support objects, among them application servers and message brokers. These environments can act on the screen objects like any other application object. Moreover, developers can extend the capabilities of the objects to meet the exact requirements of the EAI problem domain or application.
Examples of enabling technology that supports user interface-level EAI include terminal emulation software, middleware, and software development kits (SDK).
Most user interface-level EAI projects will leverage 3270 (used by most IBM-type mainframe users) or 5250 (used by most AS/400 users) terminal emulation software as a mechanism to access user interface information. Either product generally comes with two components: the emulator and the SDK to automate screen access operations, including extraction.
There are three technical approaches to extracting information from user interfaces: 3270/5250 user interface access using HLLAPI (High Level Language Application Program Interface), accessing ASCII terminals such as VT100s, accessing Windows-based GUIs using OLE automation, and converting screens into objects. While a detailed discussion of each approach is beyond the scope of this book, it's helpful to look briefly at each.
HLLAPI is an IBM programming interface allowing PC applications to communicate with a mainframe application through 3270 emulation. The HLLAPI interface is able to read and write to 3270 and 5250 screens as well as process errors and manage the connection. HLLAPI is shipped with most PC-based 3270 emulators on the market today; programming HLLAPI is something that's been going on for well over ten years, and we're pretty good at it. This is the most popular screen extraction mechanism for user interface-level EAI, and many EAI solutions, such as message brokers and application servers, use 3270 adapters (using HLLAPI) to read and write information from and to 3270 and 5250.
ASCII or ANSI terminals, such as VT100 and VT220, support simple screen scraping as well, but instead of a 3270 data stream, they leverage an ASCII data stream used by these terminals that are connected to host-based UNIX systems and DEC minicomputers. The extraction mechanism is an API that works with the VT emulator software (e.g., TELNET) to locate and extract screen information.
Of course, while 3270 and VT100 terminals were the preferred user interface several years ago, Windows has long since taken over. Extracting information from applications that run within the Windows Win32 environment is a necessity from time to time. This is done, generally, through OLE automation. OLE automation enables a client application to communicate with a server application, allowing it to exchange information between applications easily. Thus, it's a simple matter of creating an application that's able to read the application information (exposed by OLE automation) from the Windows application that we're interested in. We implement OLE automation through any number of programming tools that support it, including Visual Basic and Visual C++. We'll discuss the use of OLE automation and Microsoft's COM (Component Object Model) in more detail in Chapter 10. OLE automation and COM also lend themselves well to method-level EAI.
Screens as objects use any one of the user interface communications mechanisms explained previously but go a step further by encapsulating the data and the methods that act upon that data into a binary or program object. This may be a CORBA object, a COM object, a C++ object, or a Java object depending on what you're interacting with. The advantage of this approach is that the screen information is transformed into a set of objects. These objects are easily queried by other applications as true objects and not as a simple stream of data. This approach is much more sophisticated and is the future of user interface-level EAI; it binds user interface-level EAI to method-level EAI, which makes use of objects as a mechanism to share methods.