This chapter discusses the [EIP] concept of Message, how it is manifested in Java CAPS, and the related Java CAPS concept of Object Type Definitions (OTDs). It discusses briefly how OTDs can be built from database tables, views, and stored procedures; XML Schemas and other source formats; how to marshal (serialize) and unmarshal (deserialize) them; how to manipulate them; and how to use them for transformations and connector operations.
The [EIP] Envelope Wrapper pattern is also discussed. Diverse implementations of the pattern, intended to convey metadata with message payload, use different Java CAPS facilities ranging from a user-defined custom Envelope Wrapper OTD, to carrying unparsed XML within an XML Envelope, to ways of passing metadata using JMS message header properties.
Integration solutions typically collect messages from external systems and, after due transformations and routing, deliver messages to external systems. Both the sending and receiving external systems use message formats that are specific to them. The implementation of the interface between the external system and the messaging infrastructure requires that formats of messages to be exchanged over it be agreed upon. Any unforeseen variations from the agreed message formats will likely break some part of the implementation.
While some solutions involving an integration infrastructure are designed such that an external system tells the messaging solution how to transform or route the message, such solutions do not leverage the integration infrastructure’s capabilities and therefore do not obtain the benefits of using it. It is much more likely that the message payload is all that is exchanged over the “external system–integration solution” interface and that the message payload structure is all that is agreed upon.
Java CAPS message structures, data structures that Java CAPS components manipulate, are called Object Type Definitions (OTDs, XML Schema Definition [XSD] objects, and Web Services Description Language [WSDL] objects, depending on the type).
Unlike some other tools, Java CAPS does not use a predetermined data representation, such as XML, to represent data internally. On the contrary, in general, whenever an external message uses a native standardized data representation (XML, HL7, SAP IDOC, etc.) as distinct from a Java object, data from a data source such as a database, or data from a format-agnostic endpoint, Java CAPS uses that native format until told otherwise.
To allow components to manipulate fields within messages, the Java CAPS developer would explicitly deserialize (unmarshal) data from its native format, whatever that may be, into a Java class hierarchy for Java components such as Java Collaboration Definitions (JCDs) and into XML Document Object Model (DOM) structures for XML-based components such as XSLT Collaboration Definitions and Business Process Execution Language for Web Services (BPEL4WS)-based Business Processes. Both marshal and unmarshal methods are provided as part of the OTD when the OTD is generated or built. If a component does not need to inspect or manipulate data fields, it does not need to unmarshal data into an OTD. This, depending on the design of the solution, may result in significant efficiency gains over solutions where data is always converted from a native representation into an internal representation, such as XML.
Note
In recent years, the Extensible Markup Language (XML) gained popularity as a universal data representation format. Some zealots went to the lengths of using XML to implement procedural languages (BPEL4WS), databases, and all manner of things to which XML has dubious applicability and where XML introduces considerable overheads for little or no real gain. Whenever designing integration solutions, particularly ones where message throughput and processing efficiency are important, consider very carefully the appropriateness, or otherwise, of using XML as the underlying data representation. In a recent project, where HL7 delimited messages were converted to an alternate XML-based format, message size increased an average of 19 times over 5,700 messages processed! If issues of this sort are a consideration, do not use XML.
There are two basic types of OTDs: Connector OTDs and Message OTDs. Connector OTDs are associated with specific endpoints, such as eWays or JMS connectors. In addition to representing message and metadata, these OTDs expose methods that allow developers some control over connectivity aspects of the related adapters. The JMS object, used later in the user properties discussion, is an example of the Connector OTD. It exposes the send(), the sendTo(), and similar methods that exercise specific functionality of the underlying connector. Other Connector OTDs, such as database eWay OTDs, expose other methods pertinent to the connectors they represent. Connector OTD frameworks are installed together with the corresponding eWay or as part of the JMS messaging implementation. In many cases—for example, SAP or Database eWays—OTDs will be generated as part of solution development to represent specific IDOCs or database tables.
In contrast, Message OTDs represent message structures and facilitate deserialization (unmarshaling), serialization (marshalling), and access to components of messages. Message OTDs can be constructed as user-defined OTDs or generated from external source forms. Manual construction of user-defined OTDs is covered extensively in [eDesigner]. Generation of Message OTDs from source forms is documented in a number of places in the Java CAPS document set—for example, database eWay documentation for database OTDs.
To give you a feel for the process, this section walks through generation of a Message OTD for a table in an Oracle database. Assume the existence of a standard Oracle 9i installation containing the Scott schema with the EMP table. Listing 7-1 shows the Data Definition Language (DDL) for the EMP table, if you don’t have the Scott schema.
Example 7-1. SCOTT Schema EMP table DDL
CREATE TABLE "SCOTT"."EMP"
( "EMPNO" NUMBER(4,0),
"ENAME" VARCHAR2(10),
"JOB" VARCHAR2(9),
"MGR" NUMBER(4,0),
"HIREDATE" DATE,
"SAL" NUMBER(7,2),
"COMM" NUMBER(7,2),
"DEPTNO" NUMBER(2,0),
CONSTRAINT "PK_EMP" PRIMARY KEY ("EMPNO")
USING INDEX PCTFREE 10 INITRANS 2 MAXTRANS 255
STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT)
TABLESPACE "SYSTEM" ENABLE,
CONSTRAINT "FK_DEPTNO" FOREIGN KEY ("DEPTNO")
REFERENCES "SCOTT"."DEPT" ("DEPTNO") ENABLE
) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING
STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT)
TABLESPACE "SYSTEM"Right-click on a project name where you need the Oracle table OTD, and choose New → Object Type Definition.
In the resulting dialog, shown in Figure 7-1, choose Oracle Database, and press Next.
Provide credentials for the database schema that contains the EMP table, as shown in Figure 7-2, and then click Next.
Note
The mechanism underlying this dialog uses a Java Database Connectivity (JDBC) driver to query database metadata.
Check the Tables/Views/Aliases checkbox, as shown in Figure 7-3, and click Next. Note that using this dialog sequence you could also generate an OTD from a stored procedure or a prepared statement.
Click the Add button to add tables/views/aliases to the OTD that will be produced. Choose the schema, Scott in this case, and press Search. Select the EMP table, and press OK, as shown in Figures 7-4 and 7-5.
Press Next, name the OTD “tblEMP,” and press Next, as shown in Figure 7-6.
Review the results and complete the Wizard by pressing the Finish button, as shown in Figure 7-7.
The OTD is generated and displayed in the OTD Editor Canvas, as shown in Figure 7-8.
Notice a field for every column in the table, with the name that corresponds to the name of the underlying column. In addition, there is a series of methods, to be used in a JCD, and a series of services, to be used in a BPEL-based eInsight Business Process, for accessing and manipulating data represented by the OTD. This is pointed out in Figure 7-9.
Note
Java methods allow much more detailed control over the operations performed through the OTD, concurrency and transactional aspects of the database connection, and the resultset manipulation, than is possible in BPEL. Since you can invoke a JCD as a service from BPEL, if fine control over the database access is required, develop a New Web Service JCD to handle database access.
Note
The OTD provides prebuilt BPEL services for most common database operations—select one row, select multiple rows, select all rows, insert, update, and delete—thus greatly simplifying the developer’s work.
The OTD that represents the Oracle Scott EMP table is ready. You can now query the database and insert, update, and delete rows. Needless to say, the underlying database will reflect the changes you make through this Connector OTD.
Figure 7-10 lists some of the available OTD Wizards.
This list represents the wizards installed with the eWays and other components in the product installation used for developing examples for the book. Additional products and connectors provide additional OTD Wizards. Some notable omissions are SAP, DB2, PeopleSoft, and other specialized products available as part of the suite but not installed in the working environment.
Some systems use messages that consist of business and control data in the loosest meaning of the terms. Such messages are sometime structured to consist of a header, which contains control data, and a body, which contains business data. This header/body structure is a specialization of a general enveloped message wherein a business payload is enclosed within a message containing other, non-payload data.
Envelope Wrapper in Java CAPS is merely an OTD whose structure accommodates both the header/trailer requirements and the payload requirements. When designing an Envelope Wrapper OTD, you must consider the structure and size of the payload and the cost of marshaling and unmarshaling it. If the complexity of the header/trailer is low, and frequency of access to the header/trailer components is high, compared to the complexity of the payload and frequency of access to payload components, then the Envelope Wrapper OTD should be constructed such that the payload node in the OTD is opaque and does not get parsed when the OTD is unmarshaled. This will save the resources and time required for parsing the payload when it is not used at the cost of having to develop a payload OTD, which likely will have to be developed anyway, and the cost of explicitly unmarshaling the payload into the payload OTD when required.
Picture a payload with the structure shown in Listing 7-2.
Picture also an Envelope Wrapper with the structure shown in Listing 7-3.
Example 7-3. Envelope structure
POEnvelope
SourceSystem
DestinationSystem
SequenceNumber
Duplicate
PayloadYou could construct a user-defined, delimited OTD, with two levels of delimiters, which looks like that shown in Figure 7-11, to represent the enveloped Purchase Order, where delimiters are “|” and “:”, as shown in Figure 7-12.
Since the payload is simple, there would be little penalty in unmarshaling this structure every time access to the SourceSystem or DestinationSystem node was required.
A Java Collaboration would perform an unmarshal operation to get access to fields in the OTD, as shown in Figure 7-13.
If the Purchase Order structure was more complex, perhaps containing multiple repeating items with many fields per item, unmarshaling such a structure each time access to the header fields was required, but where no access to the purchase order fields was required at all, would be a major and unnecessary overhead.
Picture two OTDs: An Envelope OTD, shown in Figure 7-14, with a single level of delimiters (“|”), shown in Figure 7-15.
Picture also a Purchase Order OTD with a single level of delimiters (“:”), shown in Figures 7-16 and 7-17.
A Java Collaboration would unmarshal the input node into the udtPOEnvelope to get at the Header fields and, if needed, would unmarshal the POPayload field into the udtPurchaseOrder to get at the Purchase Order fields themselves. Figure 7-18 shows the two OTDs included in the Java Collaboration at design time.
First, enveloped input is unmarshaled into the Envelope Wrapper OTD, as shown in Figure 7-19.
Then payload is unmarshaled from the POPayload node of the Envelope Wrapper into the Purchase Order OTD, as shown in Figure 7-20.
Thus, if access to the Purchase Order details is not required in a particular collaboration, the second unmarshal is not performed and the cost of parsing the Purchase Order data is not incurred.
The content of the POPayload could be XML as easily as delimited. The Envelope Wrapper could also be an XML structure. If the envelope and the payload were XML structures, then the payload node in the envelope structure would have to be of type “any” in order to allow well-formed XML to be transported unparsed.
An issue of enveloping XML within XML comes up reasonably commonly in projects that make extensive use of XML and Common Message Model concepts. Usually it is couched in terms of a need for a structured XML message with a common Header section and a variable Body section. The XML Schema specification offers a few solutions.
Take an XML Schema document, OpaqueXMLPayloadEnvelope.xsd, which looks like that shown in Listing 7-4.
Example 7-4. Opaque XML Payload Envelope XML Schema
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="OpaqueXMLPayloadEnvelope">
<xs:complexType>
<xs:sequence>
<xs:element name="Header">
<xs:complexType>
<xs:sequence>
<xs:element name="Type" type="xs:string"/>
<xs:element name="Hdr02" type="xs:string"/>
<xs:element name="Hdr03" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="XMLPayload">
<xs:complexType>
<xs:sequence>
<xs:any
processContents="skip"
minOccurs="0"
maxOccurs="unbounded"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>Note the element XMLPayload of type “any” and processContents attribute of “skip.” This element can contain any well-formed XML except XML processing instructions [XMLSchema]. The XML Schema specification uses the term wildcard. We exploit this to create an OTD that allows arbitrary well-formed XML to be contained in a node of a structured OTD without having to define the structure of that node.
Create an XSD-based OTD, as shown in Figure 7-21.
Note an element called XMLPayload. Its optional, repeating subnode __AnyText__ will carry the well-formed XML.
Take an XML Schema document, XMLPayloadType_A.xsd, which looks like that shown in Listing 7-5.
Example 7-5. Payload Type A XML Schema
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="XMLPayloadType_A">
<xs:complexType>
<xs:sequence>
<xs:element name="PA_OrderNumber" type="xs:integer"/>
<xs:element name="PA_OrderQuantity" type="xs:decimal"/>
<xs:element name="PA_Description" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>Create an XSD-based OTD, as shown in Figure 7-22.
Take an XML Schema document, XMLPayloadType_B.xsd, which looks like that shown in Listing 7-6.
Example 7-6. Payload Type B XML Schema
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="XMLPayloadType_B">
<xs:complexType>
<xs:sequence>
<xs:element name="PB_OrderNumber" type="xs:integer"/>
<xs:element name="PB_OrderItems" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="ItemNumber" type="xs:integer"/>
<xs:element name="ItemDescription" type="xs:string"/>
<xs:element name="ItemQuantity" type="xs:integer"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>Create an XSD-based OTD, as shown in Figure 7-23.
Test by invoking the OTD Tester, as shown in Figure 7-24, on the OpaqueXMLPayloadEnvelope OTD and making it unmarshal two files, one containing Type A XML Instance document and one containing Type B XML Instance document.
Listing 7-7 shows the Type A XML Instance document.
Example 7-7. Type A XML instance document
<?xml version="1.0" encoding="UTF-8"?>
<OpaqueXMLPayloadEnvelope
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Header>
<Type>A</Type>
<Hdr02>Header 02 content</Hdr02>
<Hdr03>Header 03 content</Hdr03>
</Header>
<XMLPayload>
<XMLPayloadType_A
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<PA_OrderNumber>1234567890</PA_OrderNumber>
<PA_OrderQuantity>5</PA_OrderQuantity>
<PA_Description>Order Header for Order 1234567890</PA_Description>
</XMLPayloadType_A>
</XMLPayload>
</OpaqueXMLPayloadEnvelope>The OTD Tester window after the document is parsed will look like that shown in Figure 7-25.
The Type B XML Instance document is shown in Listing 7-8.
Example 7-8. Type B XML Instance document
<?xml version="1.0" encoding="UTF-8"?>
<OpaqueXMLPayloadEnvelope
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Header>
<Type>B</Type>
<Hdr02>Header 02 content</Hdr02>
<Hdr03>Header 03 content</Hdr03>
</Header>
<XMLPayload>
<XMLPayloadType_B
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<PB_OrderNumber>1234567890</PB_OrderNumber>
<PB_OrderItems>
<ItemNumber>1</ItemNumber>
<ItemDescription>First Item</ItemDescription>
<ItemQuantity>12</ItemQuantity>
<PB_OrderItems> </PB_OrderItems>
<ItemNumber>2</ItemNumber>
<ItemDescription>Second Item</ItemDescription>
<ItemQuantity>7</ItemQuantity>
</PB_OrderItems>
</XMLPayloadType_B>
</XMLPayload>
</OpaqueXMLPayloadEnvelope>The OTD Tester window after type B document is parsed is shown in Figure 7-26.
In a Java Collaboration or an eInsight Business Process, use the OpaqueXMLPayloadEnvelope_OpaqueXMLPayloadEnvelope OTD to unmarshal the XML Instance document in order to access envelope header elements. Once you determine payload type from the value of the OpaqueXMLPayloadEnvelope/Header/Type node you can choose the appropriate payload OTD into which you would unmarshal the opaque payload. In the previous examples, it would be either the XMLPayloadType_A or XMLPayloadType_B.
Remember that this technique allows you to envelope arbitrary well-formed XML within a node of another XML document and unmarshal the envelope without having to unmarshal the payload.
A similar technique, exploiting the XML Schema specification, can be used to create an XML Envelope that can carry both well-formed XML and non-XML data within a node.
Picture an XML Schema that looks like that shown in Listing 7-9.
Example 7-9. Opaque Payload Envelope sample schema
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema
xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="OpaquePayloadEnvelope">
<xs:complexType>
<xs:sequence>
<xs:element name="Header">
<xs:complexType>
<xs:sequence>
<xs:element name="Type" type="xs:string"/>
<xs:element name="Hdr02" type="xs:string"/>
<xs:element name="Hdr03" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Payload">
<xs:complexType mixed="true">
<xs:complexContent mixed="true">
<xs:restriction base="xs:anyType">
<xs:sequence>
<xs:any processContents="skip"
minOccurs="0"
maxOccurs="unbounded"/>
</xs:sequence>
</xs:restriction>
</xs:complexContent>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>When you build an OTD from a schema containing the construct italicized in Listing 7-9, the part of an XML document instance between <Payload> and </Payload> can contain any well-formed XML or non-XML text (other than XML processing instructions)—the unmarshaler will deliver the unspecified content in a node called __AnyText__.
For example, an XML document that looks like the sample in Listing 7-10 will parse, using the OTD Tester, into the OTD structure shown in Figure 7-27.
Example 7-10. Sample XML Instance document that carries unparsed non-XML payload data
<?xml version="1.0" encoding="UTF-8"?>
<OpaquePayloadEnvelope
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Header>
<Type>C</Type>
<Hdr02>Header 2</Hdr02>
<Hdr03>Header 3</Hdr03>
</Header>
<Payload>
Litwo! Ojczyzno moja! ty jestes jak zdrowie.
Ile cie trzeba cenic, ten tylko sie dowie, Kto cie stracil.
Dzis pieknosc twa w calej ozdobie Widze i opisuje,
bo tesknie po tobie.
</Payload>
</OpaquePayloadEnvelope>
An XML document, which is presented in Listing 7-11, will also parse, using the OTD Tester, into the same OTD structure, as shown in Figure 7-28.
Example 7-11. Sample XML Instance document that carries unparsed XML payload data
<?xml version="1.0" encoding="UTF-8"?>
<OpaquePayloadEnvelope
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Header>
<Type>D</Type>
<Hdr02>Header 2</Hdr02>
<Hdr03>Header 3</Hdr03>
</Header>
<Payload>
<XMLPayloadType_A
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<PA_OrderNumber>1234567890</PA_OrderNumber>
<PA_OrderQuantity>5</PA_OrderQuantity>
<PA_Description>Order Header for Order 1234567890</PA_Description>
</XMLPayloadType_A>
</Payload>
</OpaquePayloadEnvelope>
These are some of the techniques that can be used with XML-based Envelope Wrappers.
A designer of the integration solution could define an enveloped message structure and marshal and unmarshal the payload into or from that structure in each component where specific access to metadata and payload was required.
In addition to the message payload and standard JMS message headers, such as JMSMessageID, JMSTimestamp, JMSCorrelationID or JMSExpiration, each JMS message can carry a collection of arbitrary properties, expressed as name/value pairs.
In solutions where messages are conveyed from component to component over JMS, which would be the majority of Java CAPS–based solution, the designer could use JMS user properties to pass between components the metadata that is not part of the payload but is required by components to do their work. In effect, a JMS message is already an envelope-wrapped [EIP] message. Unlike for a typical envelope-wrapped message, the designer does not need to define the envelope structure (JMS implementation already defines that) or marshal and unmarshal messages explicitly. Further, the Java CAPS implementation provides convenient accessors that allow arbitrary JMS user properties to be created and accessed. A solution can leverage that infrastructure to implement an implicit Envelope Wrapper and pass around metadata without modifying the payload or constructing and manipulating an enveloped message.
Chapter 5, section 5.2.1, “JMS User Properties Envelope Wrappers,” in Part II (located on the accompanying CD-ROM), illustrates the use of JMS user-defined properties in Java Collaborations and eInsight Business Processes with several examples.
This chapter discussed the [EIP] concept of Message, how it is manifested in Java CAPS, and the related Java CAPS concept of Object Type Definitions (OTDs). It discussed and illustrated how OTDs can be built from database tables, views, and stored procedures; XML Schemas and other source formats; and how to marshal (serialize) and unmarshal (deserialize) them.
The [EIP] Envelope Wrapper pattern was discussed and illustrated with examples. Diverse implementations of the pattern used different Java CAPS facilities ranging from user-defined custom Envelope Wrapper OTDs, to carrying unparsed XML within an XML Envelope, to ways of passing metadata using JMS message header properties.