In the early days of the OO language wars, it was unclear which languages would survive and become the dominant players among OO developers. One of the early leaders, which did not gain the wide adoption of some other languages like C++ and Java, was Borland’s Delphi. Delphi was based on Object Pascal, and first released in 1995. By 1996, Delphi featured a set of database programming tools that encouraged more of a Transaction Script approach than a true object-relational mapping. Programmers would query for and manipulate Row objects that represented the rows in a relational table. Delphi featured forms-based programming and nonvisual relational database management system (RDBMS) components that would plug into its visual form builder tooling.

Because C++ had wide adoption, many libraries attempted to address the relational mismatch. Rogue Wave is still one of the most popular C++ library providers around. Their DBTools libraries attempted to address the object relational problem by overloading the stream inputs and outputs to accept SQL. SQL queries could then be projected onto objects and vice versa. Although programmers would write their own SQL, it allowed one to think about object representation of their domain. Today, many developers still feel comfortable writing SQL, and modern frameworks like iBATIS take advantage of this fact.

In 1992 the developers at NeXT (founded by Steve Jobs) released their first stab at addressing the issue of connecting objects written in Objective-C, the primary programming language of the NeXT and NeXTStep, to relational databases. This effort resulted in the DbKit, which was an abstraction layer that allowed access to different relational databases without the need to write database-specific code. This approach presaged many later efforts such as JDBC and similar frameworks. In time (and after at least one complete rewrite), the NeXTStep developers released the Enterprise Object Framework (EOF), which eventually became a complete object-relational mapping framework.

Our story really begins in 1995, however, with the release of the initial version of the TopLink object-relational mapping tool by The Object People, a small Canadian Smalltalk consulting company [Smith]. At the time, an advanced object-oriented development community of professors and students had formed at Carleton University, resulting in the creation of a number of small but important companies in the history of OO development. TopLink wasn’t the first OR mapping tool for Smalltalk (for instance, the ParcPlace ObjectLens toolset, which was included as part of the ParcPlace VisualWorks development environment for Smalltalk, was released in 1993), but it differed from some of the earlier tools for Smalltalk in that it contained nearly all the features now expected in an ORM tool. For instance, TopLink 1.0 contained the following:

This toolset was the standard by which other Smalltalk relational database tools were judged, and it implemented many of the patterns that are now standard practice in Java object-relational mappers as well.

But TopLink wasn’t the only game in town, even in Smalltalk. IBM had begun supporting the Smalltalk language with the purchase of Object Technology International, another small Canadian company founded by students and professors from Carleton University. OTI developed Smalltalk technology—which became the basis of IBM’s VisualAge^® for Smalltalk product. Seeing the need to provide relational database connectivity to its Enterprise customers, IBM developed its own object-relational mapping tool for VisualAge for Smalltalk, which was internally used with several customers before it was released as part of VisualAge (called the ObjectExtender) in 1998. Even though IBM was a late entry into the Smalltalk persistence game, their experience is important to the story, as discussed later. The ObjectExtender contained many of the same types of features as TopLink:

Using the Java Database Connectivity API (JDBC) is often a person’s first exposure to manipulating a database with Java. JDBC 1.0 introduced most of the ideas that are still found even in the most recent JDBC 4.0 specification. Note that these are not new concepts first introduced in JDBC; JDBC is itself based off of Microsoft’s ODBC API, which was in turn based on the X/Open SQL CLI (Call Level Interface). In short, there are three major concepts:

In JDBC 1.0 you could only move forward through a ResultSet. JDBC 2.0 (released with JDK 1.2 in December 1998) expanded this to allow backward movement through a ResultSet, and also allowed changes to the values of the ResultSet (which result in implicit SQL UPDATE statements). Simultaneous with the release of JDBC 2.0 was the release of the first JDBC Standard Extension, which introduced concepts such as database pooling, accessing named pools through the Java Naming and Directory Interface API (JNDI), and support for two-phase commit transactions through the use of the Java Transaction API (JTA) or Enterprise Java Bean components (EJBs).

Given their experience with TopLink for Smalltalk, in 1996 the TopLink team began work on a version of TopLink for Java. The product officially came available in 1997. The TopLink ORM for Java contained all the same types of features as TopLink for Smalltalk, and over the years it would also come to support multiple application server vendors as well as working standalone. TopLink was acquired by WebGain in 1999 and became part of Oracle in 2002.

In late 1997 and 1998, engineers from Sun, IBM, and many other companies began working together and releasing early versions of what would become the first Enterprise Java Bean (EJB) specification. The EJB specification was an ambitious project to redefine the way in which distributed applications were built. The EJB specification attempted to build from the foundation laid by the CORBA specifications and the lessons learned from the Java RMI specification to result in a framework for building enterprise applications that would combine distribution, persistence, and two-phase commit cross-datasource transactions.

From a persistence point of view, the EJB 1.0 specification introduced the idea of an entity bean, which was an EJB component that exposed information contained in external datasources as first-class Java objects. There were two flavors of entity beans described in the first EJB specification: those with “bean-managed” persistence (BMP), and those with container-managed persistence (CMP). In a BMP, logic implementing persistence would be explicitly coded by the developer, using facilities like JDBC, within the methods of the entity EJB. A CMP made the logic associated with persistence implicit, with the promise that vendors would provide tools and frameworks to provide object-relational mapping.

The initial EJB specification left a lot of implementation details open to interpretation for the vendors to fill in. Somewhat representative of the very early EJB persistence implementations was the VisualAge Persistence Builder (or VAP), released in 1998. This product was the first IBM toolkit for creating persistent Java objects similar to EJB components. It was developed by the same team that developed the ObjectExtender framework, and shared a number of the same design principles and goals.

VAP occupied an interesting niche that differentiated it from some other early EJB 1.0 implementations; it wasn’t part of a full EJB container, but instead was a standalone EJB runtime environment designed to be upward-compatible with the architecture of IBM’s EJB server environments. Like several other persistence solutions that followed, it was a server environment that could be plugged into a Servlet or EJB container to provide persistence.

The way in which VAP achieved this “amibidextrous” nature is by beginning with a set of tools built in the VisualAge for Java Enterprise Edition environment that were capable of generating objects (that were, in fact, Plain Old Java Objects or POJOs). These POJOs implemented the entity bean interfaces, together with other objects that implemented the EJB Home interfaces, and the other parts of its persistence implementation. VAP code would be packaged into a JAR file that would then run on top of the VAP runtime framework, which, as noted earlier, could run inside a Servlet or EJB 1.0 container, or could also run within a Java application.

VAP supported almost all the standard ORM features, such as these:

However, the very strength of VAP (its relationship support) hid a number of issues that developers soon discovered with the EJB 1.0 specification. For example, the specification did not include details of how relationships were to be described for CMP entity components. As a result, each vendor’s approach to capturing this important information in a “deployment descriptor” was different. Another issue that people soon ran into with the EJB 1.0 specification was that EJBs were always considered remote objects. This made it possible, for instance, to access the individual attributes of an entity EJB (which represented a single database table row) one-at-a-time across the network, something that was neither efficient nor scalable. These drawbacks, among others, led to the EJB specification being significantly revised in EJB 2.0.

EJB 2.0 [EJB 2] was finally released in 2001 (after a long revision process that included two separate final drafts that differed significantly from one another). Although EJB 2.0 introduced a number of new features such as Message Driven Beans that do not concern this story, a major portion of the revised sections of the specification dealt with persistence. In short, EJB 2.0 attempted to improve the poor portability situation of EJB 1.0 by describing relationship support directly within the specification. EJB 2.0 accomplished its relationship support by substantially enlarging the role of the XML deployment descriptor that had been introduced as a class in the EJB 1.0 specification, and then changed into an XML file in the EJB 1.1 specification (1999).

At the heart of this was the simple idea that a substantial portion of the description of how the Container-Managed Persistence Entity bean is implemented can be described in an XML file. EJB 1.1 had introduced the idea that the persistent fields of an Entity bean can be described in the EJB-JAR.xml deployment descriptor using the <cmp-field> element. EJB 2.0 extended this idea by giving the responsibility for generating the actual concrete classes that implemented those persistent fields to the EJB container, which used the field-level information in the deployment descriptor (plus the information in the EJB Entity abstract class) to build the persistence layer. It was in representing links between Entity classes that the EJB 2.0 spec took a step forward. By introducing the <ejb-relation> element and <ejb-relationship-role> element, the EJB 2.0 spec allowed the definition of relationships with varying multiplicity (1:1, 1:N, or M:N) between entity classes.

The second major element added in the EJB 2.0 specification that was applicable to persistence was the introduction of the Local EJB interface. As noted previously, in EJB 1.0 and 1.1 all EJBs were remote—a feature that makes sense for large-grained Session beans, but not for fine-grained Entity beans. By introducing Local EJB interfaces, EJB 2.0 allowed the developer to declare his Entities to be local only—and thus accessible only within a transaction controlled by an external Session bean. This approach made explicit some of the best practices and patterns that had been developed for EJB 1.0 and 1.1—most notably how Distributed Façade in Fowler [Fowler] was applied to EJB components and known as Session Façade by Deepak Alur [Alur] and Kyle Brown [Brown] in their books on J2EE design patterns.

The Hibernate story begins in November 2001 when Gavin King started a SourceForge project to build an open-source object-relational database mapping system for Java. Given the dissatisfaction with the EJB 2.0 specification at the time, and the paucity of other cheap object-relational mapping solutions, the Hibernate project began attracting a significant amount of interest. By mid-2002 the project was well-established enough to announce a 1.0 version with considerable fanfare on sites like TheServerSide. Almost a year later, in June 2003, Hibernate 2.0 was released, which was a fully featured, robust system that was on par with commercial ORM offerings and that also featured tooling available on several open-source frameworks such as NetBeans and Eclipse. In October 2003, Hibernate “hit the big time” when the lead Hibernate developers were hired by JBoss.

Several simple principles underlie the success of Hibernate and its influence on later ORM specifications and frameworks. Chief among them is the fact that Hibernate is extremely lightweight, and that the programming model is very simple—Hibernate is based on a POJO model, and you do not have to implement special interfaces or extend special classes to render your classes persistent.

Some of its design principles we have already seen in the development of the EJB 2.0 specification—for every persistent class in Hibernate, there are corresponding entries in an XML mapping document that (for instance) lists the names and types of the persistent attributes of that class. Likewise, to represent collections (a 1:N relationship), you would simply add an element in the mapping document that declares a set, list, or other collection. However, one unique part of the Hibernate mapping file is that it contains all the information necessary to determine the mapping to a relational database. Table and column names are defined along with class and attribute names. This “all in one” approach made it possible to implement persistent objects quickly and easily. Another key aspect is that Hibernate can run inside or outside of an EJB container. Because of this, Hibernate is less restrictive and more complete in mapping object-oriented concepts to relational databases.

Perhaps the most important contributor to Hibernate’s success was its price—free. Because it was an open-source project, Hibernate was easy and cost-effective for developers to try first and then decide they liked both it and the idea of object-relational mapping. Although the EJB spec did much to make object-relational mapping acceptable to the Java development community, it was Hibernate that popularized it among developers.

The iBATIS framework took a different route than Hibernate. Rather than hiding SQL, iBATIS created a persistence framework much like Rogue Wave’s DBTools, in which SQL statements were projected to and from POJOs. The iBATIS framework grew out of a JPetStore Demo in response to an early 2002 Microsoft published paper claiming that .Net was 10 times faster and 4 times more productive than J2EE. Realizing that this was simply not the case, the iBATIS project (which previously had been focused on cryptographic software solutions) quickly responded with the JPetStore 1.0 release. Based on the same Pet Store requirements, JPetStore demonstrated that Java not only could be more productive than .Net, but also could do so while achieving a better, more open architecture than was used in the Microsoft implementation.

JPetStore made use of an interesting persistence layer that quickly captured the attention of the open-source community. Shortly after JPetStore was released, questions and requests for the SQL Maps and Data Access Object (DAO) frameworks spawned the project that would become known as iBATIS Database Layer. The iBATIS Database Layer includes two frameworks that simply happen to be packaged together: SQL Maps and DAO.

Today the iBATIS project is still heavily focused on the persistence layer frameworks known as SQL Maps and Data Access Objects (DAO). JPetStore lives on as the official example of typical usage of these frameworks.

The object database counterculture began to move toward standardization in 1991 with the formation of the Object Data Management Group (ODMG, later the Object Database Management Group). This group released the ODMG 1.0 standard in 1993, and concluded its work in 2001 with the release of the ODMG 3.0 standard [ODMG].

Some key technical ideas were taken from object databases and the ODMG standards that became important in later standards. In particular, these important ideas were included:

After the release of the ODMG 3.0 standard, the ODMG was disbanded and its work was taken over by a working group within the OMG [Object Management Group]. However, one of the products of the ODMG continued its lifecycle when the ODMG 3.0 Java binding was submitted to the Java Community Process (JCP) as the basis for the Java Data Objects (JDO) specification [JDO].

JDO took much of its inspiration from the notions of object databases, and in particular, the ODMG specifications. The JDO 1.0 specification was completed and released through the JCP in April 2002 as JSR 12. Its primary API included the following concepts:

The release of JDO 1.0 resulted in multiple implementations of the JDO interface, with companies like Versant and Xcalia releasing commercial projects that implemented JDO 1.0, and several open-source projects including Apache OJB and JPOX adopting it as well.

As discussed earlier, by the time EJB 2.0 started to make its way into products, yet another counter-current was building in the Java community that was looking for alternate ways of providing access to persistent data within Java applications. When the EJB 3.0 committee began meeting, it became clear that revisiting persistence would be a key feature of the new specification. The opinion of the committee was that something significant needed to be changed—and as a result, the committee made the following decisions:

So, the EJB committee combined the best ideas from several sources such as TopLink, Hibernate, and JDO to create a new persistence architecture, which was released as a separate part of EJB 3.0 specification and dubbed the Java Persistence Architecture (JPA) [EJB 3 Spec].

There are several vendors with open-source initiatives that have implemented the JPA specification, including JBoss (Hibernate JPA), Oracle (TopLink Essentials), and Apache (OpenJPA). OpenJPA is backed by IBM and BEA.

Applying the SOA paradigm to persistence has resulted in a recent advancement that treats “information as a service.” This thought shift aims to free information from the application silos in which it typically resides. With this approach, information is delivered as a service. Applications needing access to the data interact with the service; and the information is exposed via open standards leveraging Web services and XML. Applications can access their data via language neutral technologies such as XQuery and XPath. Because of the use of open, language neutral standards, more applications (even those not based on Java) can have access to the information they need.

IBM’s DB2^® has been extended to include native storage for XML. This makes DB2 Version 9 the industry’s first database designed with both native XML data management and relational data capability. In the past, XML data had to be reformatted or placed into large objects—character or binary large objects (CLOBs or BLOBs). With this approach, data is stored as a single unit or continuous byte string. Such a storage approach unearthed performance challenges when querying for data within the XML document or when one had to update even just a small portion of the XML document. The entire XML document had to be read and parsed before a query could be processed. Another approach commonly used to deal with storing and querying XML documents is the practice of document shredding, in which an XML document is mapped to one or more relational columns. However, this approach does not leave the XML document intact.

The information as a service technology stores XML in a direct form, as a preparsed annotated tree. XML is stored in the database as using a hierarchical node-based model. In this model, each node is linked not only to its parent, but also to its children. This “pure XML” storage capability is an efficient method for accessing and storing entire XML documents, presenting major performance benefits to the consuming application. The capability to deliver information as a service that can be efficiently accessed makes the technology particularly enticing for SOA environments.

DB2 9 allows for the creation of hybrid tables that consist of both relational and XML data. The database supports a new XML data type. Such a hybrid table can be accessed via industry standard extensions to SQL named SQL/XML that allow for SELECT statements to retrieve XML documents in whole or in part. Additionally, DB2 supports the XQuery language, which allows for one to use path expressions to navigate through a stored XML document’s hierarchical structure.

In contrast to large companies that could afford to host their own web sites, many smaller companies prefer to outsource their web sites to external companies that provide the infrastructure, such as Yahoo hosting. These external companies relied on scripting languages, like PHP and Perl, with embedded SQL, that provide a quick and easy way to build web sites. These facilities reduced the amount of code one had to write to create a web site, but were not focused on developing reusable business components. Further, with the emergence of AJAX in the browser for rich dynamic content, and community web sites like MySpace and FaceBook, the need to quickly combine web data sources to create new applications came to the forefront. This focus on rapid development led to a new paradigm called “Situational Applications.”

Enterprises are now looking to take advantage of this paradigm to create new business opportunities. As such, exposing data through scripting languages using embedded SQL has become popular with enterprise web developers. With the advent of Information as a Service and Web 2.0, companies are looking to exploit these techniques within enterprise Java applications.

To meet this need, pureQuery is an IBM solution for Java data access designed for performance and exploiting SQL. As with iBATIS, the goal is to embrace SQL rather than hide it. But unlike iBATIS, SQL can be bound statically in a mapping file as well as directly in the Java code. pureQuery also embraces the notion of tools to greatly increase developer productivity. pureQuery can be used with a set of Eclipse Java plug-ins that can provide SQL assist inside Java programs. Besides having its own API for direct programmatic access, pureQuery can be layered underneath another framework, such as Hibernate or OpenJPA.

One example of layering on top of pureQuery can be found within ProjectZero. ProjectZero is an IBM incubator project focused on agile development of Web 2.0 applications following the SOA. Web 2.0 applied to SOA allows web artifacts to extend the reach of SOA. This can be thought of as Web Extended SOA.