Chapter 11. Remote access
In this chapter
- Making your application available to software clients
- Leveraging HTTP via the REST pattern
- Exporting services to remote clients
Now that you have a good grounding in securing an application from attack and unauthorized access, it’s time to look at how other software (other than web browsers) can use and interact with your Grails applications.
The prominence of controllers and GSPs shows how heavily geared Grails is toward browser-based web applications, but there’s often no reason to limit yourself to a single UI—anyone who has used Twitter knows that you can use a variety of different clients to view and post tweets. An HTML interface isn’t much use for such software clients because they usually control the presentation of the information themselves. What we need is a way for software clients to interact with the data in the system—in effect, a public API to the application.
Such an API can be exposed in many different ways, and this chapter will introduce you to a few of the more common ones. The primary focus will be on a mechanism that has become popular in recent years and that suits web applications well: REST. Grails has core support for this technique, and that’s another reason to look at it closely. We’ll also take a peek at a couple of plugin options that may suit your requirements better: remoting (with RMI) and Web Services via the SOAP protocol.
By the time you have finished this chapter, you’ll understand how to design and implement a public API in Grails quickly and easily, while avoiding some of the pitfalls. You’ll also know enough to make an informed decision on what implementation to use based on your requirements and the nature of the application.
We’ll start by adding a simple API to Hubbub that will allow software clients to view and post messages.
11.1. Using a RESTful solution
The first step in creating an API for an application is deciding what it should allow clients to do. After all, if a feature isn’t available in the API, a client can’t use it. Working out what should go into the API isn’t always an easy task, but during development you have scope to experiment. Once the API is published, that’s it. You have to avoid changes that will disrupt those software clients that rely on it.
Tip
Adding elements to an API often causes fewer problems than changing or removing existing elements. A good strategy is to start minimally and grow the API as required.
Hubbub could have quite an extensive API, but for now we’ll keep it simple. Clients should be able to do the following:
- List existing posts
- Create new posts
- Retrieve the content of existing posts
Note that these statements don’t contain any information about implementation, and that’s the best way to start. Once you have your API in this form, it’s time to look at the implementation.
In Java, an API is typically expressed as a set of public interfaces with well-defined behavior, but this isn’t much use for a web application whose clients may not be written in Java. What other options are there? Consider how a web browser communicates with your web applications. They talk to each other via the platform-neutral HTTP protocol—why not try using that?
11.1.1. Your first steps in REST
You may think HTTP is just the language of the web, but it embodies the REST (short for representational state transfer) architectural style that works for any client/server communication, not just web browsers. The idea is that an application should be viewed as a collection of resources that can be operated on by a few methods. Take, for example, a wiki: each page is a single resource that you can fetch, modify, or delete. You can even create new pages.
Why the name REST? Each method (except delete) involves transferring the contents of a particular page between the client (the browser) and the wiki. The page contents are a representation of the state (data) of that page. You can see another example in figure 11.1, where a Hubbub post is the resource.
Figure 11.1. The key concepts of REST. The Hubbub post (a domain instance) is our resource, and the XML returned to the client is a representation of that resource.
The sheer scale of the web and the variety you can find within it should be explanation enough as to why REST has become a popular paradigm. On top of that, it has the advantage of being a simple concept, and HTTP provides us with a ready-made implementation (REST, itself, is akin to a design pattern).
The four basic HTTP verbs (or methods) are GET, POST, PUT, and DELETE. The first two are pretty common among normal web applications and are used by browsers. Every time you open a web page, the browser is issuing a GET request, and when you submit data to a website, the information is typically sent as a POST.
To implement a REST API in Hubbub, all you have to do is decide what resources should be available and ensure that the application can handle all four HTTP methods for each of them. We’ll only look at Hubbub posts in the following examples, but the same techniques can be used to expose other parts of the application, such as adding and removing friends.
As you know from browsing the web, you specify the location of a resource with a URL. Ideally, the URLs you use should have the following attributes:
- Unique to a resource— Each URL should only represent a single resource. For example, you shouldn’t have a single URL that identifies two different posts. You can have multiple URLs per resource, but we think it helps avoid confusion if you don’t.
- Long-lived— A URL should always point to the same resource, preferably until the end of time. That may sound like a bit much to ask for, and the HTTP specification acknowledges this with temporary and permanent redirects. Yet you should endeavor to ensure that your URLs are long-lived, or you might get a bad reputation for broken links—one of the banes of the web.
Bearing these points in mind, what should the URLs for our Hubbub posts look like? Simplicity is good, so why not use http://.../posts to represent all the posts in the system and http://.../posts/$id to represent a single post, where $id is the internal ID of that post?
Good URL design
We only highlight two recommendations for URLs here, but they aren’t the only contributing factors to good URL design. Some would argue that URLs should also be short, easy to understand, and easy to remember. These aren’t directly related to REST, but such principles can help users and developers. You can find out more by searching for “good URL design” on the web.
Note
Our URLs include the location of the server (and possibly the servlet context). As long as the ID is unique within the application, each URL is guaranteed to be globally unique.
All this raises the question of what we should use for the post’s ID. The obvious choice is the ID of the domain instance, but although this is the default behavior in Grails scaffolding, there are implications you need to be aware of.
Choosing Resource IDS
We want long-lived URLs for our REST API. In a standard Grails application, the ID is generated by the database (it’s the primary key). As long as your application runs against the same database for its entire life, that’s not a problem. But what if your application becomes so popular you need to migrate the data to the latest and greatest database?
To do so, you would probably export your data as SQL statements (including the current ID of each record) and import all of it into the new database. This would certainly work for the existing records, but what would happen when your application created a new domain instance? If you’re lucky, the auto-increment feature would automatically start generating IDs after the last record in the corresponding table, but that’s not something you want to rely on.
Another downside to using the standard primary key in your URLs is that attackers can easily guess valid values. It also gives attackers clues as to the underlying implementation of your application. This may not be an issue for you, but if you’re storing highly confidential information, you should consider an alternative source of IDs.
A couple of options to consider are using GUIDs or UUIDs for the domain class’s primary key or using a different (but unique) field as the ID in the URL (such as a username or an email address). Hubbub isn’t a high-risk application, so we’ll use the standard primary key to keep things simple.
We’ve now decided on the format of the URLs we’ll use for posts, so the next stage is to provide implementations for each HTTP method.
Providing Access to the Resources
So far in this book, you have only seen one-to-one mappings between URLs and controller actions. We could continue with that approach and have our action do something different based on the HTTP method, like this:
def handlePosts = {
if (request.method == "GET") {
...
}
else if (request.method == "POST") {
...
}
}
This does the job, but those condition statements won’t be easy to maintain, and they will also make refactoring difficult.
Let’s try another way, using URL mappings:
class UrlMappings {
def mappings = {
...
"/posts"(controller: "postRest") {
action = [ GET: "list", POST: "save" ]
}
"/posts/$id"(controller: "postRest") {
action = [ GET: "show", PUT: "update", DELETE: "delete" ]
}
}
}
As you can see, we can set a URL mapping’s action parameter to a map that associates HTTP methods with action names. In this case, a GET request on /posts is handled by the list action, whereas PUT on /posts/$id is dealt with by update. Note that we’ve mapped the URLs to the postRest controller rather than the existing post controller to keep things simple. You’ll soon see that you don’t have to use a separate controller for REST requests.
POST versus PUT
A common source of confusion in REST revolves around when you should use POST and when PUT. According to the specification (http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html), PUT is used for creating and updating resources at a given URL.
In most cases with Grails, the ID of a new resource isn’t known before it’s created, so PUT doesn’t have a URL to work against. Instead, you perform a POST to the parent URL, say /posts, which creates a subordinate resource, such as /posts/10.
In the case of an existing resource, you know the ID, so you can use PUT to update it.
The mapping isn’t much good without a controller to back it, so let’s rectify that now. The one big decision we have to make is what form our post data should take. This is the “representational state” bit of REST. You could technically choose any form you like, even binary, but XML is probably the most common format in use today. Partly for that reason, we’ll use XML here—it also helps that Grails provides a dedicated syntax to make life easy for us.
Our GET handlers (the list and show actions) are straightforward:
package com.grailsinaction
import grails.converters.XML
class PostRestController {
def list = { render Post.list() as XML }
def show = {
Post post = Post.get(params.id)
render post as XML
}
}
It doesn’t get much simpler than that. Both actions generate XML responses using some magic that we’ll look at in the next subsection. For now, try out the show action by running the application and pointing your browser at http://localhost:8080/hubbub/posts/1. You should see a page containing some XML that looks a little like the fragment shown earlier in figure 11.1.
How are those XML responses generated? To understand that, you need to learn a little about Grails converters.
11.1.2. Serializing and deserializing domain instances
The XML class is an example of a Grails converter. The sole purpose of these converters is to convert or serialize domain instances into a particular format, such as XML. Although Grails only provides two converters by default (XML and JSON), you can add your own by extending the AbstractConverter class and implementing the required methods (see the API documentation for more information).
Under the hood, Grails overrides the asType() Groovy method for domain classes (via the core converters plugin) so that you can do as XML and as JSON. You can also use the as keyword with your own converters. But this is only part of the story: as XML won’t generate an XML string for you; it will create an instance of the XML class. To generate the corresponding text, you must use either the render(Writer) or render (HttpServletResponse) method on the converter instance, as demonstrated by this code fragment:
def xml = obj as XML
xml.render(response)
It’s the latter of these two methods that’s used when you call render post as XML from your controller.
The converters plugin not only overrides the asType() method but also adds an extra render(Converter) method to the controllers that delegates the response-writing to the converter. In addition to writing the response, the converter also sets the appropriate content type for it.
With almost no effort, we’ve implemented the GET method for our posts so that an XML representation is returned. The next step is to create new posts via the HTTP POST method. To do this, we can make use of a dynamic property added to the request by the converters plugin, as demonstrated by the save action shown in listing 11.1.
Listing 11.1. Handling a POST request containing XML
We start by retrieving the XML from the request
. The object returned by the XML property is a GPathResult, which is the usual result you get from parsing XML with Groovy’s XmlSlurper. We then use that result to populate a new Post instance
with the data provided in the request. We assume that the root element contains child content and user elements like the XML generated by post as XML:
<post>
<content>...</content>
<user id="..."/>
</post>
We then finish off by saving the new post and sending back an XML response
indicating whether the post was successfully created.
To be a truly good citizen, we’d normally include some information in the XML about the reasons for any failure, but we avoid that here to keep the code simple. Ideally, we’d also set the status of the response to an HTTP error code, such as 400. We’ll look more closely at status codes at the end of section 11.3.
XML content types
When sending XML content over HTTP, you can choose from several different MIME types: text/xml, application/xml, or application/...+xml. The last of these is a pattern for specific XML languages, such as application/xhtml+xml for XHTML.
If your content conforms to a specific XML language with an associated MIME type, it’s best to use that. Otherwise, for general XML, RFC-3023 recommends that you use text/xml if a casual user can read and understand the content, or application/xml if not. This is because tools that don’t understand XML will display text/xml as plain text.
One final thing: if you choose to set the response content type to text/xml, add the charset parameter as we’ve done in the example. Otherwise, well-behaved software clients will assume an encoding of US-ASCII.
We now have a properly functioning REST API for Hubbub posts, but you shouldn’t take that on trust alone. Why don’t you try it out? Testing the GET handling is easy because you can point your web browser at the relevant URL and see what response you get back. The other HTTP methods are more problematic because web browsers only use POST when submitting forms (no XML here!) and most don’t support PUT or DELETE.
What you need is a dedicated REST client that will allow you to send any type of HTTP request with any content you want. If you use the Firefox web browser, your best option is to install the Poster plugin. Other options include a Java client that uses a Swing user interface (http://code.google.com/p/rest-client/) or if you like the command line, you can’t get much better than the flexible curl utility (http://curl.haxx.se/).
To test the REST API, you can send the following XML content in a POST request to the /posts URL of Hubbub:
<post>
<content>I'm posting this via REST!</content>
<user id="1"/>
</post>
Make sure that the user ID you specify is valid—"1" should be fine when running against HSQLDB or MySQL, but it’s unlikely to work with a default PostgreSQL configuration. Figure 11.2 shows the results of sending this request via the Poster Firefox plugin.
Figure 11.2. Sending a test POST request to Hubbub using the Poster plugin for Firefox. The request should create a new post for the specified user, and the ID of the new post should be returned in the response.
Manual testing using a client like Poster is great for experimentation, but in the long run you know you need automated tests. That may sound difficult, but it’s not much harder than testing standard controllers.
11.1.3. Testing the API
Because we’re using controllers to implement REST, we can unit-test them in the same way we tested controllers in chapter 7. By extending ControllerUnitTestCase, our own test gets all the goodies we need, including an extra feature that hasn’t been discussed before.
Listing 11.2 contains a unit test for the postRest controller that you saw in the previous subsection. It makes sure that both the show and save actions work as expected, although it isn’t as thorough as it could be.
Listing 11.2. Unit-testing a REST controller
You’re already familiar with much of the setup for the test, so let’s focus on what’s new. When testing the show action, we set the id parameter, call the action directly, and then parse the response content using XmlSlurper
. Why parse it? XML is tricky to deal with because neither whitespace nor the order of element attributes is significant.
You could have one XML document with nice indenting over multiple lines and another compressed into a single line, both semantically
equivalent! As soon as you start doing string comparisons with XML, you leave yourself open to major problems. Parsing the
XML string first makes your test robust in the face of changes to the XML formatting.
In order to test the save action, we have to somehow provide XML content for it to process. This is easily done with the setXmlRequestContent() method
. The particular variant we use in this example generates the XML using markup builder syntax. You can also set the content
with a string using setXmlRequestContent(String). Once any variant of this method is called, the action can access the XML content via both the request’s input stream and
its dynamic XML property.
Finally, we check that the action succeeded by looking at the content of the response
. It’s not worth parsing XML that contains a single element, because it’s unlikely to have different formatting. We use a
simple string comparison instead.
That’s it! Unit-testing a REST controller is that simple. We’ll demonstrate a few more features of the unit-test framework in the next section, but you’ve already learned the bulk of what you need to know. We also recommend performing functional tests on your REST API to make sure everything is working properly, but how you do that depends on the testing framework you use. At the time of writing, the Functional Test plugin doesn’t support sending XML requests, but that feature is on the roadmap and may be available by the time you read this. Check out the plugin’s home page.
With confirmation that our REST API is working as expected, is there anything else to do? One weakness of the approach we took is that we’ve limited ourselves to XML requests and responses. What about JSON? Your initial thought may be to create a whole new set of controller actions, but then you would need separate URLs for XML and JSON—not RESTful! There’s another way: content negotiation.
11.2. Negotiating the representation
Imagine a traveler wandering through a market in a remote part of the world. He sees something he wants to buy, but in order to do that he needs to communicate with the trader. The first step involves agreeing on a language that both sides understand—even if it’s just improvised sign language. This is analogous to how a client and server must agree on a format for data interchange in REST.
Grails provides a facility called “content negotiation” that manages different formats and handles the negotiation with the client. The client can specify its preferred content format a variety of ways, and, on the server side, Grails makes it easy to render different content based on that format. Figure 11.3 illustrates how this works at a high level.
Figure 11.3. The process of content negotiation in action. The client lets the server know what types of content it would like to receive, and the server returns the content as one of those formats.
How does content negotiation affect your REST API? As far as the application is concerned, you only have to consider two aspects: dealing with the content of incoming requests and rendering the appropriate content to the response. For requests, you don’t want to write code to handle each type of content, so it may come as some relief to hear that Grails unifies access to the request data through the standard params object.
Note
Content negotiation assumes symmetry. This means that Grails expects the content type of the request to match the content type of the response, except in the cases of GET and DELETE.
11.2.1. REST and the params property
You already know that the params object contains both URL parameters and POST data (from HTML forms). What you may not realize is that when content negotiation is active, Grails can add both XML and JSON data from the request. Of course, it helps to know how the structured data in the request is mapped to the params object, so we’ll take a quick look at how an arbitrary XML document becomes parameter data.
Let’s say our request contains this XML:
<post>
<content>A new post.</content>
<tags>
<tag>Music</tag>
<tag>US</tag>
</tags>
<user id="10">
<userId>dilbert</userId>
<password>hjdj23r0fn</password>
</user>
</post>
Note that this XML is simply for illustration—it won’t work with Hubbub.
Assuming that the client and server have negotiated a content type of XML (you’ll see how that happens in subsection 11.2.3), Grails adds an entry to the params object with a key of post (named after the root element) and the following map as its value:
[ content: "A new post.",
tags: "Music US",
"user.id": "10",
user: [ userId: "dilbert", password: "hjdj23r0fn" ],
"user.userId": "dilbert",
"user.password": "hjdj23r0fn"
]
Each element in the XML that has an id attribute corresponds to a nested map. All other elements map to strings (even the contents of the tags element). This map of data can readily be bound to either domain or command objects, as you have seen done with normal parameter data:
Post post = new Post(params["post"])
...
The preceding line should be pretty familiar from previous controller code, and it will work regardless of whether the data is passed in as XML, JSON, or HTML form data.
Grails won’t automatically parse the content of requests, so you have to explicitly enable this feature. You have two options: use either the resource or parseRequest parameter in your URL mapping.
The resource parameter replaces the controller and action parameters to make a RESTful mapping:
class UrlMappings {
"/posts/$id?"(resource: "postRest")
...
}
This mapping will apply the default REST behavior to the postRest controller so that GET maps to the show action, POST to save, PUT to update, and DELETE to delete. It also causes Grails to automatically parse the content of matching requests.
The parseRequest parameter, on the other hand, only affects the parsing of requests. Like resource, it’s a per-mapping setting:
class UrlMappings {
"/items/$id?"(controller: "item", parseRequest: true) {
action = [ GET: "list", POST: "save" ]
}
...
}
With one simple parameter, your action can access the XML content via params or request.XML.
Tip
The resource parameter is problematic. You may remember from before that PUT expects an id in the URL, but POST doesn’t. Implementing that behavior correctly with resource would require more work than not using it in the first place! If you want a correctly behaving REST API, we recommend that you avoid the resource parameter.
Before we move on, there are some important points to make about mapping XML to entries in the params object. First, object relationships only work if the id attribute is given. That means, for example, that you can’t create a post and its associated user at the same time, but you can create a new post linked to an existing user. Second, one-to-many relationships aren’t supported at all.
Finally, the default scaffolded views don’t add a prefix to the names of their input fields, so they won’t work with the previous fragment of controller code (post = new Post(params["post"])). The scaffolded edit page would contain something like this in the HTML:
<input type="text" name="content" id="content" value="..." />
But it needs to be this:
<input type="text" name="post.content" id="post.content" value="..." />
Note how the input field name is prefixed by post., which results in the value being available via params["post"].content.
If you can live with these limitations, using the params object for REST-style requests works quite well. We’ll discuss other options in section 11.3, but for now let’s move away from handling request content and look at the response instead.
11.2.2. Handling multiple response formats
Let’s consider the show action we created for the postRest controller. In its current incarnation, it will always render an XML response; but what if the client wants the post data in JSON form instead?
Your first impulse is probably to check the requested format in a condition, which is possible because Grails adds a dynamic format property to the request object:
def show = {
Post post = Post.get(params.id)
if (request.format == "xml") {
render post as XML
}
else if (request.format == "json") {
render post as JSON
}
else { ... }
}
Although this works, we’ve come to expect better of Grails. Listing 11.3 demonstrates a much cleaner and more readable approach using the special withFormat block. You can try this out with a browser by appending a format parameter to the URL and setting its value to one of the supported formats, such as xml or json (say http://.../post/1?format=json). You’ll see in the next subsection why this URL parameter works, when we look into how Grails determines the format for a request.
Listing 11.3. Using withFormat to generate format-specific responses
The syntax of the withFormat block
is straightforward. For each format you want to support, you add an extra block with the name of that format. In this example,
we respond to requests for HTML, XML, and JSON. If the action has no entry for the request’s format, Grails picks the first
one (html, in this case).
You should treat each format block as a separate controller action. That means if you want to render the appropriate view
(show.gsp, in this case), you return a map as we’ve done for the HTML and XML
formats. You can also use the render() method directly if you choose (as we’ve done with JSON). The main requirement you need to be aware of is that the withFormat block must be the last bit of code in the controller action.
Unit-testing withFormat
If you want to unit-test an action that uses withFormat, all you have to do is set the format on the request like so:
this.controller.request.format = "xml"You don’t even have to mock the withFormat method, because it’s already done for you by ControllerUnitTestCase.
Something in that last paragraph may have jarred. In effect, we said that show.gsp is used for both the HTML and XML responses, but how can that be? We’re unlikely to want to return the same content for both, and we want to make sure that the appropriate content type is set on the response. Don’t worry: this is easily resolved with another nice Grails convention. You can create GSP files with the naming convention <action>.<format>.gsp, which means that you can have different views for each format.
For our example, we’d have views/postRest/show.xml.gsp, looking something like this:
<%@ page contentType="text/xml; charset=utf-8" %>
<post>
<content>${post.content}</content>
<user id="5"/>
</post>
This is a standard GSP that generates XML rather than HTML. The most important aspect of the preceding code is the @page directive, which ensures that the content type of the response is set to XML. Without that line, the response type would be text/html, which would be not only incorrect, but a particularly bad idea with clients that treat HTML and XML content differently.
Format order
The order in which you declare your withFormat handlers is important because if the format has a value of all or no value at all, Grails triggers the first handler. Several old web browsers send an Accept header that Grails interprets as requesting the all format, so we recommend that you always put the html handler first (unless your action doesn’t support HTML). We look at the Accept header in the next section.
Although we’ve used a format-specific view in conjunction with the withFormat block, such views will work equally well without one. In fact, if we dispense with the JSON handler in listing 11.3, we can get rid of the withFormat block too, because the model for the XML view is the same as the one for HTML:
def show = {
Post post = Post.get(params.id)
[ post: post ]
}
This means you only need to use the withFormat block if you do different things in the action for each of the formats.
We’ve covered a fair bit about content negotiation already, but an important piece is missing: how does Grails decide what the format for a given request is?
11.2.3. How the response format is decided
Several factors influence the format that Grails settles on for a request. Here they are, in order of precedence:
- The URL file extension
- The format URL parameter
- The request content type (via the HTTP Content-Type header)
- The HTTP Accept header
We’ll look at them in turn.
The URL File Extension
The first of these factors is pretty self-explanatory. Given a URL like http://localhost:8080/hubbub/post/1.xml, Grails will set the format to xml—the bit after the last period (.). This means both 1.xml and 1.tmp.xml correspond to a format of xml. But if you use this approach to serve a single resource in multiple forms, you’ll have multiple URLs representing a single resource—not good practice for URLs, as we said before.
Another thing to bear in mind is that the extension is only treated as a format if it’s recognized by Grails. There’s no need to panic: if you take a look at the Config.groovy file for your project, you’ll see a grails.mime.types property with a map as its value. Grails recognizes any format that appears as a key in that map.
This is the default setting:
grails.mime.types = [ html: ['text/html','application/xhtml+xml'],
xml: ['text/xml', 'application/xml'],
text: 'text/plain',
js: 'text/javascript',
rss: 'application/rss+xml',
atom: 'application/atom+xml',
css: 'text/css',
csv: 'text/csv',
all: '*/*',
json: ['application/json','text/json'],
form: 'application/x-www-form-urlencoded',
multipartForm: 'multipart/form-data' ]
By default, Grails recognizes most of the important content types used on the web. It’s trivial to add your own by inserting extra map entries.
Warning
By default, Grails strips the extension from the URL if it matches a known format. This means that you can’t match URLs with patterns like /post/$filename.xml in your URL mappings. You can either use the pattern without the extension, /post/$filename, and generate the correct content based on the request format, or you can disable this behavior by setting the grails.mime.file.extensions option in your configuration to false.
The Format URL Parameter
The format URL parameter acts in a similar fashion to the file extension, but it has no impact on the name of the resource because it’s a query parameter: http://localhost:8080/hubbub/post/1?format=xml. This is probably the simplest and most reliable way for the client to request a particular format.
The Request Content Type
The last two items in the list are different because they deal with MIME types rather than simple format names. This is why the grails.mime.types configuration option is a map rather than a list or set: Grails needs to know how to go from a MIME type to the name of the corresponding format.
The Content-Type header is straightforward, but it’s limited in that most GET requests have no content type associated with them, nor should they. In practice, this option only makes sense for POSTs and PUTs. Even then, you need to watch out: browsers post data with a content type of application/x-www-form-urlencoded, yet they generally want the response back as HTML. This is one example where the symmetry inherent in Grails’ content negotiation isn’t ideal.
The HTTP Accept Header
The Accept header is a different kettle of fish: it contains a list of preferred content types that the client can handle, along with some q (or quality) ratings that rank them in order of preference.
Consider, for example, this value for the Accept header:
application/xhtml+xml,application/xml,text/html;q=0.9;text/xml;q=0.8
This states that the client prefers both XHTML and XML (via application/xml) over HTML, which, in turn, it prefers over the text/xml MIME type. The q rating has a value of 0 to 1, with higher values corresponding to greater preference. If a particular MIME type has no q rating (as with XHTML in the example), Grails assigns it a value of 1.
We could go into more detail about the Accept header and its interpretation, but we don’t think it will help at this stage. You can find a bit more information in the Grails reference guide if you need it. The problem is that Grails has a rather complex (partially heuristic) algorithm for determining the request format from the Accept header. This is designed so that you end up with a request format of html for most of the popular web browsers, even though they all have quite different Accept headers. Unfortunately, that complexity means that Accept headers from other types of clients may not work as well.
That description may be a bit too abstract to grasp at first, but put simply it means that relying on the Accept header for content negotiation can be fraught with difficulties. We recommend you use the format query parameter wherever possible: it’s simple and consistent in its behavior.
While we’re on the topic of the Accept header, remember that special all format we mentioned earlier? That exists because many older browsers send an Accept header with the value */*, which means “I accept all types of content.” This MIME type corresponds to the all format, as you can see by looking at the grails.mime.types map in the configuration.
Those are the basics (and not so basics) of Grails REST support. You now have the full set of tools at your disposal, so you can quickly implement a REST interface for any Grails application. The question is, how do you best take advantage of those tools?
11.3. REST in practice
Implementing a simple REST interface for your Grails application is straightforward using the various features we’ve shown you so far. The trouble is, there are some problems with the built-in support. In this section, we introduce some best practices and the reasons behind them.
11.3.1. Keeping the API stable
Let’s start with one of the neatest features: the converters. In one line of code, you can serialize a network of domain instances to XML, JSON, or some custom format. So what happens when you change a domain class, by removing a field, for example? The serialized version of the domain class also changes—the field you removed no longer appears in the XML (or whatever format you’re converting to).
This is a problem, because REST interfaces are typically public APIs, which means that they should be stable and maintain backward compatibility. If you generate XML responses (for example) using the standard converter, your public API becomes dependent on internal implementation details—a bad thing indeed! Ideally, you should decouple your REST interface from the implementation.
On the other side of the conversion coin, we have the issue of getting data from XML or JSON requests. As you saw earlier, Grails will automatically parse HTML form data, XML, and JSON from requests and populate the params map with the data. This is fine as long as all the data ends up there, but that isn’t the case for XML. The parser ignores all attributes except id, and it only creates nested maps for elements with an id attribute. Take this fragment of arbitrary XML as an example:
<item category="general">
<name>New message</name>
<owner>
<id>glen</id>
<firstName>Glen</firstName>
<lastName>Smith</lastName>
</owner>
</item>
The category attribute on the root element won’t appear in the params object, nor will any of the structured information inside the owner element, which has no id attribute. That’s a lot of significant information to lose.
What’s the alternative? For a truly stable API, you must define the request and response messages supported by each URL and build your implementation around them. That effectively rules out converters, but for XML you could do a lot worse than using XmlSlurper to parse request messages and render() (with markup builder syntax) or GSP to generate the response messages.
What about JSON requests and responses?
Typically, REST interfaces use XML as the prime resource representation, so REST recommendations and techniques are mainly focused on that format. The same considerations apply to JSON as well, but the techniques less so. For example, the structure of JSON matches the nested maps approach of the params object quite nicely, so you might be better off letting Grails automatically parse JSON requests and retrieving the required information from the params map.
On the response front, the JSON syntax doesn’t sit well with GSP syntax, so having a view like show.json.gsp is less appealing than using the render() method with JSON markup.
Developers writing clients for your application generally appreciate a definition of the API, which should include the URLs and the structure of the messages. You could provide this with some online human-readable documentation, but there are now two standards that allow you to formalize the definition of a REST API: Web Application Description Language (WADL) and Web Service Definition Language (WSDL) 2.0.
We don’t have much experience with either, but the advantage of using WADL or WSDL 2.0 is tool support. It’s also easier to precisely define the behavior of your API when using a dedicated language. Neither is particularly easy to read, though, so it’s a good idea to have some human-readable documentation as well.
Note
You can find a primer for WSDL 2.0 at http://www.w3.org/TR/2007/REC-wsdl20-primer-20070626/ and the specification for WADL at https://wadl.dev.java.net/wadl20061109.pdf.
Now that you know the best approach to keeping your REST API stable, let’s take a look at it in action, using Hubbub as an example.
11.3.2. Applying the theory
You’ve seen the theory, so now let’s apply some of this reasoning to Hubbub. We’ll start with the URL mappings you saw earlier:
class UrlMappings {
def mappings = {
"/posts"(controller: "postRest") {
action = [ GET: "list", POST: "save" ]
}
"/posts/$id"(controller: "postRest") {
action = [ GET: "show", PUT: "update", DELETE: "delete" ]
}
}
}
Now we need to consider what form the XML messages will take for each of these mappings. To keep things simple, we’ll only look at GET and POST for the /posts URL and GET for /posts/$id, but the example source code for Hubbub includes the complete implementation.
The GET method on /posts/$id should return an XML response looking like this:
<post>
<content>Message content goes here</content>
<created>2008-01-05</created>
<user>
<username>glen</username>
<fullName>Glen Smith</fullName>
</user>
</post>
This is superficially similar to the XML generated by render post as XML, but it doesn’t include any domain instance IDs. We return the username (corresponding to the userId field of the User class) instead, which also uniquely identifies the user in the system.
The problem with domain instance IDs is that they’re opaque, which makes debugging problems in the API difficult. Imagine that you’re trying to debug Hubbub by looking at the XML content passing between the client and the application. Which would you prefer to see: an integer ID or the username? Also, using domain instance IDs leaks an implementation detail into your API.
The GET for /posts returns a root element, <posts>, containing multiple child <post> elements. For POST we use something even simpler:
<post>
<content>New content goes here</content>
</post>
This will create a new Hubbub post for the logged-in user. We discuss how users log in via REST at the end of this section.
Now that we’ve defined the messages, we can move on to the server-side implementation. Listing 11.4 contains the latest code for our postRest controller, which implements the API we’ve just defined.
Listing 11.4. Manually parsing XML requests in a REST controller
We’re limiting ourselves to supporting only XML in listing 11.4 to keep the code short. The list action has a simple implementation because we delegate the XML generation to the view
.
The same goes for the show action, and this is its GSP (grails-app/views/postRest/show.xml.gsp):
<%@ page contentType="text/xml; charset=utf-8" %>
<post>
<content>${post.content}</content>
<created>
<g:formatDate date="${post.created}" format="yyyy-MM-dd"/>
</created>
<user>
<userId>${post.user.userId}</userId>
<fullName>${post.user.profile.fullName</fullName>
</user>
</post>
The show action is slightly more interesting than list because it will also send a 404 error
if there’s no post with the requested ID. Using the HTTP status codes like this is good practice, and you’ll read more about
them at the end of this subsection. Normally, you don’t have to mess with the status code, because the default 200 code is
fine for actions that complete successfully.
On sendError()
The sendError(int code) method on the response is the standard way for Java web applications to report errors via HTTP status codes. There are some limitations to it, though. First, you can’t provide any content for the response. Second, the response content type will be text/html.
You have two options to get around these limitations: set the response status manually and generate the content for the response as you normally would, or add a status code mapping to your URL mappings:
"500"(controller: "errors", action: "serverError")
The save action is a little more involved, although the code should be familiar to you by now. We first get hold of the currently
logged-in user so that we can associate the new post with that user, and then we extract the content for the new post from
the request’s XML
. After that, we delegate the creation of the post to the relevant service and check that it was, in fact, created.
According to the HTTP specification, successful creation of a new resource should result in a 201 status code, so that’s what
we do
. We should also provide a value for the Location HTTP header field, but problems with reverse URL mappings make that difficult without hard-coding the URL. On the other hand,
if the new post can’t be created for any reason, we create an XML response that contains the details of why it failed
. We also use an HTTP status code of 403 to indicate that the request failed and that resubmitting the request won’t help.
Using HTTP Status Codes
Identifying the appropriate status code to use for any given response can be tricky, so we’ve listed some of the most common ones in table 11.1, along with recommendations on when to use each of them. For a full list with descriptions, see http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html.
Table 11.1. Common HTTP status codes and when to use them
|
Code |
Short name |
Description |
|---|---|---|
|
200 |
OK |
The request completed normally. |
|
201 |
Created |
A new resource was created. The Location header should contain the “most specific” URI of the new resource, and the response content should include a list of possible URIs. |
|
301 |
Redirected Permanently |
The resource has moved to a different URI permanently. |
|
302 |
Found |
This indicates a temporary redirect. The redirect() method works by returning this status code. |
|
400 |
Bad Request |
The content of the request is incorrect or malformed. For example, use this code when the wrong XML message is used for a particular URL or when the message contains invalid XML. |
|
401 |
Unauthorized |
The client isn’t authorized to access the resource. This code is only valid if one of the standard HTTP authentication methods is used (Basic, for example). |
|
403 |
Forbidden |
The request isn’t allowed. This is a multipurpose status code that covers validation errors and any other client-side errors that can’t be matched to another status code. |
|
404 |
Not Found |
The resource was not found—everyone knows this one! |
|
500 |
Internal Server Error |
This usually indicates an error in the server code. Servlet containers typically return 500 if there’s an uncaught exception. |
Used wisely, status codes enrich a REST API and help to provide clients with tremendously useful feedback that’s easy to access. Combined with suitable content for the responses, you have a system that can handle pretty much any situation.
Status codes are also related to how we might authenticate software clients via REST. If you take another look at table 11.1, you might see a clue: the code 401.
Authenticating Software Clients
We have a strong access-control system in place for Hubbub, but in order for it to work, clients must first authenticate. Human users can do this via the home page, which contains the login form, but software clients don’t tend to work against HTML pages. They could access the login URL directly, but then they’d have to manage the relevant cookies.
A simpler solution is provided by HTTP: Basic authentication. Clients can provide the identity and credentials via HTTP headers in each request. If the credentials are valid, access is granted; otherwise the server returns a status code of 401.
There are two main problems with this approach. First, you have the overhead of passing the credentials in every request. The application also has to verify those credentials each time. This isn’t usually a significant problem. Second, the credentials are passed in plain text, so anybody can discover them by intercepting the traffic between the client and the server.
In Hubbub’s case, neither is an issue, because we don’t store any particularly confidential information. For systems that do, you should use Basic authentication in combination with HTTPS.
That concludes our coverage of REST. The basic principles are quite simple, and Grails offers a good selection of features to support your endeavor, from implementation to testing. Add in a suitable security plugin, and everything is covered.
As you have seen, REST is a powerful and flexible architecture pattern that can work well for many applications. That doesn’t mean it’s suitable for all situations, though. Consider a trivial example: an application that simply echoes back whatever is sent to it. In this case, there are no resources, just the “echo” operation, so REST is the wrong pattern to use.
A more appropriate solution would be an operation-oriented approach, and we’ll look at several such implementations in the next section that work with services rather than controllers.
11.4. Operation-oriented remoting
Grails’ REST support is inherently controller-based because it works with resources and representations of their content. Another remoting model in wide use is based instead on services and methods. This model is often easier for object-oriented programmers to understand and use because they call methods on objects in day-to-day life. It’s an approach they know well.
Java comes with its own remoting protocol that you may already be familiar with: Remote Method Invocation (RMI). This allows you to call methods on Java objects that are located in a different JVM or even on a different machine.
We’ll start by showing you how to make your Grails services available using RMI. Then, we’ll introduce the other protocols that you can use and discuss the pros and cons of each (including RMI). Finally, we’ll take a look at the big daddy of remote service invocation: Web Services. Although we aren’t fans of the technology, it’s used widely enough to deserve a discussion. But first, simple remote object invocations.
11.4.1. The Remoting plugin
An easy way to give software clients access to your application is to allow Remote Procedure Calls (RPCs) on your Grails service. The traditional way of doing this in Java applications is RMI, and we’ll show you how easy it is to RMI-enable your application by doing it for Hubbub’s post service.
Let’s get started by installing the Remoting plugin:
grails install-plugin remoting
The next step involves a bit more work. Grails services usually don’t implement interfaces, but without at least one interface, services can’t be exported via RMI or any of the other protocols. The interface is the API definition, and the Grails service is the implementation. That means we need to create an interface for our post service.
Where should the interface go? The best location is under the src directory, which is where all the code that isn’t Grails-specific goes. Should we use Java or Groovy? Interfaces are simple beasts, so there’s little benefit to writing them in Groovy. Write it in Java, and you can copy it directly into either a Java or Groovy project.
We’re only going to make the createPost() method available to remote clients, so the interface is pretty simple:
package com.grailsinaction;
public interface RemotePostService extends java.rmi.Remote {
long createPost(String userId, String content)
throws java.rmi.RemoteException;
}
This definition goes into the src/java/com/grailsinaction/RemotePostService.java file. We’ve called it RemotePostService rather than PostService to avoid a name clash with the service implementation. Unfortunately, RMI forces us to extend the Remote interface and declare all methods to throw RemoteException; otherwise the code would be simpler and cleaner.
Now that we have our interface, we need to modify the service implementation to use it. The signature of the current service method looks like this:
Post createPost(long userId, String content)
That doesn’t match the signature in the interface, so we either have to change the current implementation or add an extra method. Because we have code that depends on the existing method, we’ll add a new one. You can see the result in listing 11.5, which also shows you how to activate RMI for the service.
Listing 11.5. RMI-enabling the Hubbub post service
Enabling RMI for a service is straightforward: add a static expose property, and set its value to a list containing "rmi"
. You can also add other protocols to the list, as you’ll see shortly.
We deliberately avoided some complications by only using basic types in the method signature, which is why the interface method returns a long rather than an instance of Post. That doesn’t mean to say you can’t pass types like Post between the client and server (you most definitely can), but there’s a bit more work involved.
First, you need to make sure that the type is available on the client. A good approach is to package all interfaces and types that are shared between the client and server into a separate JAR file, which you then include as a dependency in both. In the case of Hubbub, we might call it hubbub-client.jar, for example. Second, the types should be in packages, because only classes in the default package can access other classes in it. Third, domain classes don’t always travel well over the wire, and clients definitely don’t need to see the constraints, mappings, and other internal information. For that and other reasons, people tend to copy the necessary data from the domain objects into what are known as Data Transfer Objects (DTOs)—simple classes that contain only data fields. That’s all there is to it.
A client is useful for testing the service, so we’ve included a simple one with the book’s sample source code. You can find it in the rmi-client directory under this chapter’s folder. It includes a simple Ant build file that allows you to run the client:
ant -Dmessage='This is a new Hubbub post via RMI.' run
If you’re feeling particularly adventurous, consider creating a second Grails application that uses the Remoting plugin to access the Hubbub post service via RMI. You can find out more about turning a Grails application into a remoting client from the plugin’s documentation: http://grails.org/Remoting+Plugin.
Now that you know how to export a Grails service via a remote protocol, let’s take a look at the available protocols and how they stack up against each other.
11.4.2. Comparing the remoting protocols
Along with RMI, the Remoting plugin provides support for three other protocols, which you can enable by adding these entries to the expose property: httpinvoker, hessian, and burlap. All of the protocols can happily work side by side, so you can put any combination of them in the expose list, or even all four. Which of them should you use, and why?
RMI
RMI is the original remoting protocol but not necessarily the best. RMI has existed since JDK 1.1, so it has been around for quite a while. If your clients will only ever be written in Java, RMI is worth contemplating, particularly as it doesn’t depend on anything beyond the JDK itself.
The major downside to RMI, other than it being Java-only, is that it’s invasive. Any interface you expose via RMI must extend java.rmi.Remote, and all its methods must be declared to throw java.rmi.RemoteException. These days, developers rightly expect that implementation details like this should not leak into a public API.
It also doesn’t help that RMI requires a separate registry server and doesn’t work over HTTP. That makes it a poor candidate in environments with firewalls. Fortunately, there’s an alternative that eliminates such problems and is even easier to use.
HTTP Invoker
Spring comes with a custom protocol that it calls “HTTP invoker.” In essence, it’s RMI over HTTP using standard Java serialization. That means it works well with firewalls, and you don’t need a separate registry because the URL defines the location of the remote service.
Those are big advantages over RMI, but our favorite improvement is that it’s non-invasive. Your classes and interfaces don’t need to extend or implement anything specific to HTTP invoker. For these reasons, we strongly recommend you use this protocol instead of RMI.
It would be remiss of us not to mention the downsides of the protocol. Like RMI, it only works with Java, but there’s an additional constraint: your client will have to use Spring. Whether this is a deal-breaker depends on your requirements, but we like to see it as a good reason to use Spring in all your applications!
Hessian and Burlap
The remaining two protocols come from the same company, Caucho. In fact, Burlap is an XML-based version of Hessian (a binary protocol). The major advantage of these protocols over the other two is that you can use them from non-Java clients. They also use HTTP as their transport, so you should have few problems with firewalls.
On the downside, you might run into problems with complex object graphs because the protocols don’t use standard Java serialization. You might also find that Hessian is a bit slower than HTTP invoker, but probably not enough to worry about.
We only recommend Hessian if non-Java clients need access to your application. As for Burlap, we don’t see much point in it. The only advantage it has over Hessian is that you can read the messages passed between client and server because they’re XML, so it may be worth switching to for debugging purposes.
As you have seen, both REST and remoting are nice and simple approaches to interapplication communication; but because they’re simple, they lack some heavy-duty features such as secure and reliable messaging, control over routing of messages, and plenty of others. If the simple solutions don’t meet your requirements, you probably need to enter the world of Web Services.
11.4.3. Web Services via SOAP
If you have been working in the Java field for any length of time, you’ll almost certainly have heard of Web Services. Many of the big names in software have put a lot of effort into developing and promoting the technology. Certainly, the promise of Web Services is alluring: secure and reliable messaging over the internet with complete interoperability between different systems. In practice, the associated complexity has caused problems on the interoperability front in the past. Efforts since then, such as the Web Services Interoperability Organization (WS-I), have improved the situation somewhat.
What are web services? Fundamentally, they’re services that interact with clients via SOAP, usually over HTTP. Some people use the term “web service” to describe a REST interface too, but the family of technologies that come under the “Web Services” umbrella are all SOAP-based.
SOAP itself was originally a Remote Procedure Call (RPC) mechanism using XML to describe the methods, their arguments, and the return data. Nowadays it’s often better to view it as a message-passing system where you send an XML message to the web service and get another one back. Figure 11.4 shows the simple request-response nature of most SOAP interactions and highlights the basic structure of a SOAP request. As you can see, the model of a SOAP request isn’t all that different from an HTTP request.
Figure 11.4. How a client interacts with a web service using the request-response model. Beneath that is the basic structure of a SOAP request—note the similarity to HTTP.
The world of Web Services is large and deserves a book to itself. We can’t do it credit in a short section here, but we’ll show you a quick example using one of the available Grails plugins and follow up with a discussion of when to use Web Services (if ever).
Web Service Plugins
At the time of writing, there are two main plugins for Web Services, each using a different Java library: xfire and axis2. Xfire is based on the XFire library, which has now been superceded by CXF. It’s likely that the plugin will start using CXF in the not too distant future. In the following example, we’ll use axis2, which is based on the Apache Axis 2 library.
To get started, we install the axis2 plugin in the usual way:
grails install-plugin axis2
Next, we export the post service as a web service by adding the string axis2 to the expose list:
class PostService {
static expose = [ "rmi", "axis2" ]
...
}
That’s it! Don’t believe us? Run the application, and point your browser at the URL: http://localhost:8080/hubbub/services/post?wsdl. All being well, you should see some rather horrific XML.
The name for the XML language is the Web Service Definition Language (WSDL), and it’s used to describe the interface for a web service, such as the permitted operations and the content of the SOAP messages. You should be able to pass this WSDL to your favorite Web Services library or toolkit, which will generate the classes you need to create a client application.
This example barely scratches the surface of SOAP and Web Services, so it doesn’t provide a particularly useful yardstick for either the complexity of the technology or its benefits. Should you go down this road?
To Soap, or Not to Soap
Although it appeared trivially easy to create a web service based on SOAP, if you want to get serious about Web Services, you need to start with the request and response messages rather than the service implementation, like with REST. That means dealing with WSDL and, by extension, XML namespaces—you might even need to dust off your XML Schema skills. As you saw, WSDL isn’t the most succinct XML language ever devised, nor is it easily comprehended by mere mortals. That said, there are tools to help with developing and using WSDL.
This complexity makes Web Services flexible and powerful, and many features have been built on top of SOAP and WSDL: attachments, for including binary data with a message; WS-Addressing, for determining where messages should go; WSS, for access control to web services and encrypting messages; and others. In fact, the range of extensions to the basic platform is truly bewildering.
We think this comes at the cost of developer productivity, because web services can be difficult to understand and debug. Certainly, tools can make life easier for you; but if we’ve learned anything from our years in software development, it’s that you eventually have to understand what’s happening under the hood. That’s not to say we think you should never touch Web Services—far from it. We recommend that you understand all the options available so that you can choose the appropriate solution for any given situation. Unnecessary complexity is a productivity killer, but a solution that doesn’t meet your requirements is also bad news.
Although REST has become popular in recent years, we hope we’ve demonstrated that it isn’t the only option when it comes to interapplication communication. When you control both the client and server, remoting can be far simpler and quicker to implement, whereas web services often form the basis of service-oriented architecture (SOA) solutions. The key as always is to use the right tool for the job.
11.5. Summary and best practices
Giving software clients access to your application requires a different approach from the usual HTML-based user interfaces. Whereas real users require data presented in a manner that’s easy to read and understand, software clients only care about the data. You could rely on clients screen-scraping your application (parsing the raw HTML) to interact with it, but that’s unreliable and inconvenient. As you’ve seen, it’s relatively easy to provide an interface to your application that makes life easy for software clients, so why not do it?
We’ve mainly focused in this chapter on implementing the REST architectural pattern, which is ideally suited to web applications and supported by Grails out of the box. By leveraging the HTTP protocol, you can easily create a secure and reliable API for your application.
Despite the heavy focus on REST, you should be aware of the other options available to you—the operation-oriented mechanisms can be a more appropriate fit for your API. Also, Java developers tend to find remoting technologies based on services easier to understand and work with, because they’re based on the familiar concepts of interfaces and methods.
Here are a few guidelines to ensure that your API becomes a favorite of developers:
- Use the easiest solution for your needs. Developers often find themselves caught up in the latest buzz surrounding some technology and feel obliged to use it because it’s the current or next big thing. Remember, you’re trying to develop a useful API as quickly as possible, not promote a particular piece of technology. Go for the simplest solution that matches your requirements. If you control both the client and the server, that might be remoting—even though it’s seriously uncool. If you have complex requirements, Web Services may be the only answer. For many situations, though, REST is both powerful and simple enough to be the right solution.
- Put some thought into your URLs. Good URLs are long-lived, consistent, and easy to understand. Although poor URLs are unlikely to have an impact on your API feature-wise, they can make life harder for those using it. If you do that, users will not bother using it. You can find plenty of articles online about good URL design.
- Take a message-centric approach with REST. If you develop a REST interface to your application that you expect to be used for years, it’s critical to decouple the API from the implementation. That means you should start by designing both the URLs and the messages that go in the relevant requests and responses. Once you have designed those messages, document them so that users know how you use your API. Also consider using either WADL or WSDL 2.0 for a formal specification.
- Use the appropriate HTTP methods. Map the various HTTP methods to actions that conform to the expectations and requirements of the methods. For example, don’t map GET to an action that adds, removes, or modifies data on the server. Doing so breaks the contract defined by HTTP.
- Make use of HTTP status codes. It’s easy to return a status code of 200 for all requests and use the content of the response to determine whether a request was successful or not. Easy, but bad practice. You would be giving up a flexible and well-defined mechanism for error reporting—one that can be understood by any HTTP client (such as a browser) and not just those written specifically for your API.
- Apply access control. You usually don’t want human users to have free access to all the data in the system, and the same goes for software clients. Remember, anyone can access your application via HTTP, not just well-behaved clients. Make sure that your REST interface has appropriate access control in place.
From interapplication communication, we move on to internal communication systems that enable you to coordinate either different parts of an application or multiple internal applications or subsystems.