Grails is more than just a web framework—it is an application framework. And almost all applications contain functionality that must be executed on a periodic basis (every 15 minutes, once an hour, twice a day, daily, weekly, month, quarterly, yearly). This is known as batch processing.
The Grails team anticipated the need for batch processing and decided to leverage a popular open source third-party enterprise job scheduling library: Quartz,1 from OpenSymphony. Since the Spring Framework is a core component of Grails, and the Spring Framework already includes a Quartz integration, this was a natural choice. A Quartz Grails plug-in makes it easy to use the Quartz library.
Quartz is similar to the Unix cron facility in that it provides the ability to execute a job in the future. However, Quartz is different from the Unix cron facility because it runs within the application server and has full access to all of the application components.
This chapter explores batch-processing functionality. We will start by installing the Quartz plug-in and creating a simple job. Then we will move on to creating a sample batch-reporting facility.
As we mentioned, Grails leverages Quartz for job-scheduling functionality. The Quartz plug-in2 integrates Quartz into Grails and makes Quartz easy to use.
To begin, from within the project directory, execute the following command:
>grails install-plugin quartz
__________
Welcome to Grails 1.0 - http://grails.org/
Licensed under Apache Standard License 2.0
Grails home is set to: C:Appsgrailsgrails-1.0
Base Directory: <<WORKSPACE>>collab-todo
Environment set to development
Running script C:Appsgrailsgrails-1.0scriptsInstallPlugin.groovy
[mkdir] Created dir: <<USER_HOME>>.grails1.0 pluginsquartz
[get] Getting: http://plugins.grails.org/grails-quartz/tags/RELEASE_0_2/grails-quartz-0.2.zip
[get] To: <<USER_HOME>>.grails1.0 pluginsquartzgrails-quartz-0.2.zip
.................................
[copy] Copying 1 file to <<WORKSPACE>>collab-todoplugins
[mkdir] Created dir: <<WORKSPACE>>collab-todopluginsquartz-0.2
[unzip] Expanding: <<WORKSPACE>>collab-todopluginsgrails-quartz-0.2.zip
into <<WORKSPACE>>collab-todopluginsquartz-0.2
Executing quartz-0.2 plugin post-install script ...
[mkdir] Created dir: <<WORKSPACE>>collab-todograils-appjobs
Compiling plugin quartz-0.2 ... ...
Compiling 9 source files to <<USER_HOME>>.grails1.0 projectscollab-todoclasses
Plugin quartz-0.2 installed
Plug-in provides the following new scripts:
------------------------------------------
grails create-job
As you can see, the plug-in installation created the quartz-0.2
directory under the plugins
directory, created the jobs
directory under grails-app
, and added a create-job
script to the Grails command line.
A job is a program that contains the code you wish to run. In Grails, the job defines what to do and when to do it.
As a simple demonstration of Quartz in action, let's create a job that prints a message and the current time. The first step is to create the job, as follows:
> grails create-job first
The command generates the FirstJob
class, in the grails/job
directory:
class FirstJob {
def timeout = 5000l // execute job once in 5 seconds
def execute() {
// execute task
}
}
Note Look closely at the timeout
value, and you'll see an l
after the 5000
. The l
makes the variable a Long
. Also notice that create-job
follows conventions just like other create-*
commands, and appends the suffix Job
to the end of the job name.
The create-job
command creates a skeleton job that is preconfigured to run once, five seconds after the application server starts. So, five seconds after the server starts, the code in the execute()
method will be executed. Add the following code to the execute()
method:
println "Hello from FirstJob: "+ new Date()
Listing 11-1 shows the completed FirstJob
class.
Listing 11-1. Completed FirstJob
class FirstJob {
def timeout = 5000l // execute job once in 5 seconds
def execute() {
println "Hello from FirstJob: "+ new Date()
}
Start the application by issuing the following command:
> grails run-app
While the comment for the timeout
property in Listing 11-1 could be interpreted to mean that the FirstJob
is executed only once, you can see from the output that it is executed every five seconds:
Welcome to Grails 1.0 - http://grails.org/
Licensed under Apache Standard License 2.0
Grails home is set to: C:Appsgrailsgrails-1.0
. . .
Loading with installed plug-ins: ["QuartzGrailsPlugin"
, "WebtestGrailsPlugin"] .
. . .
Running Grails application..
. . .
Server running. Browse to http://localhost:8080/collab-todo
Hello from FirstJob: Sun Dec 16 11:23:08 EST 2007
Hello from FirstJob: Sun Dec 16 11:23:13 EST 2007
Hello from FirstJob: Sun Dec 16 11:23:18 EST 2007
Now that you've seen how to create a simple job, let's move on to something a bit more useful: a batch-reporting facility.
As an example, we will build a batch-reporting facility that generates to-do reports and e-mails them to the user nightly. We will leverage a couple of services created in earlier chapters: EMailAuthenticatedService
from Chapter 8 and ReportService
from Chapter 10. Figure 11-1 shows is an overview of the nightly reporting process.
Figure 11-1. Nightly reporting process
The process starts with the NightlyReportJob.
When the NightlyReportJob
is invoked by Quartz, it immediately invokes the BatchService
. The BatchService
is the main control routine. It facilitates the interaction with other solution components. First, the BatchService
retrieves all User
objects that have an e-mail address. For each user, the BatchService
retrieves the Todo
objects. The BatchService
then uses the ReportService
to generate a PDF report. Finally, the BatchService
uses the EmailAuthenticatedService
to send the user an e-mail attachment of the report.
Building the batch-reporting facility requires the following steps:
NightlyReportJob
Issue the following command to create the NightlyReportJob
:
> grails create-job NightlyReport
In addition to the timeout
property, which you saw earlier in the FirstJob
job, several additional properties can be used to control job execution.
Setting the Name and Group
The name
and group
properties are used to help you identify jobs when interacting with the Quartz scheduler:
class NightlyReportJob {
def name = "NightlyReport" // Job name
def group = "CollabTodo" // Job group
Note Grails automatically binds the Hibernate session to the jobs. Having a bound session allows the job to retrieve data from the database. If for some (rare) reason, you don't want Grails to do this, you can tell Grails to not bind a Hibernate session by setting the sessionRequired
property to false
.
Controlling Execution Frequency
There are two techniques for controlling the job execution frequency:
Use the
startDelay
andtimeout
properties: These two properties allow you to control the execution frequency of the job. ThestartDelay
property delays starting the job for a number of milliseconds after the application starts up. This can be useful when you need to let the system start up before the job starts. Grails defaults thestartDelay
property to 0. Thetimeout
property is the number of milliseconds between executions of the job. Grails defaults thetimeout
property to 60,000 milliseconds, or 1 minute.Use the
cronExpression
property: For all of the Unix geeks out there, this works just as you would expect. It is a string that describes the execution frequency using a crontab format. If you're not familiar with this approach, don't worry—we'll explain the format in more detail here.
Both techniques have their place in controlling execution frequency. Determining which technique to use depends on the job requirements. If the job can be handled by a timer, then setting the startDelay
and timeout
properties should be sufficient, as in this example:
def startDelay = 20000 // Wait 20 seconds to start the job
def timeout = 60000 // Execute job once every 60 seconds
If the job is very time-sensitive, then using the cronExpression
property is probably more appropriate. But note that during development and initial testing of the job, you will probably want to use the startDelay/timeout
technique, and then switch to the cronExpression
approach later.
Caution Depending on the execution frequency and duration of the job, It's possible to have multiple instances of a job executing concurrently. This could happen if a job is long running and still running when the cronExpression
property causes it be invoked again. Having jobs running concurrently may or may not be desirable. By default, the Quartz plug-in permits the job to run concurrently. Most of the time, you probably won't want to allow a job to run concurrently. You can change this behavior by setting the concurrent
property on the job to false
.
A cron expression tells the job scheduler when to run the job. The cronExpression
property value is composed of six fields, separated by whitespace, representing seconds, minutes, hours, day, month, day of week, and an optional seventh field for the year. A cron expression expresses the fields left to right:
Seconds Minutes Hours DayOfMonth Month DayOfWeek Year
For example, we define a cronExpression
property to have the job run 1:00 a.m. every day as follows:
def cronExpression = "0 0 1 * * *" // Run every day at 1:00 a.m.
Table 11-1 describes the cron expression fields, and Table 11-2 summarizes some of the more commonly used special characters.3
__________
3. See the Quartz documentation for a more complete explanation of the special characters: http://www.opensymphony.com/quartz/wikidocs/CronTriggers Tutorial.html
Table 11-1.Cron Expression Fields
Field | Values | Special Characters |
Seconds | 0–59 | , - * / |
Minutes | 0–59 | , - * / |
Hours | 0–23 | , - * / |
DayOfMonth | 1–31 | , - * ? / L W C |
Month | 1–12 or JAN–DEC | , - * / |
DayOfWeek | 1–7 or SUN–SAT | , - * ? / L # |
Year(optional) | Empty or 1970–2099 | , - * / |
Table 11-2. Cron Expression Special Characters
Character | Function | Example |
* |
All values—matches all allowed values within a field. | * in the Hours field matches every hour of the day, 0–23. |
? |
No specific value—used to specify something in one of the two fields in which it is allowed, but not the other. | To execute a job on the tenth day of the month, no matter what day of the week that is, put 10 in the DayOfMonth field and ? in the DayOfWeek field. |
- |
Used to specify a range of values. | 2-6 in the DayOfWeek field causes the job to be invoked on Monday, Tuesday, Wednesday, Thursday, and Friday. |
, | Used to create a list of values. | MON,WED,FRI in the DayOfWeek field causes the job to be invoked on Monday, Wednesday, and Friday. |
/ |
Used to specify increments.character before the slash indicates when to start. The character after the slash represents the increment. | 0/15 in the Minutes field causes the job to be invoked on the quarter hour—0, 15, 30, and 45 minutes. |
Cron expressions are very powerful. With a little imagination, you can specify a multitude of times. Table 11-3 shows some sample cron expressions.
Table 11-3. Cron Expression Examples
Listing 11-2 shows the definition for the NightlyReportJob
. Notice that it includes both techniques for controlling execution frequency, with the startDelay
/timeout
definitions commented out.
Listing 11-2. NightlyReportJob Name, Group, and Execution Frequency Configuration
class NightlyReportJob {
def cronExpression = "0 0 1 * * *" // Run every day at 1:00 a.m.
def name = "Nightly" // Job name
def group = "CollabTodo" // Job group
// def startDelay = 20000 // Wait 20 seconds to start the job
// def timeout = 60000 // Execute job once every 60 seconds
You can see why the Grails team chose to integrate Quartz instead of creating something new. It is very powerful. Armed with this knowledge, you are ready to move on and implement the core logic of the nightly report job.
The next step is to leverage Spring's auto-wired dependency injection to inject the BatchService
into the job, as follows:
> grails create-service Batch
Listing 11-3 illustrates injection and execution of the BatchService
.
Listing 11-3. NightlyReportJob with Batch Service
class NightlyReportJob {
def cronExpression = "0 0 1 * * *" // Run every day at 1:00 a.m.
def name = "Nightly" // Job name
def group = "CollabTodo" // Job group
// def startDelay = 20000 // Wait 20 seconds to start the job
// def timeout = 60000 // Execute job once every 60 seconds
def batchService
def execute() {
log.info "Starting Nightly Job: "+new Date()
batchService.nightlyReports.call()
log.info "Finished Nightly Job: "+new Date()
}
}
The code is straightforward. It defines when the job is to run and delegate to the BatchService
.
The next step is to create the nightly
closure on the batch service. It will contain the code to retrieve the user's to-dos. Listing 11-4 illustrates adding the nightly closure and retrieving the user's to-dos.
Listing 11-4. Batch Service Nightly Closure
class BatchService
. . .
/*
* Runs nightly reports
*/
def nightlyReports = {
log.info "Running Nightly Reports Batch Job: "+new Date()
// 1. Gather user w/ email addresses.
def users = User.withCriteria {
isNotNull('email')
}
users?.each { user ->
// 2. Invoke report service for each user.
// Can't reuse ReportController because it makes too
// many assumptions, such as access to session.class.
//
// Reuse Report Service and pass appropriate params.
// Gather the data to be reported.
def inputCollection = Todo.findAllByOwner(user)
// To be completed in the next section
}
log.info "Completed Nightly Reports Batch Job: "+new Date()
}
The BatchService.nightlyReports
gets all users with an e-mail address, and then for each user, gets their to-dos and prepares to invoke the report service.
In Chapter 10, you used JasperReports to build a report facility. You can reuse components of the report facility to create a to-do report PDF to attach to the e-mail.
Your first thought might be to use the ReportController
. Well, that doesn't work. The report controller is dependent on the HTTP session and renders the PDF to the output stream. You need to go one level deeper and use the ReportService
directly.
We have already retrieved the user's to-dos. Now all we need to do is pass the to-dos, report template, and a username parameter to the report service. The highlighted section of Listing 11-5 illustrates the required steps.
Listing 11-5. Invoke the Report Service
class BatchService {
ReportService reportService // Inject ReportService
def nightlyReports = {
. . .
users?.each { user ->
// 2. Invoke Report Service for each user.
// Reuse Report Service and pass appropriate params.
// Gather the data to be reported.
def inputCollection = Todo.findAllByOwner(user)
Map params = new HashMap()
params.inputCollection = inputCollection
params.userName = user.firstName+" "+user.lastName
// Load the report file.
def reportFile = this.class.getClassLoader().getResource(
"web-app/reports/userTodo.jasper")
ByteArrayOutputStream byteArray = reportService.generateReport(reportFile
,
reportService.PDF_FORMAT,params )
Map attachments = new HashMap()
attachments.put("TodoReport.pdf", byteArray.toByteArray())
// 3. Email results to the user.
sendNotificationEmail(user, attachments)
}
}
The new code works as follows:
ReportService
into the BatchService
.HashMap
of parameters that will be passed to the ReportService
. The parameters include the list of to-dos for the current user.reportService.generateReport
to pass the report template, report format (PDF), and parameters.Now that you have a PDF report, the next step is to e-mail it to the user.
In Chapter 8, you implemented an SMTP e-mail service, called EMailAuthenticatedService
. You can use your e-mail service to send the to-do report to the user. Listing 11-6 contains the code required to create and send the e-mail.
Listing 11-6. Sending the E-mail
01 class BatchService implements ApplicationContextAware
{
02 boolean transactional = false
03
04 public void setApplicationContext(ApplicationContext applicationContext) {
05 this.applicationContext = applicationContext
06 }
07 def ApplicationContext applicationContext
08 def EMailAuthenticatedService EMailAuthenticatedService // injected
09
10 ReportService reportService
11
12 def nightlyReports = {
13
14 . . .
15
16 // Load the report file
17 def reportFile = this.class.getClassLoader().getResource(
18 "web-app/reports/userTodo.jasper")
19 ByteArrayOutputStream byteArray =
20 reportService.generateReport(reportFile,
21 reportService.PDF_FORMAT,params )
22
23 Map attachments = new HashMap()
24 attachments.put("TodoReport.pdf", byteArray.toByteArray())
25
26 // 3. Email results to the user.
27 sendNotificationEmail(user, attachments)
28 }
29 log.info "Completed Nightly Batch Job: "+new Date()
30 }
31
32 def private sendNotificationEmail = {User user, Map attachments ->
33 def emailTpl = this.class.getClassLoader().getResource(
34 "web-app/WEB-INF/nightlyReportsEmail.gtpl")
35 def binding = ["user": user]
36 def engine = new SimpleTemplateEngine()
37 def template = engine.createTemplate(emailTpl).make(binding)
38 def body = template.toString()
39 def email = [
40 to: [user.email]
,
41 subject: "Your Collab-Todo Report",
42 text: body
43 ]
44 try {
45 EMailProperties eMailProperties =
46 applicationContext.getBean("eMailProperties")
47 eMailAuthenticatedService.sendEmail(email, eMailProperties, attachments)
48 } catch (MailException ex) {
49 log.error("Failed to send emails", ex)
50 }
51 }
52 }
The highlighted lines contain the changes made to the batch service. Lines 1 and 4–7 make the batch service (Spring) application context-aware; in other words, the Spring application context is injected into the service. You will use the application context later to look up some information. Line 8 takes advantages of Spring auto-wiring to inject the EmailAuthenticatedService
. Lines 23 and 24 add the PDF report to a map of attachments for e-mail. Line 27 invokes a local sendNotificationEmail
closure.
Lines 32–51 contain the code to send the to-do report e-mail to the user. Line 33 loads an e-mail template. Lines 36–38 use the Groovy SimpleTemplateEngine
4 to generate the e-mail body. Lines 39–43 define a map of e-mail parameters that will be passed to the e-mail service. Line 45 uses the Spring application context to look up e-mail properties, including the "from" address. Line 47 invokes the e-mail service, sending the e-mail map, e-mail properties, and the attachments.
__________
This chapter demonstrated Grails' ability to reuse open source, third-party Java libraries. You installed the Quartz plug-in, created a simple job, and saw how to control the frequency of execution using the timeout
property.
Next, you started to build the batch-reporting facility. You created a NightlyReportJob
and configured it to run at 1:00 a.m. using the cronExpression
property. You learned that cron expressions are very robust and provide fine-grained control over when the NightlyReportJob
is invoked.
The NightlyReportJob
delegated to a batch service that was injected using Spring auto-wiring injection and invoked nightlyReports
. nightlyReports
iterated through a list of users, gathered their to-dos, invoked the report service built in Chapter 10 to generate a PDF attachment, and e-mailed the attachment to the user using the EmailAuthenticatedService
built in Chapter 8.
This chapter provided a brief introduction to the Quartz package. For more information, check out the Quartz web site.5
__________