Chapter 7. Taking a peek inside with the Actuator
This chapter covers
- Actuator web endpoints
- Adjusting the Actuator
- Shelling into a running application
- Securing the Actuator
Have you ever tried to guess what’s inside a wrapped gift? You shake it, weigh it, and measure it. And you might even have a solid idea as to what’s inside. But until you open it up, there’s no way of knowing for sure.
A running application is kind of like a wrapped gift. You can poke at it and make reasonable guesses as to what’s going on under the covers. But how can you know for sure? If only there were some way that you could peek inside a running application, see how it’s behaving, check on its health, and maybe even trigger operations that influence how it runs?
In this chapter, we’re going to explore Spring Boot’s Actuator. The Actuator offers production-ready features such as monitoring and metrics to Spring Boot applications. The Actuator’s features are provided by way of several REST endpoints, a remote shell, and Java Management Extensions (JMX). We’ll start by looking at the Actuator’s REST endpoints, which offer the most complete and well-known way of working with the Actuator.
7.1. Exploring the Actuator’s endpoints
The key feature of Spring Boot’s Actuator is that it provides several web endpoints in your application through which you can view the internals of your running application. Through the Actuator, you can find out how beans are wired together in the Spring application context, determine what environment properties are available to your application, get a snapshot of runtime metrics, and more.
The Actuator offers a baker’s dozen of endpoints, as described in table 7.1.
Table 7.1. Actuator endpoints
|
HTTP method |
Path |
Description |
|---|---|---|
| GET | /autoconfig | Provides an auto-configuration report describing what auto-configuration conditions passed and failed. |
| GET | /configprops | Describes how beans have been injected with configuration properties (including default values). |
| GET | /beans | Describes all beans in the application context and their relationship to each other. |
| GET | /dump | Retrieves a snapshot dump of thread activity. |
| GET | /env | Retrieves all environment properties. |
| GET | /env/{name} | Retrieves a specific environment value by name. |
| GET | /health | Reports health metrics for the application, as provided by HealthIndicator implementations. |
| GET | /info | Retrieves custom information about the application, as provided by any properties prefixed with info. |
| GET | /mappings | Describes all URI paths and how they’re mapped to controllers (including Actuator endpoints). |
| GET | /metrics | Reports various application metrics such as memory usage and HTTP request counters. |
| GET | /metrics/{name} | Reports an individual application metric by name. |
| POST | /shutdown | Shuts down the application; requires that endpoints.shutdown.enabled be set to true. |
| GET | /trace | Provides basic trace information (timestamp, headers, and so on) for HTTP requests. |
To enable the Actuator endpoints, all you must do is add the Actuator starter to your build. In a Gradle build specification, that dependency looks like this:
compile 'org.springframework.boot:spring-boot-starter-actuator'
For a Maven build, the required dependency is as follows:
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency>
Or, if you’re using the Spring Boot CLI, the following @Grab should do the trick:
@Grab('spring-boot-starter-actuator')
No matter which technique you use to add the Actuator to your build, auto-configuration will kick in when the application is running and you enable the Actuator.
The endpoints in table 7.1 can be organized into three distinct categories: configuration endpoints, metrics endpoints, and miscellaneous endpoints. Let’s take a look at each of these endpoints, starting with the endpoints that provide insight into the configuration of your application.
7.1.1. Viewing configuration details
One of the most common complaints lodged against Spring component-scanning and autowiring is that it’s hard to see how all of the components in an application are wired together. Spring Boot auto-configuration makes this problem even worse, as there’s even less Spring configuration. At least with explicit configuration, you could look at the XML file or the configuration class and get an idea of the relationships between the beans in the Spring application context.
Personally, I’ve never had this concern. Maybe it’s because I realize that before Spring came along there wasn’t any map of the components in my applications.
Nevertheless, if it concerns you that auto-configuration hides how beans are wired up in the Spring application context, then I have some good news! The Actuator has endpoints that give you that missing application component map as well as some insight into the decisions that auto-configuration made when populating the Spring application context.
Getting a bean wiring report
The most essential endpoint for exploring an application’s Spring context is the /beans endpoint. This endpoint returns a JSON document describing every single bean in the application context, its Java type, and any of the other beans it’s injected with. By performing a GET request to /beans (http://localhost:8080/beans when running locally), you’ll be given information similar to what’s shown in the following listing.
Listing 7.1. The /beans endpoint exposes the beans in the Spring application context
Listing 7.1 is an abridged listing of the beans from the reading-list application. As you can see, all of the bean entries carry five pieces of information about the bean:
- bean—The name or ID of the bean in the Spring application context
- resource—The location of the physical .class file (often a URL into the built JAR file, but this might vary depending on how the application is built and run)
- dependencies—A list of bean IDs that this bean is injected with
- scope—The bean’s scope (usually singleton, as that is the default scope)
- type—The bean’s Java type
Although the beans report doesn’t draw a specific picture of how the beans are wired together (for example, via properties or constructor arguments), it does help you visualize the relationships of the beans in the application context. Indeed, it would be reasonably easy to write a utility that processes the beans report and produces a graphical representation of the bean relationships. Be aware, however, that the full bean report includes many beans, including many auto-configured beans, so such a graphic could be quite busy.
Explaining auto-configuration
Whereas the /beans endpoint produces a report telling you what beans are in the Spring application context, the /autoconfig endpoint might help you figure out why they’re there—or not there.
As mentioned in chapter 2, Spring Boot auto-configuration is built upon Spring conditional configuration. It provides several configuration classes with @Conditional annotations referencing conditions that decide whether or not beans should be automatically configured. The /autoconfig endpoint provides a report of all the conditions that are evaluated, grouping them by which conditions passed and which failed.
Listing 7.2 shows an excerpt from the auto-configuration report produced for the reading-list application with one passing and one failing condition.
Listing 7.2. An auto-configuration report for the reading-list app
In the positiveMatches section, you’ll find a condition used to decide whether or not Spring Boot should auto-configure a JdbcTemplate bean. The match is named DataSourceAutoConfiguration.JdbcTemplateConfiguration#jdbcTemplate, which indicates the specific configuration class where this condition is applied. The type of condition is an OnBeanCondition, which means that the condition’s outcome is determined by the presence or absence of a bean. In this case, the message property makes it clear that the condition checks for the absence of a bean of type JdbcOperations (the interface that JbdcTemplate implements). If no such bean has already been configured, then this condition passes and a JdbcTemplate bean will be created.
Similarly, under negativeMatches, there’s a condition that decides whether or not to configure an ActiveMQ. This decision is an OnClassCondition, and it hinges on the presence of ActiveMQConnectionFactory in the classpath. Because ActiveMQConnectionFactory isn’t in the classpath, the condition fails and ActiveMQ will not be auto-configured.
Inspecting configuration properties
In addition to knowing how your application beans are wired together, you might also be interested in learning what environment properties are available and what configuration properties were injected on the beans.
The /env endpoint produces a list of all of the environment properties available to the application, whether they’re being used or not. This includes environment variables, JVM properties, command-line parameters, and any properties provided in an application.properties or application.yml file.
The following listing shows an abridged example of what you might get from the /env endpoint.
Listing 7.3. The /env endpoint reports all properties available
Essentially, any property source that can provide properties to a Spring Boot application will be listed in the results of the /env endpoint along with the properties provided by that endpoint.
In listing 7.3, properties come from application configuration (application.yml), Spring profiles, servlet context initialization parameters, the system environment, and JVM system properties. (In this case, there are no profiles or servlet context initialization parameters.)
It’s common to use properties to carry sensitive information such as database or API passwords. To keep that kind of information from being exposed by the /env endpoint, any property named (or whose last segment is) “password”, “secret”, or “key” will be rendered as “” in the response from /env. For example, if there’s a property named “database.password”, it will be rendered in the /env response like this:
"database.password":"******"
The /env endpoint can also be used to request the value of a single property. Just append the property name to /env when making the request. For example, requesting /env/amazon.associate_id will yield a response of “habuma-20” (in plain text) when requested against the reading-list application.
As you’ll recall from chapter 3, these environment properties come in handy when using the @ConfigurationProperties annotation. Beans annotated with @ConfigurationProperties can have their instance properties injected with values from the environment. The /configprops endpoint produces a report of how those properties are set, whether from injection or otherwise. Listing 7.4 shows an excerpt from the configuration properties report for the reading-list application.
Listing 7.4. A configuration properties report
The first item in this excerpt is the amazonProperties bean we created in chapter 3. This report tells us that it’s annotated with @ConfigurationProperties to have a prefix of “amazon”. And it shows that the associateId property is set to “habuma-20”. This is because in application.yml, we set the amazon.associateId property to “habuma-20”.
You can also see an entry for serverProperties—it has a prefix of “server” and several properties that we can work with. Here they all have default values, but you can change any of them by setting a property prefixed with “server”. For example, you could change the port that the server listens on by setting the server.port property.
Aside from giving insight into how configuration properties are set in the running application, this report is also useful as a quick reference showing all of the properties that you could set. For example, if you weren’t sure how to set the maximum number of threads in the embedded Tomcat server, a quick look at the configuration properties report would give you a clue that server.tomcat.maxThreads is the property you’re looking to set.
Producing endpoint-to-controller map
When an application is relatively small, it’s usually easy to know how all of its controllers are mapped to endpoints. But once the web interface exceeds more than a handful of controllers and request-handling methods, it might be helpful to have a list of all of the endpoints exposed by the application.
The /mappings endpoint provides such a list. Listing 7.5 shows an excerpt of the mappings report from the reading-list application.
Listing 7.5. The controller/endpoint mappings for the reading-list app
Here we see a handful of endpoint mappings. The key for each mapping is a string containing what appears to be the attributes of Spring MVC’s @RequestMapping annotation. Indeed, this string gives you a good idea of how the controller is mapped, even if you haven’t seen the source code. The value of each mapping has two properties: bean and method. The bean property identifies the name of the Spring bean that the mapping comes from. The method property gives the fully qualified method signature of the method for which the mapping is being reported.
The first two mappings are for the request-handing methods in our application’s ReadingListController. The first shows that an HTTP GET request for the root path (“/”) will be handled by the readersBooks() method. The second shows that a POST request is mapped to the addToReadingList() method.
The next mapping is for an Actuator-provided endpoint. An HTTP GET request for the /autoconfig endpoint will be handled by the invoke() method of Spring Boot’s EndpointMvcAdapter class. There are, of course, many other Actuator endpoints that aren’t shown in listing 7.5, but those were omitted from the listing for brevity’s sake.
The Actuator’s configuration endpoints are great for seeing how your application is configured. But it’s also interesting and useful to see what’s actually happening within your application while it’s running. The metrics endpoints help give a snapshot into an application’s runtime internals.
7.1.2. Tapping runtime metrics
When you go to the doctor for a physical exam, the doctor performs a battery of tests to see how your body is performing. Some of them, such as determining your blood type, are important but will not change over time. These kinds of tests give the doctor insight into how your body is configured. Other tests give the doctor a snapshot into how your body is performing during the visit. Your heart rate, blood pressure, and cholesterol level are useful in helping the doctor evaluate your health. These metrics are temporal and likely to change over time, but they’re still helpful runtime metrics.
Similarly, taking a snapshot of the runtime metrics is helpful in evaluating the health of an application. The Actuator offers a handful of endpoints that enable you to perform a quick checkup on your application while it’s running. Let’s take a look at them, starting with the /metrics endpoint.
Viewing application metrics
There are a lot of interesting and useful bits of information about any running application. Knowing the application’s memory circumstances (available vs. free), for instance, might help you decide if you need to give the JVM more or less memory to work with. For a web application, it can be helpful knowing at a glance, without scouring web server log files, if there are any requests that are failing or taking too long to serve.
The /metrics endpoint provides a snapshot of various counters and gauges in a running application. The following listing shows a sample of what the /metrics endpoint might give you.
Listing 7.6. The metrics endpoint provides several useful pieces of runtime data
{
mem: 198144,
mem.free: 144029,
processors: 8,
uptime: 1887794,
instance.uptime: 1871237,
systemload.average: 1.33251953125,
heap.committed: 198144,
heap.init: 131072,
heap.used: 54114,
heap: 1864192,
threads.peak: 21,
threads.daemon: 19,
threads: 21,
classes: 9749,
classes.loaded: 9749,
classes.unloaded: 0,
gc.ps_scavenge.count: 22,
gc.ps_scavenge.time: 122,
gc.ps_marksweep.count: 2,
gc.ps_marksweep.time: 156,
httpsessions.max: -1,
httpsessions.active: 1,
datasource.primary.active: 0,
datasource.primary.usage: 0,
counter.status.200.beans: 1,
counter.status.200.env: 1,
counter.status.200.login: 3,
counter.status.200.metrics: 2,
counter.status.200.root: 6,
counter.status.200.star-star: 9,
counter.status.302.login: 3,
counter.status.302.logout: 1,
counter.status.302.root: 5,
gauge.response.beans: 169,
gauge.response.env: 165,
gauge.response.login: 3,
gauge.response.logout: 0,
gauge.response.metrics: 2,
gauge.response.root: 11,
gauge.response.star-star: 2
}
As you can see, a lot of information is provided by the /metrics endpoint. Rather than examine these metrics line by line, which would be tedious, table 7.2 groups them into categories by the type of information they offer.
Table 7.2. Gauges and counters reported by the /metrics endpoint
Notice that some of these metrics, such as the data source and Tomcat session metrics, are only available if the necessary components are in play in the running application. You can also register your own custom application metrics, as you’ll see in section 7.4.3.
The HTTP counters and gauges demand a bit more explanation. The number following the counter.status prefix is the HTTP status code. What follows that is the path requested. For instance, the metric named counter.status.200.metrics indicates the number of times that the /metrics endpoint was served with an HTTP status of 200 (OK).
The HTTP gauges are similarly structured but report a different kind of metrics. They’re all prefixed with gauge.response, indicating that they are gauges for HTTP responses. Following that prefix is the path that the gauge refers to. The value of the metric indicates the time in milliseconds that it took to serve that path the most recent time it was served. For instance, the gauge.response.beans metric in table 7.6 indicates that it took 169 milliseconds to serve that request the last time it was served.
You’ll notice that there are a few special cases for the counter and gauge paths. The root path refers to the root path or /. And star-star is a catchall that refers to any path that Spring determines is a static resource, including images, JavaScript, and stylesheets. It also includes any resource that can’t be found, which is why you’ll often see a counter.status.404.star-star metric indicating the count of requests that were met with HTTP 404 (NOT FOUND) status.
Whereas the /metrics endpoint fetches a full set of all available metrics, you may only be interested in a single metric. To fetch only one metric value, append the metric’s key to the URL path when making the request. For example, to fetch only the amount of free memory, perform a GET request for /metrics/mem.free:
$ curl localhost:8080/metrics/mem.free 144029
It may be useful to know that even though the result from /metrics/{name} appears to be plain text, the Content-Type header in the response is set to “application/json;charset=UTF-8”. Therefore, it can be processed as JSON if you need to do so.
Tracing web requests
Although the /metrics endpoint gives you some basic counters and timers for web requests, those metrics lack any details. Sometimes it can be helpful, especially when debugging, to know more about the requests that were handled. That’s where the /trace endpoint can be handy.
The /trace endpoint reports details of all web requests, including details such as the request method, path, timestamp, and request and response headers. Listing 7.7 shows an excerpt of the /trace endpoint’s output containing a single request trace entry.
Listing 7.7. The /trace endpoint records web request details
[
...
{
"timestamp": 1426378239775,
"info": {
"method": "GET",
"path": "/metrics",
"headers": {
"request": {
"accept": "*/*",
"host": "localhost:8080",
"user-agent": "curl/7.37.1"
},
"response": {
"X-Content-Type-Options": "nosniff",
"X-XSS-Protection": "1; mode=block",
"Cache-Control":
"no-cache, no-store, max-age=0, must-revalidate",
"Pragma": "no-cache",
"Expires": "0",
"X-Frame-Options": "DENY",
"X-Application-Context": "application",
"Content-Type": "application/json;charset=UTF-8",
"Transfer-Encoding": "chunked",
"Date": "Sun, 15 Mar 2015 00:10:39 GMT",
"status": "200"
}
}
}
}
]
As indicated by the method and path properties, you can see that this trace entry is for a /metrics request. The timestamp property (as well as the Date header in the response) tells you when the request was handled. The headers property carries header details for both the request and the response.
Although listing 7.7 only shows a single trace entry, the /trace endpoint will report trace details for the 100 most recent requests, including requests for the /trace endpoint itself. It maintains the trace data in an in-memory trace repository. Later, in section 7.4.4, you’ll see how to create a custom trace repository implementation for a more permanent tracing of requests.
Dumping thread activity
In addition to request tracing, thread activity can also be useful in determining what’s going on in a running application. The /dump endpoint produces a snapshot of current thread activity.
Listing 7.8. The /dump endpoint provides a snapshot of an application’s threads
[
{
"threadName": "container-0",
"threadId": 19,
"blockedTime": -1,
"blockedCount": 0,
"waitedTime": -1,
"waitedCount": 64,
"lockName": null,
"lockOwnerId": -1,
"lockOwnerName": null,
"inNative": false,
"suspended": false,
"threadState": "TIMED_WAITING",
"stackTrace": [
{
"className": "java.lang.Thread",
"fileName": "Thread.java",
"lineNumber": -2,
"methodName": "sleep",
"nativeMethod": true
},
{
"className": "org.apache.catalina.core.StandardServer",
"fileName": "StandardServer.java",
"lineNumber": 407,
"methodName": "await",
"nativeMethod": false
},
{
"className": "org.springframework.boot.context.embedded.
tomcat.TomcatEmbeddedServletContainer$1",
"fileName": "TomcatEmbeddedServletContainer.java",
"lineNumber": 139,
"methodName": "run",
"nativeMethod": false
}
],
"lockedMonitors": [],
"lockedSynchronizers": [],
"lockInfo": null
},
...
]
The complete thread dump report includes every thread in the running application. To save space, listing 7.8 shows an abridged entry for a single thread. As you can see, it includes details regarding the blocking and locking status of the thread, among other thread specifics. There’s also a stack trace that, in this case, indicates the thread is a Tomcat container thread.
Monitoring application health
If you’re ever wondering if your application is up and running or not, you can easily find out by requesting the /health endpoint. In the simplest case, the /health endpoint reports a simple JSON structure like this:
{"status":"UP"}
The status property reports that the application is up. Of course it is. It doesn’t really matter what the response is; any response at all is an indication that the application is running. But the /health endpoint has more information than a simple “UP” status.
Some of the information offered by the /health endpoint can be sensitive, so unauthenticated requests are only given the simple health status response. If the request is authenticated (for example, if you’re logged in), more health information is exposed. Here’s some sample health information reported for the reading-list application:
{
"status":"UP",
"diskSpace": {
"status":"UP",
"free":377423302656,
"threshold":10485760
},
"db":{
"status":"UP",
"database":"H2",
"hello":1
}
}
Along with the basic health status, you’re also given information regarding the amount of available disk space and the status of the database that the application is using.
All of the information reported by the /health endpoints is provided by one or more health indicators, including those listed in table 7.3, that come with Spring Boot.
Table 7.3. Spring Boot’s out-of-the-box health indicators
|
Key |
Reports |
|
|---|---|---|
| ApplicationHealthIndicator | none | Always “UP” |
| DataSourceHealthIndicator | db | “UP” and database type if the database can be contacted; “DOWN” status otherwise |
| DiskSpaceHealthIndicator | diskSpace | “UP” and available disk space, and “UP” if available space is above a threshold; “DOWN” if there isn’t enough disk space |
| JmsHealthIndicator | jms | “UP” and JMS provider name if the message broker can be contacted; “DOWN” otherwise |
| MailHealthIndicator | “UP” and the mail server host and port if the mail server can be contacted; “DOWN” otherwise | |
| MongoHealthIndicator | mongo | “UP” and the MongoDB server version; “DOWN” otherwise |
| RabbitHealthIndicator | rabbit | “UP” and the RabbitMQ broker version; “DOWN” otherwise |
| RedisHealthIndicator | redis | “UP” and the Redis server version; “DOWN” otherwise |
| SolrHealthIndicator | solr | “UP” if the Solr server can be contacted; “DOWN” otherwise |
These health indicators will be automatically configured as needed. For example, DataSourceHealthIndicator will be automatically configured if javax.sql.DataSource is available in the classpath. ApplicationHealthIndicator and DiskSpaceHealthIndicator will always be configured.
In addition to these out-of-the-box health indicators, you’ll see how to create custom health indicators in section 7.4.5.
7.1.3. Shutting down the application
Suppose you need to kill your running application. In a microservice architecture, for instance, you might have multiple instances of a microservice application running in the cloud. If one of those instances starts misbehaving, you might decide to shut it down and let the cloud provider restart the failed application for you. In that scenario, the Actuator’s /shutdown endpoint will prove useful.
In order to shut down your application, you can send a POST request to /shutdown. For example, you can shut down your application using the curl command-line tool like this:
$ curl -X POST http://localhost:8080/shutdown
Obviously, the ability to shut down a running application is a dangerous thing, so it’s disabled by default. Unless you’ve explicitly enabled it, you’ll get the following response from the POST request:
{"message":"This endpoint is disabled"}
To enable the /shutdown endpoint, configure the endpoints.shutdown.enabled property to true. For example, add the following lines to application.yml to enable the /shutdown endpoint:
endpoints:
shutdown:
enabled: true
Once the /shutdown endpoint is enabled, you want to make sure that not just anybody can kill your application. You should secure the /shutdown endpoint, requiring that only authorized users are allowed to bring the application down. You’ll see how to secure Actuator endpoints in section 7.5.
7.1.4. Fetching application information
Spring Boot’s Actuator has one more endpoint you might find useful. The /info endpoint reports any information about your application that you might want to expose to callers. The default response to a GET request to /info looks like this:
{}
Obviously, an empty JSON object isn’t very useful. But you can add any information to the /info endpoint’s response by simply configuring properties prefixed with info. For example, suppose you want to provide a contact email in the /info endpoint response. You could set a property named info.contactEmail like this in application.yml:
info: contactEmail: [email protected]
Now if you request the /info endpoint, you’ll get the following response:
{
"contactEmail":"[email protected]"
}
Properties in the /info response can also be nested. For example, suppose that you want to provide both a support email and a support phone number. In application.yml, you might configure the following properties:
info:
contact:
email: [email protected]
phone: 1-888-555-1971
The JSON returned from the /info endpoint will include a contact property that itself has email and phone properties:
{
"contact":{
"email":"[email protected]",
"phone":"1-888-555-1971"
}
}
Adding properties to the /info endpoint is just one of many ways you can customize Actuator behavior. Later in section 7.4, we’ll look at other ways that you can configure and extend the Actuator. But first, let’s see how to secure the Actuator’s endpoints.
7.2. Connecting to the Actuator remote shell
You’ve seen how the Actuator provides some very useful information over REST endpoints. An optional way to dig into the internals of a running application is by way of a remote shell. Spring Boot integrates with CRaSH, a shell that can be embedded into any Java application. Spring Boot also extends CRaSH with a handful of Spring Boot-specific commands that offer much of the same functionality as the Actuator’s endpoints.
In order to use the remote shell, you’ll need to add the remote shell starter as a dependency. The Gradle dependency you’ll need looks like this:
compile("org.springframework.boot:spring-boot-starter-remote-shell")
If you’re building your project with Maven, you’ll need the following dependency in your pom.xml file:
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-remote-shell</artifactId> </dependency>
And if you’re developing an application to run with the Spring Boot CLI, the following @Grab is what you’ll need:
@Grab("spring-boot-starter-remote-shell")
With the remote shell added as a dependency, you can now build and run the application. As it’s starting up, watch for the password to be written to the log in a line that looks something like this:
Using default security password: efe30c70-5bf0-43b1-9d50-c7a02dda7d79
The username that goes with that password is “user”. The password itself is randomly generated and will be different each time you run the application.
Now you can use an SSH utility to connect to the shell, which is listening for connections on port 2000. If you use the UNIX ssh command to connect to the shell, it might look something like this:
~% ssh user@localhost -p 2000 Password authentication Password: . ____ _ __ _ _ /\ / ___'_ __ _ _(_)_ __ __ _ ( ( )\___ | '_ | '_| | '_ / _` | \/ ___)| |_)| | | | | || (_| | ) ) ) ) ' |____| .__|_| |_|_| |_\__, | / / / / =========|_|==============|___/=/_/_/_/ :: Spring Boot :: (v1.3.0.RELEASE) on habuma.local >
Great! You’re connected to the shell. Now what?
The remote shell offers almost two dozen commands that you can execute within the context of the running application. Most of those commands come out of the box with CRaSH, but Spring Boot adds a handful of commands. These Spring Boot-specific commands are listed in table 7.4.
Table 7.4. CRaSH shell commands provided by Spring Boot
|
Command |
Description |
|---|---|
| autoconfig | Produces an auto-configuration explanation report. Similar to the /autoconfig endpoint, except that the results are plain text instead of JSON. |
| beans | Displays the beans in the Spring application context. Similar to the /beans endpoint. |
| endpoint | Invokes an Actuator endpoint. |
| metrics | Displays Spring Boot metrics. Similar to the /metrics endpoint, except presented as a live list of metrics that’s updated as the values change. |
Let’s take a look at how to use each of the shell commands added by Spring Boot.
7.2.1. Viewing the autoconfig report
The autoconfig command produces a report that’s very similar to the report produced by the Actuator’s /autoconfig endpoint. Figure 7.1 shows an abridged screenshot of the output produced by the autoconfig command.
Figure 7.1. Output of autoconfig command
As you can see, the results are split into two groups—positive matches and negative matches—just like the results from the /autoconfig endpoint. In fact, the only significant difference is that the autoconfig command produces a textual report whereas the /autoconfig endpoint produces JSON output. Otherwise, they are the same.
We’re not going to dwell on any of the shell commands provided natively by CRaSH, but you might want to consider piping the results of the autoconfig command to CRaSH’s less command:
> autoconfig | less
The less command is much like the same-named command in Unix shells; it enables you to page back and forth through a file. The autoconfig output is lengthy, but piping it to less will make it easier to read and navigate.
7.2.2. Listing application beans
The output from the autoconfig shell command and the /autoconfig endpoint were similar but different. In contrast, you’ll find that the results from the beans command are exactly the same as those from the /beans endpoint, as the screenshot in figure 7.2 shows.
Figure 7.2. Output of beans command
Just like the /beans endpoint, the beans command produces a list of all beans in the Spring application context, along with any dependency beans, in JSON format.
7.2.3. Watching application metrics
The metrics shell command produces the same information as the Actuator /metrics endpoint. But unlike the /metrics endpoint, which produces a snapshot of the current metrics in JSON format, the metrics command takes over the shell and displays its results in a live dashboard. Figure 7.3 shows what the metrics dashboard looks like.
Figure 7.3. The metrics dashboard
It’s difficult to demonstrate the live dashboard behavior of the metrics command with a static figure in a book. But try to imagine that as memory, heap, and threads are consumed and released and as classes are loaded, the numbers shown in the dashboard will change to reflect the current values.
Once you’re finished looking at the metrics offered by the metrics command, press Ctrl-C to return to the shell.
7.2.4. Invoking Actuator endpoints
You’ve probably realized by now that not all of the Actuator’s endpoints have corresponding commands in the shell. Does that mean that the shell can’t be a full replacement for the Actuator endpoints? Will you still have to query the endpoints directly for some of the internals offered by the Actuator? Although the shell doesn’t pair a command up with each of the endpoints, the endpoint command enables you to invoke Actuator endpoints from within the shell.
First, you need to know which endpoint you want to invoke. You can get a list of endpoints by issuing endpoint list at the shell prompt, as shown in figure 7.4. Notice that the endpoints listed are referred to by their bean names, not by their URL paths.
Figure 7.4. Getting a list of endpoints
When you want to call one of the endpoints from the shell, you’ll use the endpoint invoke command, giving it the endpoint’s bean name without the “Endpoint” suffix. For example, to invoke the health endpoint, you’d issue endpoint invoke health at the shell prompt, as shown in figure 7.5.
Figure 7.5. Invoking the health endpoint
Notice that the results coming back from the endpoint are in the form of a raw, unformatted JSON document. Although it may be nice to be able to invoke the Actuator’s endpoints from within the shell, the results can be a bit difficult to read. Out of the box, there’s not much that can be done about that. But if you’re feeling adventurous, you can create a custom CRaSH shell command that accepts the unformatted JSON via a pipe and pretty-prints it. And you can always cut and paste it into a tool of your choosing for further review or formatting.
7.3. Monitoring your application with JMX
In addition to the endpoints and the remote shell, the Actuator also exposes its endpoints as MBeans to be viewed and managed through JMX (Java Management Extensions). JMX is an attractive option for managing your Spring Boot application, especially if you’re already using JMX to manage other MBeans in your applications.
All of the Actuator’s endpoints are exposed under the org.springframework.boot domain. For example, suppose you want to view the request mappings for your application. Figure 7.6 shows the request mapping endpoint as viewed in JConsole.
Figure 7.6. Request mapping endpoint as viewed in JConsole
As you can see, the request mapping endpoint is found under requestMappingEndpoint, which is under Endpoint in the org.springframework.boot domain. The Data attribute contains the JSON reported by the endpoint.
As with any MBean, the endpoint MBeans have operations that you can invoke. Most of the endpoint MBeans only have accessor operations that return the value of one of their attributes. But the shutdown endpoint offers a slightly more interesting (and destructive!) operation, as shown in figure 7.7
Figure 7.7. Shutdown button invokes the endpoint.
If you ever need to shut down your application (or just like living dangerously), the shutdown endpoint is there for you. As shown in figure 7.7, it’s waiting for you to click the “shutdown” button to invoke the endpoint. Be careful, though—there’s no turning back or “Are you sure?” prompt. The very next thing you’ll see is shown in figure 7.8.
Figure 7.8. Application immediately shuts down.
After that, your application will have been shut down. And because it’s dead, there’s no way it could possibly expose another MBean operation for restarting it. You’ll have to restart it yourself, the same way you started it in the first place.
7.4. Customizing the Actuator
Although the Actuator offers a great deal of insight into the inner workings of a running Spring Boot application, it may not be a perfect fit for your needs. Maybe you don’t need everything it offers and want to disable some of it. Or maybe you need to extend it with metrics custom-suited to your application.
As it turns out, the Actuator can be customized in several ways, including the following:
- Renaming endpoints
- Enabling and disabling endpoints
- Defining custom metrics and gauges
- Creating a custom repository for storing trace data
- Plugging in custom health indicators
We’re going to see how to customize the Actuator, bending it to meet our needs. We’ll start with one of the simplest customizations: renaming the Actuator’s endpoints.
7.4.1. Changing endpoint IDs
Each of the Actuator endpoints has an ID that’s used to determine that endpoint’s path. For example, the /beans endpoint has beans as its default ID.
If an endpoint’s path is determined by its ID, then it stands to reason that you can change an endpoint’s path by changing its ID. All you need to do is set a property whose name is endpoints.endpoint-id.id.
To demonstrate how this works, consider the /shutdown endpoint. It responds to POST requests sent to /shutdown. Suppose, however, that you’d rather have it handle POST requests sent to /kill. The following YAML shows how you might assign a new ID, and therefore a new path, to the /shutdown endpoint:
endpoints:
shutdown:
id: kill
There are a couple of reasons you might want to rename an endpoint and change its path. The most obvious is that you might simply want to name the endpoints to match the terminology used by your team. But you might also think that renaming an endpoint will hide it from anyone who might be familiar with the default names, thus creating a sense of security by obscurity.
Unfortunately, renaming an endpoint doesn’t really secure it. At best, it will only slow down a hacker looking to gain access to an endpoint. We’ll look at how you can secure Actuator endpoints in section 7.5. For now, let’s see how to completely disable any (or all) endpoints that you don’t want anyone to have access to.
7.4.2. Enabling and disabling endpoints
Although all of the Actuator endpoints are useful, you may not want or need all of them. By default, all of the endpoints (except for /shutdown) are enabled. We’ve already seen how to enable the /shutdown endpoint by setting endpoints.shutdown.enabled to true (in section 7.1.1). In the same way, you can disable any of the other endpoints by setting endpoints._endpoint-id.enabled to false.
For example, suppose you want to disable the /metrics endpoint. All you need to do is set the endpoints.metrics.enabled property to false. In application.yml, that would look like this:
endpoints:
metrics:
enabled: false
If you find that you only want to leave one or two of the endpoints enabled, it might be easier to disable them all and then opt in to the ones you want to enable. For example, consider the following snippet from application.yml:
endpoints:
enabled: false
metrics:
enabled: true
As shown here, all of the Actuator’s endpoints are disabled by setting endpoints.enabled to false. Then the /metrics endpoint is re-enabled by setting endpoints.metrics.enabled to true.
7.4.3. Adding custom metrics and gauges
In section 7.1.2, you saw how to use the /metrics endpoint to fetch information about the internal metrics of a running application, including memory, garbage collection, and thread metrics. Although these are certainly useful and informative metrics, you may want to define custom metrics to capture information specific to your application.
Suppose, for instance, that we want a metric that reports how many times a user has saved a book to their reading list. The easiest way to capture this number is to increment a counter every time the addToReadingList() method is called on ReadingListController. A counter is simple enough to implement, but how would you expose the running total along with the other metrics exposed by the /metrics endpoint?
Let’s also suppose that we want to capture a timestamp for the last time a book was saved. We could easily capture that by calling System.currentTimeMillis(), but how could we report that time in the /metrics endpoint?
As it turns out, the auto-configuration that enables the Actuator also creates an instance of CounterService and registers it as a bean in the Spring application context. CounterService is an interface that defines three methods for incrementing, decrementing, or resetting a named metric, as shown here:
package org.springframework.boot.actuate.metrics;
public interface CounterService {
void increment(String metricName);
void decrement(String metricName);
void reset(String metricName);
}
Actuator auto-configuration will also configure a bean of type GaugeService, an interface similar to CounterService that lets you record a single value to a named gauge metric. GaugeService looks like this:
package org.springframework.boot.actuate.metrics;
public interface GaugeService {
void submit(String metricName, double value);
}
We don’t need to implement either of these interfaces; Spring Boot already provides implementations for them both. All we must do is inject the CounterService and GaugeService instances into any other bean where they’re needed, and call the methods to update whichever metrics we want.
For the metrics we want, we’ll need to inject the CounterService and GaugeService beans into ReadingListController and call their methods from the addToReadingList() method. Listing 7.9 shows the necessary changes to ReadingListController.
Listing 7.9. Using injected gauge and counter services
This change to ReadingListController uses autowiring to inject the CounterService and GaugeService beans via the controller’s constructor, which then stores them in instance variables. Then, each time that the addToReadingList() method handles a request, it will call counterService.increment("books.saved") and gaugeService.submit("books.last.saved") to adjust our custom metrics.
Although CounterService and GaugeService are simple to use, there are some metrics that are hard to capture by incrementing a counter or recording a gauge value. For those cases, we can implement the PublicMetrics interface and provide as many custom metrics as we want. The PublicMetrics interface defines a single metrics() method that returns a collection of Metric objects:
package org.springframework.boot.actuate.endpoint;
public interface PublicMetrics {
Collection<Metric<?>> metrics();
}
To put PublicMetrics to work, suppose that we want to be able to report some metrics from the Spring application context. The time when the application context was started and the number of beans and bean definitions might be interesting metrics to include. And, just for grins, let’s also report the number of beans that are annotated as @Controller. Listing 7.10 shows the implementation of PublicMetrics that will do the job.
Listing 7.10. Publishing custom metrics
The metrics() method will be called by the Actuator to get any custom metrics that ApplicationContextMetrics provides. It makes a handful of calls to methods on the injected ApplicationContext to fetch the numbers we want to report as metrics. For each one, it creates an instance of Metric, specifying the metric’s name and the value, and adds the Metric to the list to be returned.
As a consequence of creating ApplicationContextMetrics as well as using CounterService and GaugeService in ReadingListController, we get the following entries in the response from the /metrics endpoint:
{
...
spring.context.startup-date: 1429398980443,
spring.beans.definitions: 261,
spring.beans: 272,
spring.controllers: 2,
books.count: 1,
gauge.books.save.time: 1429399793260,
...
}
Of course, the actual values for these metrics will vary, depending on how many books you’ve added and the times when you started the application and last saved a book. In case you’re wondering, spring.controllers is 2 because it’s counting ReadingListController as well as the Spring Boot–provided BasicErrorController.
7.4.4. Creating a custom trace repository
By default, the traces reported by the /trace endpoint are stored in an in-memory repository that’s capped at 100 entries. Once it’s full, it starts rolling off older trace entries to make room for new ones. This is fine for development purposes, but in a production application the higher traffic may result in traces being discarded before you ever get a chance to see them.
One way to remedy that problem is to declare your own InMemoryTraceRepository bean and set its capacity to some value higher than 100. The following configuration class should increase the capacity to 1000 entries:
package readinglist;
import org.springframework.boot.actuate.trace.InMemoryTraceRepository;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class ActuatorConfig {
@Bean
public InMemoryTraceRepository traceRepository() {
InMemoryTraceRepository traceRepo = new InMemoryTraceRepository();
traceRepo.setCapacity(1000);
return traceRepo;
}
}
Although a tenfold increase in the repository’s capacity should keep a few of those trace entries around a bit longer, a sufficiently busy application might still discard traces before you get a chance to review them. And because this is an in-memory trace repository, we should be careful about increasing the capacity too much, as it will have an impact on our application’s memory footprint.
Alternatively, we could store those trace entries elsewhere—somewhere that’s not consuming memory and that will be more permanent. All we need to do is implement Spring Boot’s TraceRepository interface:
package org.springframework.boot.actuate.trace;
import java.util.List;
import java.util.Map;
public interface TraceRepository {
List<Trace> findAll();
void add(Map<String, Object> traceInfo);
}
As you can see, TraceRepository only requires that we implement two methods: one that finds all stored Trace objects and another that saves a Trace given a Map containing trace information.
For demonstration purposes, perhaps we could create an instance of TraceRepository that stores trace entries in a MongoDB database. Listing 7.11 shows such an implementation of TraceRepository.
Listing 7.11. Saving trace data to Mongo
The findAll() method is straightforward enough, asking the injected MongoOperations to find all Trace objects. The add() method is only slightly more interesting, instantiating a Trace object given the current date/time and the Map of trace info before saving it via MongoOperations.save(). The only question you might have is where MongoOperations comes from.
In order for MongoTraceRepository to work, we’re going to need to make sure that we have a MongoOperations bean in the Spring application context. Thanks to Spring Boot starters and auto-configuration, that’s simply a matter of adding the MongoDB starter as a dependency. The Gradle dependency you need is as follows:
compile("org.springframework.boot:spring-boot-starter-data-mongodb")
If your project is built with Maven, this is the dependency you’ll need:
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-mongodb</artifactId> </dependency>
By adding this starter, Spring Data MongoDB and supporting libraries will be added to the application’s classpath. And because those are in the classpath, Spring Boot will auto-configure the beans necessary to support working with a MongoDB database, including a MongoOperations bean. The only other thing you’ll need to do is be sure that there’s a MongoDB server running for MongoOperations to talk to.
7.4.5. Plugging in custom health indicators
As we’ve seen, the Actuator comes with a nice set of out-of-the-box health indicators for common needs such as reporting the health of a database or message broker that the application is using. But what if your application interacts with some system for which there’s no health indicator?
Because our application includes links to Amazon for books in the reading list, it might be interesting to report whether or not Amazon is reachable. Sure, it’s not likely that Amazon will go down, but stranger things have happened. So let’s create a health indicator that reports whether Amazon is available. Listing 7.12 shows a HealthIndicator implementation that should do the job.
Listing 7.12. Defining a custom Amazon health indicator
The AmazonHealth class isn’t terribly fancy. The health() method simply uses Spring’s RestTemplate to perform a GET request to Amazon’s home page. If it works, it returns a Health object indicating that Amazon is “UP”. On the other hand, if an exception is thrown while requesting Amazon’s home page, then health() returns a Health object indicating that Amazon is “DOWN”.
The following excerpt from the /health endpoint’s response shows what you might see if Amazon is unreachable:
{
"amazonHealth": {
"status": "DOWN"
},
...
}
You wouldn’t believe how long I had to wait for Amazon to crash so that I could get that result![1]
Not really. I just disconnected my computer from the network. No network, no Amazon.
If you’d like to add additional information to the health record beyond a simple status, you can do so by calling withDetail() on the Health builder. For example, to add the exception’s message as a reason field in the health record, the catch block could be changed to return a Health object created like this:
return Health.down().withDetail("reason", e.getMessage()).build();
As a result of this change, the health record might look like this when the request to Amazon fails:
"amazonHealth": {
"reason": "I/O error on GET request for
"http://www.amazon.com":www.amazon.com;
nested exception is java.net.UnknownHostException:
www.amazon.com",
"status": "DOWN"
},
You can add as many additional details as you want by calling withDetail() for each additional field you want included in the health record.
7.5. Securing Actuator endpoints
We’ve seen that many of the Actuator endpoints expose information that may be considered sensitive. And some, such as the /shutdown endpoint, are dangerous and can be used to bring your application down. Therefore, it’s very important to be able to secure these endpoints so that they’re only available to authorized clients.
As it turns out, the Actuator endpoints can be secured the same way as any other URL path: with Spring Security. In a Spring Boot application, this means adding the Security starter as a build dependency and letting security auto-configuration take care of locking down the application, including the Actuator endpoints.
In chapter 3, we saw how the default security auto-configuration results in all URL paths being secured, requiring HTTP Basic authentication where the username is “user” and the password is randomly generated at startup and written to the log file. This was not how we wanted to secure the application, and it’s likely not how you want to secure the Actuator either.
We’ve already added some custom security configuration to restrict the root URL path (/) to only authenticated users with READER access. To lock down Actuator endpoints, we’ll need to make a few changes to the configure() method in SecurityConfig.java.
Suppose, for instance, that we want to lock down the /shutdown endpoint, requiring that the user have ADMIN access. Listing 7.13 shows the changes required in the configure() method.
Listing 7.13. Securing the /shutdown endpoint
Now the only way to access the /shutdown endpoint is to authenticate as a user with ADMIN access.
The custom UserDetailsService we created in chapter 3, however, is coded to only apply READER access to users it looks up via the ReaderRepository. Therefore, you may want to create a smarter UserDetailsService implementation that is able to apply ADMIN access to some users. Optionally, you can configure an additional authentication implementation, such as the in-memory authentication shown in listing 7.14.
Listing 7.14. Adding an in-memory admin authentication user
With the in-memory authentication added, you can authenticate with “admin” as the username and “s3cr3t” as the password and be granted both ADMIN and READER access.
Now the /shutdown endpoint is locked down for everyone except users with ADMIN access. But what about the Actuator’s other endpoints? Assuming you want to lock them down with ADMIN access as for /shutdown, you can list each of them in the call to antMatchers(). For example, to lock down /metrics and /configprops as well as /shutdown, call antMatchers() like this:
.antMatchers("/shutdown", "/metrics", "/configprops")
.access("hasRole('ADMIN')")
Although this approach will work, it’s only suitable if you want to secure a small subset of the Actuator endpoints. It becomes unwieldy if you use it to lock down all of the Actuator’s endpoints.
Rather than explicitly list all of the Actuator endpoints when calling antMatchers(), it’s much easier to use wildcards to match them all with a simple Ant-style expression. This is challenging, however, because there’s not a lot in common between the endpoint paths. And we can’t apply ADMIN access to “/**” because then everything except for the root path (/) would require ADMIN access.
Instead, consider setting the endpoint’s context path by setting the management.context-path property. By default, this property is empty, which is why all of the Actuator’s endpoint paths are relative to the root path. But the following entry in application.yaml will prefix them all with /mgmt.
management: context-path: /mgmt
Optionally, you can set it in application.properties like this:
management.context-path=/mgmt
With management.context-path set to /mgmt, all Actuator endpoints will be relative to the /mgmt path. For example, the /metrics endpoint will be at /mgmt/metrics.
With this new path, we now have a common prefix to work with when assigning ADMIN access restriction to the Actuator endpoints:
.antMatchers("/mgmt/**").access("hasRole('ADMIN')")
Now all requests beginning with /mgmt, which includes all Actuator endpoints, will require an authenticated user who has been granted ADMIN access.
7.6. Summary
It can be difficult to know for sure what’s going on inside a running application. Spring Boot’s Actuator opens a portal into the inner workings of a Spring Boot application, exposing components, metrics, and gauges to help understand what makes the application tick.
In this chapter, we started by looking at the Actuator’s web endpoints—REST endpoints that expose runtime details over HTTP. These include endpoints for viewing all of the beans in the Spring application context, auto-configuration decisions, Spring MVC mappings, thread activity, application health, and various metrics, gauges, and counters.
In addition to web endpoints, the Actuator also offers two alternative ways to dig into the information it provides. The remote shell offers a way to securely shell into the application itself and issue commands that expose much of the same data as the Actuator’s endpoints. Meanwhile, all of the Actuator’s endpoints are exposed as MBeans that can be monitored and managed by a JMX client.
Next, we took a look at how to customize the Actuator. We saw how to change Actuator endpoint paths by changing the endpoint IDs as well as how to enable and disable endpoints. We also plugged in a few custom metrics and created a custom trace repository to replace the default in-memory trace repository.
Finally, we looked at how to secure the Actuator’s endpoints, restricting access to authorized users.
Coming up in the next chapter, we’ll see how to take an application from the coding phase to production, looking at how Spring Boot helps when deploying an application to a variety of platforms, including traditional application servers and the cloud.