Before Modularization

As a first step, you need to ensure that your library can be used as is with Java 9. Many applications will be using your library on the classpath, even on Java 9. Furthermore, library maintainers need to get their libraries in shape for use as automatic modules in applications. In many cases, no code changes are necessary. The only changes to make at this point are to prevent showstoppers, such as the use of encapsulated or removed types from the JDK.

Before making a library into a module (or collection of modules), you should take the same initial step as when migrating applications, as detailed in Chapter 7. Ensuring that the library runs on Java 9 means it should not use encapsulated types in the JDK. If it does use such types, library users are possibly faced with warnings, or exceptions (if they run with the recommended --illegal-access=deny setting). This forces them to use --add-opens or --add-exports flags. Even if you document this with your library, it’s not a great user experience. Usually the library is just one of many in an application, so tracking down all the right command-line flags is painful for users. It’s better to use jdeps to find uses of encapsulated APIs in the library and change them to the suggested replacement. Use jdeps -jdkinternals, as described in “Using jdeps to Find Removed or Encapsulated Types and Their Alternatives”, to quickly spot problems in this area. When these replacement APIs are only available starting with Java 9, you cannot directly use them while supporting earlier Java releases. In “Multi-Release JARs”, you’ll see how multi-release JARs can solve this problem.

At this stage, we’re not yet creating a module descriptor for the library. We can postpone thinking about what packages need to be exported or not. Also, any dependencies the library has on other libraries can be left implicit. Whether the library is put on the classpath, or on the module path as an automatic module, it can still access everything it needs without explicit dependencies.

After this step, the library can be used on Java 9. We’re not using any new features from Java 9 in the library implementation yet. In fact, if there are no uses of encapsulated or removed APIs, there’s no need to even recompile the library.

Choosing a Library Module Name

There are only two hard things in Computer Science: cache invalidation and naming things.

Phil Karlton

It is important at this point to think about the name your module should have when it becomes a real module later. What is a good name for a library module? On the one hand, you want the name to be simple and memorable. On the other hand, we are talking about a widely reusable library, so the name must be unique. There can be only one module on the module path for any given name.

A long-standing tradition for making globally unique names in the Java world is to use the reverse DNS notation. When you have a library called mylibrary, its module name could be com.mydomain.mylibrary. Applying this naming convention to non-reusable application modules is unnecessary, but with libraries, the extra visual noise is warranted. Several open source Java libraries, for example, are named spark. If no precautions are taken by these library maintainers, they may end up claiming the same module name. That would mean applications could no longer use those libraries together in an application. Claiming a reverse DNS–based module name is the best way to prevent clashes.

Digits are allowed in module names. It may be tempting to add a version number in the module name (e.g., com.mydomain.mylibrary2). Don’t do this. Versioning is a separate concern from identifying your library. A version can be set when creating the modular JAR (as discussed in “Versioned Modules”), but should never be part of the module name. Just because you upgrade your library with a major version doesn’t mean the identification of your library should change. Some popular libraries already painted themselves into this corner a long time ago. The Apache commons-lang library, for example, decided on the commons-lang3 name when moving from version 2 to 3. Currently, versions belong to the realm of build tools and artifact repositories, not the module system. An updated version of a library should not lead to changes in module descriptors.

You’ve learned that automatic modules derive their name from the JAR filename in Chapter 8. Unfortunately, the ultimate filename is often dictated by build tools or other processes that are out of the hands of library maintainers. Applications using the library as an automatic module then require your library through the derived name in their module descriptors. When your library switches to being an explicit module later, you’re stuck with this derived name. That’s a problem when the derived filename isn’t quite correct or uniquely identifying. Or worse, when different people use different filenames for your library. Expecting every application that uses the library to later update their requires clauses to your new module name is unrealistic.

That puts library maintainers in an awkward position. Even though the library itself is not a module yet, it will be used as such through the automatic modules feature. And when applications start doing so, you’re effectively stuck with an automatically derived module name that may or may not be what you want it to be. To solve this conundrum, an alternative way of reserving a module name is possible.

Start by adding an Automatic-Module-Name: <module_name> entry to a nonmodular JAR in its META-INF/MANIFEST.MF. When the JAR is used as automatic module, it will assume the name as defined in the manifest, instead of deriving it from the JAR filename. Library maintainers can now define the name that the library module should have, without creating a module descriptor yet. Simply adding the new entry with the correct module name to MANIFEST.MF and repackaging the library is enough. The jar command has an -m <manifest_file> option telling it to add entries from a given file to the generated MANIFEST.MF in the JAR (➥ chapter10/modulename):

jar -cfm mylibrary.jar src/META-INF/MANIFEST.MF -C out/ .

With this command, the entries from src/META-INF/MANIFEST.MF are added to the generated manifest in the output JAR.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-jar-plugin</artifactId>
  <configuration>
    <archive>
      <manifestEntries>
        <Automatic-Module-Name>
          com.mydomain.mylibrary
        </Automatic-Module-Name>
      </manifestEntries>
    </archive>
  </configuration>
</plugin>

Reserving a module name with Automatic-Module-Name in the manifest is something you should do as quickly as possible. Naming is hard, and picking a name should be done deliberately. However, after settling on a module name, reserving it with Automatic-Module-Name is straightforward. It is a low-effort, high-impact move: no code changes or recompilation necessary.

Why not create a module descriptor instead? There are several reasons.

First, it involves thinking about what packages to expose or not. You can export everything explicitly, as would happen implicitly when the library is used as an automatic module. However, sanctioning this behavior explicitly in your module descriptor is something you cannot easily take back later. People using your library as an automatic module know that access to all packages is a side effect of it being an automatic module, subject to change later.

Second, and more important, your library can have external dependencies itself. With a module descriptor, your library is no longer an automatic module on the module path. That means it won’t automatically have a requires relation with all other modules and the classpath (unnamed module) anymore. All dependencies must be made explicit in the module descriptor. Those external dependencies may not yet be modularized themselves, which prevents your library from having a correct module descriptor. Never publish your library with a dependency on an automatic module if that dependency doesn’t at least have the Automatic-Module-Name manifest entry yet. The name of such a dependency is unstable in that case, causing your module descriptor to be invalid when the dependency’s (derived) module name changes

Last, a module descriptor must be compiled with the Java 9 compiler. These are all significant steps that take time to get right. Reserving the module name with a simple Automatic-Module-Name entry in the manifest before taking all these steps is sensible.

Creating a Module Descriptor

Now that the library is properly named and usable with Java 9, it’s time to think about turning it into an explicit module. For now, we assume the library is a single JAR (mylibrary.jar) to be converted to a single module. Later, you may want to revisit the packaging of the library and split it up further.

You have two choices with regard to creating a module descriptor: create one from scratch, or use jdeps to generate one based on the current JAR. In any case, it’s important that the module descriptor features the same module name as chosen earlier for the Automatic-Module-Name entry in the manifest. This makes the new module a drop-in replacement for the old version of the library when it was used as an automatic module. With a module descriptor in place, the manifest entry can be dropped.

Our example mylibrary (➥ chapter10/generate_module_descriptor) is fairly simple and consists of two classes in two packages. The central class MyLibrary contains the following code:

package com.javamodularity.mylibrary;

import com.javamodularity.mylibrary.internal.Util;

import java.sql.SQLException;
import java.sql.Driver;
import java.util.logging.Logger;

public class MyLibrary {

   private Util util = new Util();
   private Driver driver;

   public MyLibrary(Driver driver) throws SQLException {
     Logger logger = driver.getParentLogger();
     logger.info("Started MyLibrary");
   }

}

Functionally, it doesn’t really matter what this code does; the important part is in the imports. When creating a module descriptor, we need to establish what other modules we require. Visual inspection shows the MyLibrary class uses types from java.sql and java.logging in the JDK. The ...internal.Util class comes from a different package in the same mylibrary.jar. Instead of trying to come up with the right requires clauses ourselves, we can use jdeps to list the dependencies for us. Besides listing the dependencies, jdeps can even generate an initial module descriptor:

jdeps --generate-module-info ./out mylibrary.jar

This results in a generated module descriptor in out/mylibrary/module-info.java:

module mylibrary {
    requires java.logging;
    requires transitive java.sql;
    exports com.javamodularity.mylibrary;
    exports com.javamodularity.mylibrary.internal;
}

jdeps analyzes the JAR file and reports dependencies to java.logging and java.sql. Interestingly, the former gets a requires clause, whereas the latter gets a requires transitive clause. That’s because the java.sql types used in MyLibrary are part of the public, exported API. The java.sql.Driver type is used as an argument to the public constructor of MyLibrary. Types from java.logging, on the other hand, are used only in the implementation of MyLibrary and are not exposed to library users. By default, all packages are exported in the jdeps-generated module descriptor.

You may think this module descriptor provides the same behavior as when mylibrary is used as an automatic module. And that’s largely the case. However, automatic modules are open modules as well. The generated module descriptor doesn’t define an open module, nor does it open any packages. Users of the library will notice this only when doing deep reflection on types from mylibrary. When you expect users of your library to do this, you can generate an open module descriptor instead:

jdeps --generate-open-module ./out mylibrary.jar

This generates the following module descriptor:

open module mylibrary {
    requires java.logging;
    requires transitive java.sql;
}

All packages will be open because an open module is generated. No exports statements are generated by this option. If you add exports for all packages to this open module, its behavior is close to using the original JAR as an automatic module.

It’s preferable to create a nonopen module, exporting only the minimum number of packages necessary. One of the main draws of turning a library into a module is to benefit from strong encapsulation, something an open module does not offer.

Exporting all packages rarely is the right thing to do. For the mylibrary example, it makes sense to remove exports com.javamodularity.mylibrary.internal. There is no need for users of mylibrary to depend on internal implementation details.

Furthermore, if your library uses reflection, jdeps won’t find those dependencies. You need to add the right requires clauses for modules you reflectively load from yourself. These can be requires static if the dependency is optional, as discussed in “Compile-Time Dependencies”. If your library uses services, these uses clauses must be added manually as well. Any services that are provided (through files in META-INF/services) are automatically picked up by jdeps and turned into provides .. with clauses.

Finally, jdeps suggests a module name based on the filename, as with automatic modules. The caveats discussed in “Choosing a Library Module Name” still apply. For libraries, it’s better to create a fully qualified name by using reverse-DNS notation. In this example, com.javamodularity.mylibrary is the preferred module name. When the JAR you’re generating the module descriptor from already contains an Automatic-Module-Name manifest entry, this name is suggested instead.

Updating a Library with a Module Descriptor

After creating or generating a module descriptor, we’re left with a module-info.java that still needs to be compiled. Only Java 9 can compile module-info.java, but that does not mean you need to switch compilation to Java 9 for your whole project. In fact, it’s possible to update the existing JAR (compiled with an earlier Java version) with just the compiled module descriptor. Let’s see how that works for mylibrary.jar, where we take the generated module-info.java and add it:

mkdir mylibrary
cd mylibrary
jar -xf ../mylibrary.jar 
cd ..
javac -d mylibrary out/mylibrary/module-info.java 
jar -uf mylibrary.jar -C mylibrary module-info.class 

Extract the class files into ./mylibrary.

Compile just module-info.java with the Java 9 compiler into the same directory as the extracted classes.

Update the existing JAR file with the compiled module-info.class.

With these steps, you can create a modular JAR from a pre-Java 9 JAR. The module descriptor is compiled into the same directory as the extracted classes. This way, javac can see all the existing classes and packages that are mentioned in the module descriptor, so it won’t produce errors. It’s possible to do this without having access to the sources of the library. No recompilation of existing code is necessary—unless, of course, code needs to be changed, for example, to avoid use of encapsulated JDK APIs.

After these steps, the resulting JAR file can be used in various setups:

• On the classpath in pre-Java 9 versions

• On the module path in Java 9 and later

• On the classpath in Java 9

The compiled module descriptor is ignored when the JAR is put on the classpath in earlier Java versions. Only when the JAR is used on the module path with Java 9 or later does the module descriptor come into play.

Targeting Older Java Versions

What if you need to compile the library sources as well as the module descriptor? In many cases, you’ll want to target a Java release before 9 with your library. You can achieve this in several ways. The first is to use two JDKs to compile the sources and the module descriptor separately.

Let’s say we want mylibrary to be usable on Java 7 and later. In practice, this means the library source code can’t use any language features introduced after Java 7, nor any APIs added after Java 7. By using two JDKs, we can ensure that our library sources don’t depend on Java 7+ features, while still being able to compile the module descriptor:

jdk7/bin/javac -d mylibrary <all sources except module-info>
jdk9/bin/javac -d mylibrary src/module-info.java

Again, it’s essential for both compilation runs to target the same output directory. The resulting classes can then be packaged into a modular JAR just as in the previous example. Managing multiple JDKs can be a bit cumbersome. A new feature, added in JDK 9, allows the use of the latest JDK to target an earlier version.

The mylibrary example can be compiled using the new --release flag with just JDK 9:

jdk9/bin/javac --release 7 -d mylibrary <all sources except module-info>
jdk9/bin/javac --release 9 -d mylibrary src/module-info.java

This new flag is guaranteed to support at least three major previous releases from the current JDK. In the case of JDK 9, this means you can compile toward JDK 6, 7, and 8. As an added bonus, you get to benefit from bug fixes and optimizations in the JDK 9 compiler even when your library itself targets earlier releases. If you need to support even earlier versions of Java, you can always fall back to using multiple JDKs.

Library Module Dependencies

So far, we’ve assumed that the library to be migrated doesn’t have any dependencies beyond modules in the JDK. In practice, that’s not always the case. There are two main reasons a library has dependencies:

1. The library consists of multiple, related JARs.

2. External libraries are used by the library.

In the first case, there are dependencies between JARs within the library. In the second case, the library needs other external JARs. Both scenarios are addressed next.

Internal Dependencies

We’re going to explore the first scenario based on a library you’ve already seen in Chapter 8: Jackson. Jackson already consists of multiple JARs. The example was based on Jackson Databind, with two related Jackson JARs, as shown in Figure 10-1.

Figure 10-1. Three related Jackson JARs

Turning those JARs into modules is the obvious thing to do, thereby preserving the current boundaries. Luckily, jdeps can also create several module descriptors at once for related JAR files (➥ chapter10/generate_module_descriptor_jackson):

jdeps --generate-module-info ./out *.jar

This results in three generated module descriptors:

module jackson.annotations {
    exports com.fasterxml.jackson.annotation;
}

module jackson.core {
    exports com.fasterxml.jackson.core;
    // Exports of all other packages omitted for brevity.
    provides com.fasterxml.jackson.core.JsonFactory with
        com.fasterxml.jackson.core.JsonFactory;
}

module jackson.databind {
    requires transitive jackson.annotations;
    requires transitive jackson.core;
    requires java.desktop;
    requires java.logging;
    requires transitive java.sql;
    requires transitive java.xml;
    exports com.fasterxml.jackson.databind;
    // Exports of all other packages omitted for brevity.
    provides com.fasterxml.jackson.core.ObjectCodec with
        com.fasterxml.jackson.databind.ObjectMapper;
}

We can see in the last two module descriptors that jdeps also takes into account service providers. When the JAR contains service provider files (see “ServiceLoader Before Java 9” for more information on this mechanism), they are translated into provides .. with clauses.

Warning

Services uses clauses, on the other hand, cannot be automatically generated by jdeps. These must be added manually based on ServiceLoader usage in the library.

The jacksons.databind descriptor requires the right platform modules based on the jdeps analysis. Furthermore, it requires the correct other Jackson library modules, whose descriptors are generated in the same run. Jackson’s latent structure automatically becomes explicit in the generated module descriptors. Of course, the hard task of demarcating the actual API of the modules is left to the Jackson maintainers. Leaving all packages exported is definitely not desirable.

Jackson is an example of a library that was already modular in structure, consisting of several JARs. Other libraries have made different choices. For example, Google Guava has chosen to bundle all its functionality in a single JAR. Guava aggregates many independently useful parts, ranging from alternative collection implementations to an event bus. However, using it is currently an all-or-nothing choice. The main reason cited by Guava maintainers for not modularizing the library is backward compatibility. Depending on Guava as a whole must be supported in future versions.

Creating an aggregator module that represents the whole library is one way to achieve this with the module system. In “Building a Facade over Modules”, we discussed this pattern in the abstract. For Guava, it might look like this:

module com.google.guava {
  requires transitive com.google.guava.collections;
  requires transitive com.google.guava.eventbus;
  requires transitive com.google.guava.io;
  // .. and so on
}

Each individual Guava module then exports the associated packages for that feature. Guava users can now require com.google.guava and transitively get all Guava modules, just as before. Implied readability ensures they can access all Guava types exported by the individual, smaller modules. Or, they can require only the individual modules necessary for the application. It’s the familiar trade-off between ease of use at development time or a smaller resolved dependency graph leading to a lower footprint at run-time.

When your library consists of a single large JAR, consider splitting it up when modularizing. In many cases, the maintainability of the library increases, while at the same time users can be more specific about what APIs they want to depend on. Through an aggregator module, backward compatibility is offered for those who want to depend on everything in one go.

Especially if different independent parts of your API have different external dependencies, modularizing your library helps users. They can avoid unnecessary dependencies by requiring only the individual part of your API they need. Consequently, they’re not burdened by dependencies thrusted upon them through parts of the API they don’t use.

As a concrete example, recall the java.sql module and its dependency on java.xml (as discussed in “Implied Readability”). The only reason this dependency exists is the SQLXML interface. How many users of the java.sql module are using their database’s XML feature? Probably not that many.

Still, all consumers of java.sql now get java.xml “for free” in their resolved module graph. If java.sql were to be split up in java.sql and java.sql.xml, users would have a choice. The latter module then contains the SQLXML interface, doing a requires transitive java.xml (and java.sql). In that case, java.sql itself doesn’t need to require java.xml anymore. Users interested in the XML features can require java.sql.xml, whereas everyone else can require java.sql (without ending up with java.xml in the module graph).

Because this requires the breaking change of putting SQLXML in its own package (you can’t split a package over multiple modules), this was not an option for the JDK. This pattern is easier to apply to APIs that are already in different packages. If you can pull it off, segregating your modules based on their external dependencies can enormously benefit the users of your library.

Targeting Multiple Java Versions

One way to use new Java APIs in a library implementation without breaking backward compatibility is to use them optionally. Reflection can be used to locate new platform APIs if they are available, much like the scenario described in “Optional Dependencies”. Unfortunately, this leads to brittle and hard-to-maintain code. Also, this approach works only for using new platform APIs. Using new language features in the library implementation is still impossible. For example, using lambdas in a library while maintaining Java 7 compatibility is just not possible. The other alternative, maintaining and releasing multiple versions of the same library targeting different Java versions, is equally unattractive.

mrlib.jar
├── META-INF
│   ├── MANIFEST.MF
│   └── versions
│       └── 9
│           └── mrlib
│               └── Helper.class
└── mrlib
    ├── Helper.class
    └── Main.class

javac --release 7 -d mrlib/7 src/<all top-level sources> 
javac --release 9 -d mrlib/9 src9/mrlib/Helper.java 
jar -cfe mrlib.jar src/META-INF/MANIFEST.MF -C mrlib/7 . 
jar -uf mrlib.jar --release 9 -C mrlib/9 . 

mrlib.jar
├── META-INF
│   └── versions
│       ├── 10
│       │   └── mrlib
│       │       └── Helper.class
│       └── 9
│           └── mrlib
│               └── Helper.class
├── mrlib
│   ├── Helper.class
│   └── Main.class