Java
Article
By Nicolai Parlog

Inside Java 9 – Version Schema, Multi-Release JARs, and More

By Nicolai Parlog

Two months back I took you for a dive into Java 9, looking at new language features and APIs. But there are lots of things I had to leave out, so here comes a two-parter to fix that. In this first part we’ll be looking at new version strings and command line syntax, multi-release JARs, improved logging, and much more.

JShell

In the first part we skipped Jigsaw because everybody’s talking about it already. In this part we’ll do the same with JShell, the brand-new REPL for Java. (I.e. a thing where you type Java code and it immediately evaluates it.) If you don’t know (much) about it yet but would like to, watch this great introduction Robert Field gave last year at Devoxx Belgium. (JEP 222)

New Version Strings

To ease into this, let’s start with something simple: version strings.

Or so I thought until I tried to understand Java’s version naming schema. It started with JDKs 1.0 and 1.1 – so far so easy but things go downhill from there. Apparently, versions 1.2 to 1.5 were at some point rebranded as Java 2. (Remember J2SE? That’s the 2 in there.) With JDK 1.5 it became clear that this didn’t really work out so Sun started calling it Java 5. Sometime around Java 6 the whole idea of Java 2 was silently buried and ever since things cleared up a little – we simply say “Java X”. (Did you know that all Java version up to and including Java 7 had cool project names like Tiger and Mustang?)

The version strings reported by the JVM were untouched though – they always reported 1.x.... Now, with JEP 223, version strings and the naming schema align. If you check the relevant system properties (see the demo), this is the output you will get:

java.version: 9-ea
java.runtime.version: 9-ea+138-jigsaw-nightly-h5561-20161003
java.vm.version: 9-ea+138-jigsaw-nightly-h5561-20161003
java.specification.version: 9
java.vm.specification.version: 9

This is not overly informative because it’s run on an early access build. In the future java.version will report strings like 9.1.2, which follow the schema $MAJOR.$MINOR.$SECURITY:

  • $MAJOR marks the major version Oracle is planning to release every two to three years.
  • $MINOR marks the smaller releases for bug fixes and other details that follow regularly in between – it resets to zero when a new major version is released.
  • $SECURITY is really interesting – it gets bumped with every release that “contains critical fixes including those necessary to improve security” and is not reset when $MINOR increases.

To make parsing that string unnecessary we get Version, a nice little class that does it for us.

Version version = Runtime.version();
System.out.println("Reported by runtime: " + version);
switch (version.major()) {
    case 9:
        System.out.println("Modularity!");
        break;
    case 10:
        System.out.println("Value Types!");
        break;
}

GNU-style Command Line Options

Java’s tools have a lot of idiosyncrasies when it comes to the syntax of their command line options:

  • some use a single dash for long versions (-classpath), others use two (--gzip)
  • some separate words with dashes (--no-gzip), others don’t (again -classpath)
  • some have single letter forms (-d), others two letter forms (-cp, seriously what’s wrong with that option?!)
  • some assign values with the equals sign (-D<name>=<value>), others require a space (I won’t even…)

On Linux and other GNU-based systems, by contrast, pretty much all tools use the same syntax for their options:

  • two dashes for long versions
  • words are separated by dashes
  • abbreviations use a single dash and consist of a single letter

In a recklessly courageous move Java 9 changes all command line options to match these rules, thus breaking all scripts! Nah, just kidding… 😎 But JEP 293 established a guideline mirroring and adapting those rules and new options are expected to follow it. Old options might at some point be migrated towards the cleaner syntax but that’s not part of Java 9. The JEP contains a lot of details and examples – give it a read.

--ADVERTISEMENT--

Extensions And Updates

Java 9 ships with a lot of JEPs that extend or update existing functionality. Here they are, somewhat ordered by topic, some summarized, some in more detail.

Unicode

While Java itself can be written in UTF-16 (yes, your code can sport emojis), property files used to be limited to ISO-8859-1. Let’s say you have a config.properties file like this one:

money = € / \u20AC

Then accessing this file with Java 8 yields:

money = â▯¬ / €

With JEP 226 those times are finally over and no more Unicode escapes are required. Running the same access code on Java 9 shows what we expected:

money = € / €

(The full example even has a 😎 but our code highlighter doesn’t play well with that.)

Note that there are several ways to access property files and only the one via PropertyResourceBundle got updated. How exactly the encoding is determined and how that can be configured is documented in the API note of the JavaDoc. The defaults are sensible, though, and make the API “just work” for common cases:

try (InputStream propertyFile = new FileInputStream("config.properties")) {
    PropertyResourceBundle properties = new PropertyResourceBundle(propertyFile);
    properties.getKeys().asIterator().forEachRemaining(key -> {
        String value = properties.getString(key);
        System.out.println(key + " = " + value);
    });
} catch (IOException e) {
    e.printStackTrace();
}

You will find this code and a juxtaposition with the Properties API in the demo. If you try to run it on Java 8 for comparison, you will find a nifty little API change Java 9 made for those poor developers who are still using the ancient Enumeration.

In other Unicode-related news, Java 9 supports Unicode 8.0. Yay! (JEP 227, JEP 267)

Graphics

TIFF images are now supported by the Image I/O Framework (in javax.imageio). The Java Advanced Imaging (JAI) project implemented readers and writers for the format, which are now standardized and moved into javax.imageio.plugins.tiff. (JEP 262)

Retina-style HiDPI screens pose unique challenges for desktop UIs. Java already dealt with them on Mac and now follows suit on Linux and Windows. With this “windows and GUI components should have an appropriate size based on the platform recommendations, text should remain crisp despite any default scaling indicated by the HiDPI settings, and icons and images should be smooth and preferably have details appropriate for the pixel density of the display.” (JEP 263)

On Linux the Java desktop triumvirate (AWT, Swing, and JavaFX) can now use GTK 3. For the time being, the JVM will default to GTK 2 and only use GTK3 if indicated by the new system property jdk.gtk.version or “GTK 3 is required for interoperability, and this requirement can be detected sufficiently early”. (JEP 283)

JavaFX used an outdated GStreamer version, which is updated to 1.4.4. This should improve stability and performance of replaying media with JavaFX. (JEP 257)

HarfBuzz is the new OpenType layout engine and displaces ICU, which is discontinued. (JEP 258)

Security

The SHA-3 hash algorithms SHA3-224, SHA3-256, SHA3-384, and SHA3-512 were implemented. They can be used via the MessageDigest API. (JEP 287)

When using SecureRandom (on any Java version) you either get a native implementation based on your operating system’s native entropy source or a pure Java version. The latter “uses an older SHA1-based RNG implementation, which is not as strong as the algorithms used by approved DRBG [Deterministic Random Bit Generator] mechanisms.” Since older but especially embedded systems rely on the Java variant, its security was boosted by implementing the DRBG mechanisms described in NIST 800-90Ar1. Alongside that SecureRandom‘s API was improved to allow passing parameters to DRGB and future algorithms (JEP 273):

Instantiation instantiation = DrbgParameters.instantiation(128, RESEED_ONLY, null);
SecureRandom random = SecureRandom.getInstance("DRBG", instantiation);

byte[] bytes = new byte[20];
random.nextBytes(bytes);

New Java Virtual Machine Features

The machine doing all the work for us – I wonder, will rise up one day? It got some love in the form of a couple of new features, so maybe we calmed it for another few years. If not, I, for one, welcome our new insect machine overlords.

Multi-Release JARs

You might sometimes want to write code that differentiates on which Java version you’re running – do this for Java 8 and to that for Java 9. Up to now this was a little tricky but Java 9 solves an important piece of the puzzle. All relevant parts of the Java platform now create and understand multi-release JARs, JARs containing different versions of the same types for different Java versions.

Let’s see an example:

  1. I created three classes, Main, VersionDependent for Java 8, and VersionDependent for Java 9, where the former prints the result from a call to the latter and the latter simply return “Java X version” where X is either 8 or 9.

  2. Next, I compiled Main and VersionDependent (for Java 8) into a folder out-mr/java8 and VersionDependent (Java 9) into out-mr/java9.

  3. The interesting part is how to package them. The following command creates an mr.jar that contains the VersionDependent.class twice (once from each out-mr/java-x folder) and is structured so that java picks the right classes:

    jar9 --create --file out-mr/mr.jar -C out-mr/java-8 . \
        --release 9 -C out-mr/java-9 .
    
  4. Indeed, running it with java -cp out-mr/mr.jar ...Main returns “Java 8 version” when run with 8 and “Java 9 version” when run with 9.

Here’s what the JAR looks like from the inside:

└ org
    └ codefx ... (moar folders)
        ├ Main.class
        └ VersionDependent.class
└ META-INF
    └ versions
        └ 9
            └ org
                └ codefx ... (moar folders)
                    └ VersionDependent.class

This way Java versions 8 and older will simply use the classes in org but newer versions can overwrite some of them with the content from the right META-INF/versions subfolder. Neat.

Unified Logging

Debugging the JVM, maybe to explain application crashes or to find performance leaks, is complicated enough. Disparate logging options for different subsystems did not make that any easier. Thanks to JEPs 158 and 271 that will soon be a thing of the past. It ships a new command line option -Xlog (shouldn’t that be --log now?) that can be used to define logging down to an extremely detailed level. These are some of the available settings:

  • Each message can have a bunch of tags, depending on the subsystem that created it and what it was doing at the time – some examples are gc, modules, or os. Individual tags can be selected to apply the other settings to them.
  • Of course messages have levels (error, warning, info, debug, trace, develop) and can be filtered by them.
  • The output can be defined either as stdout, stderr, or a file, which can be set up for log file rotation.
  • Then there are decorations – useful information attached to each message (pid, uptime, …, technically tag and level are decorations as well). They can be included in the output.

And all of this can be stuck into a single -Xlog option. Let’s start with simply logging a couple of tags:

java9 -cp out-mr/mr.jar -Xlog:os,modules,gc ...Main

[0.002s][info][os] SafePoint Polling address: 0x00007feea4c96000
[0.002s][info][os] Memory Serialize Page address: 0x00007feea4c94000
[0.002s][info][os] HotSpot is running with glibc 2.22, NPTL 2.22
[0.009s][info][gc] Using G1
Java 9 version

Mmh, is there really nothing for modules? Let’s turn that tag to debug (if nothing is specified the level defaults to info):

java9 -cp out-mr/mr.jar -Xlog:os,modules=debug,gc org.codefx.demo.java9.internal.multi_release.Main

[0.002s][info][os] SafePoint Polling address: 0x00007f3054a22000
[0.002s][info][os] Memory Serialize Page address: 0x00007f3054a20000
[0.002s][info][os] HotSpot is running with glibc 2.22, NPTL 2.22
[0.009s][info][gc] Using G1
[0.059s][debug][modules] set_bootloader_unnamed_module(): recording unnamed module for boot loader
[0.063s][debug][modules] define_javabase_module(): Definition of module: java.base, version: 9-ea, location: jrt:/java.base, package #: 159
[... snip ... many, many more module messages ... ]

Too much. But in there we can see this one:

[0.079s][info][modules,startuptime] Phase2 initialization, 0.0366552 secs

Hey, that’s info! Why did it not show up before?! Because it has two tags and, surprisingly, it does not suffice if tags defined on the command line match one of a message’s tags – they must match all of them. We can extend our matcher to modules+startuptime or use wildcards:

java9 -cp out-mr/mr.jar -Xlog:os,modules*,gc* ...Main

[0.002s][info][os] SafePoint Polling address: 0x00007f9c7f307000
[0.002s][info][os] Memory Serialize Page address: 0x00007f9c7f305000
[0.003s][info][os] HotSpot is running with glibc 2.22, NPTL 2.22
[0.007s][info][gc,heap] Heap region size: 1M
[0.009s][info][gc     ] Using G1
[0.009s][info][gc,heap,coops] Heap address: 0x00000006c6200000, size: 3998 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
[0.077s][info][modules,startuptime] Phase2 initialization, 0.0367418 secs
Java 9 version
[0.090s][info][gc,heap,exit       ] Heap
[0.090s][info][gc,heap,exit       ]  garbage-first heap   total 256000K, used 2048K [0x00000006c6200000, 0x00000006c63007d0, 0x00000007c0000000)
[0.090s][info][gc,heap,exit       ]   region size 1024K, 3 young (3072K), 0 survivors (0K)
[0.090s][info][gc,heap,exit       ]  Metaspace       used 4225K, capacity 4532K, committed 4864K, reserved 1056768K
[0.090s][info][gc,heap,exit       ]   class space    used 414K, capacity 428K, committed 512K, reserved 1048576K

Look, even the garbage collector had something to say – in this case regarding exiting and the heap.

That was just the surface – there are many more options to tweak. The JEP does a pretty good job at explaining it all, including examples.

I alluded to the fact that this improvement does not fix everything. Reason being, that the JEPs focused on providing the infrastructure and rerouting some (many? definitely all GC messages) of the existing calls through it but not necessarily all of them. While I couldn’t find any other logging that does not yet use the new mechanism it is likely that it’s out there.

Command Line Flag Validation

Here’s a fun fact: The Java 8 VM does not validate values given to command line flags until they’re actually needed. Unfortunately that might be pretty late in an application’s lifecycle and I can see how the consequent crashes have some potential for frustration. Let’s take this example:

java -cp out-mr/mr.jar -XX:BlockLayoutMinDiamondPercentage=120 ...Main

While I have no idea what BlockLayoutMinDiamondPercentage does (and was too lazy to look it up), it does not really look like 120 is a valid percentage. Java 8 is unperturbed by this and happily executes our JAR running the entire code successfully – apparently the value is not needed in this program run. Or maybe 120 is a valid value after all? Java 9 doesn’t think so, though:

java9 -cp out-mr/mr.jar -XX:BlockLayoutMinDiamondPercentage=120 ...Main

intx BlockLayoutMinDiamondPercentage=120 is outside the allowed range [ 0 ... 100 ]
Improperly specified VM option 'BlockLayoutMinDiamondPercentage=120'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Better… Because thanks to JEP 245 Java 9 has the ability to validate all input flags at launch time. And as can be seen above, it also prints explanatory error messages.

Reserved Stack Areas

If a thread runs out of stack space, it throws a StackOverflowError. In user code this is usually the application’s death sentence, so there is often no reason to bother. But within libraries, particularly in Java’s core, it might make sense to guarantee certain invariants even under failure.

Take the ReentrantLock for example, where a StackOverflowError in the wrong moment can corrupt the lock’s state, so that it can never be unlocked again. If the application decides to catch errors and go on, the lock deadlocks all involved threads. Not exactly great.

Reserving Frames For Critical Sections

JEP 270 added the new annotation @ReservedStackAccess. With it, methods can identify themselves as critical sections, which need to complete if the system is not to end up in a corrupted state. The JVM’s behavior regarding the execution stack was altered accordingly. It reserves some extra stack space and whenever a StackOverflowError occurs it checks whether a @ReservedStackAccess-annotated method is in the stack. If so, that space is made available to it so it can do some extra work. Once it is done, the exception is thrown nonetheless.

This means using the annotation is no magic pixie dust – the exception will be thrown and the thread most likely die just the same way as without it. It only makes sense to use it if a section of code needs to be finished and can guarantee to do so in the space that is reserved for this purpose.

How much space is reserved in the stack? For now it defaults to a single memory page and “experiments have shown that this is sufficient to cover the critical sections of java.util.concurrent locks that have been annotated so far.”

Example

Coding up an example is not trivial because I found it hard to consistently overflow the stack in a controlled manner and still do something useful with the reserved space. In the demo project you will find my best shot at that but I don’t want to walk you through it in this post. Instead, I’ll show some code that consistently overflows the stack in an uncontrolled manner and without doing anything useful with the reserved space.

int depth;

@ReservedStackAccess
private void determineDepthWithReservedStack() {
    determineDepth();
}

private void determineDepth() {
    depth = 0;
    try {
        recurseToDetermineMaxDepth();
    } catch (StackOverflowError err) { }
    System.out.printf("Depth: %d%n", depth);
}

private void recurseToDetermineMaxDepth() {
    depth++;
    recurseToDetermineMaxDepth();
}

Going from bottom to top you see a method infinitely recursing and incrementing the depth field, another method wrapping it, catching the exception and printing the resulting depth and finally a third method requesting access to the reserved stack area. I make no good use of the reserved stack area as I’m simply exhausting it with more recursive calls. This means that calling either determineDepth or determineDepthWithReservedStack will eventually result in a stack overflow that triggers printing the number of calls. But the output should differ and indeed:

DEPTH USING REGULAR STACK
Depth: 21392

DEPTH USING RESERVED STACK
Java HotSpot(TM) 64-Bit Server VM warning: Potentially dangerous stack overflow in ReservedStackAccess annotated method org.codefx.demo.java9.internal.stack.ReservingStackAreas.determineDepthWithReservedStack()V [1]
Depth: 59544

The results are very unstable, though. I assume there are a lot of other mechanisms at work and this demo is extremely crude.

Internals!

In case your mouth waters now, be aware that this is internal API. To convince the compiler to give you access to the annotation, you have to break into the base module with --add-exports java.base/jdk.internal.vm.annotation=<module>, where <module> is either your module’s name (if you’re already doing that) or ALL-UNNAMED. But the JVM will still ignore the annotation unless you turn off the restriction to privileged code with -XX:-RestrictReservedStack.

See how it comes together in compile.sh and run.sh in the demo project.

But Wait, There Is Still More!

Yes, there is still more and this time you don’t have to wait two months for it – I promise. The second part should be done some time next week and will focus on performance gains and compiler improvements. I’m positive that the newsletter will give a sneak peek so if you’re curious subscribe (before Friday evening UTC) or follow us on Medium, either me or SitePoint (or both), where it will show up next week.

If you liked this article, help us spread the word and tell a friend or colleague about it!

More:
Recommended
Sponsors
Get the latest in Java, once a week, for free.