Translate

Tuesday, May 16, 2017

What is new in Java 9 (Besides Modules)


Java 9 is about the modularization of the JDK (known as Project Jigsaw). But like previous releases, it is also about lots of other features.

The most important feature of Java 9 is the Java Platform Module System (JPMS). There are other interesting features like improvements to the Process API and new tools likeJSshell. Over the past couple of years, I wasn't paying much attention to the other "smaller" changes until I attended this interesting speech at Devoxx France. Now that JDK 9 has been announced as Feature Complete earlier this year, this post compiles all those features that are interesting enough for a wide range of developers, and provides detail on each of them. Of course, not everything is mentioned below (you can look at the complete feature set here, which is currently frozen). If you think there is some feature that I missed and are worth describing on this list, please leave a comment.

Process API Updates (JEP 102)

Among the API changes is the introduction of the ProcessHandle interface, which makes common operations on native processes much easier.

Retrieve PID of Current Process

Before Java 9, there was no standard solution to get the native ID of the current process. One could use java.lang.management.ManagementFactory as follows:

// Using a combination of JMX and internal classes
java.lang.management.RuntimeMXBean runtime = java.lang.management.ManagementFactory.getRuntimeMXBean();
java.lang.reflect.Field jvm = runtime.getClass().getDeclaredField("jvm");
jvm.setAccessible(true);
sun.management.VMManagement mgmt = (sun.management.VMManagement) jvm.get(runtime);
java.lang.reflect.Method pid_method = mgmt.getClass().getDeclaredMethod("getProcessId");
pid_method.setAccessible(true);

int pid = (Integer) pid_method.invoke(mgmt);

Or rely on parsing a command result:
// Using jps and parsing the result, requires JDK tool on path
Process proc = Runtime.getRuntime().exec(new String[] { "jps", "-l" });

if (proc.waitFor() == 0) {
    InputStream in = proc.getInputStream();
    int available = in.available();
    byte[] outputBytes = new byte[available];

    in.read(outputBytes);
    String output = new String(outputBytes);

    final String[] lines = output.split("\\r?\\n");
    for (String l : lines) {
        if (l.endsWith(TestClass.class.getName())) {
            System.out.println("Your pid is " + l.split(" ")[0]);
        }
    }
}

Starting in Java 9, we can use the pid() method of the current ProcessHandle:
long pid = ProcessHandle.current().pid();


Checking If a Process Is Currently Running

Prior to the introduction of ProcessHandle, checking if a process was alive could be done by running a command such as ps -ef and parsing its output to see if the process is listed. Another workaround relies on the exitValue() method of the Process class, which throws an IllegalThreadStateException if the process has not yet terminated.
Starting Java 9, we can use the isAlive() method:
Process process = ...
boolean isAlive = process.toHandle.isAlive();


// Or given a pid
Optional<ProcessHandle> processHandle = ProcessHandle.of(pid);
boolean isAlive = processHandle.isPresent() && processHandle.get().isAlive();


Retrieving Process Information

Similarly, if we wanted to retrieve basic information about a certain process, there is no built-in solution. One could parse a command result or rely on a third-party library to do it. In Java 9, we can retrieve a ProcessHandle.Info instance, which contains this information:
Optional<ProcessHandle> processHandle = ProcessHandle.of(pid);
if(processHandle.isPresent()) {
    ProcessHandle.Info processInfo = processHandle.get().info();
    System.out.println("Process arguments: " + Arrays.toString(processInfo.arguments().orElse(new String[0])));
    System.out.println("Process executable: " + processInfo.command().orElse(""));
    System.out.println("Process command line: " + processInfo.commandLine().orElse(""));
    System.out.println("Process start time: " + processInfo.startInstant().orElse(null));
    System.out.println("Process total cputime accumulated: " + processInfo.totalCpuDuration().orElse(null));
    System.out.println("Process user: " + processInfo.user().orElse(""));
}

Running Post-Termination Code

Another convenient feature in the new process API is the ability to run code upon process termination. This can be done using the onExit method in either a Process or a ProcessHandle, which returns a CompletableFuture.
Process process = Runtime.getRuntime().exec(command);
process.onExit().thenRun(() -> { System.out.println("Finished!"); });

Getting Children and Parent Processes

A few other methods further make it easier to navigate process trees:
Method name Description
children() Returns a Stream<ProcessHandle> that captures the current direct children.
descendants() Returns a Stream<ProcessHandle> that captures the descendant processes.
allProcesses() A static method that returns a Stream<ProcessHandle> of all processes currently visible.
Note: It is important to keep in mind that the operating system can restrict some of these APIs, and for the above three methods, those processes are created and terminate asynchronously. There is no guarantee that a process in the stream is alive or that no other processes may have been created since the inception of the snapshot.

Milling Project Coin (JEP 213)

Five items in this JEP introduce small changes to the language.

@SafeVarargs Annotation Can Be Applied to Private Methods

This annotation was introduced in Java 7 to allow a programmer to signal to the compiler that a variable arity method performs safe operations on its varargs parameter. A bit of context about the annotation: Due to the type-unsafe nature of mixing generic types with array creation, and because varargs are translated into arrays behind the scenes, the compiler generates a warning for varargs methods that use such generic types, as well as warning in all method invocations where there is a generic array creation.
As an example, the following method causes a compiler warning:
m(new ArrayList<String>());  // WARNING type safety


..


static void m(List<String>... stringLists) { // WARNING type safety
    Object[] array = stringLists;
    List<Integer> tmpList = Arrays.asList(42);
    array[0] = tmpList; // Semantically invalid, but compiles without warnings
    String s = stringLists[0].get(0); // Oh no, ClassCastException at runtime!
}

To make the compiler ignore this (although it would be the wrong thing to do in this case), we can annotate the method with @SafeVarargs.
So what changed in Java 9? Before Java 9, @SafeVarargs could be applied to either static or final methods. In Java 9, the annotation can also be used on private methods.

Allow Effectively Final Variables to Be Used in a Try-With-Resources Statement

Try-with-resources statements require a local variable declaration for the resource:
// Before Java 9
try(FileReader fileReader = new FileReader("input")) {
   ...
}

Now it is possible to use the statement on a resource without declaring a local variable for it, as long as the variable referencing the resource is final or effectively final.
// Java 9
FileReader fileReader = new FileReader("input")
... // code that does not re-assign fileReader
try(fileReader) {
   ...
}


Allow the <> Operator in Anonymous Classes

For reasons related to the compiler's type inference implementation, diamond operators were not allowed when instantiating anonymous classes in Java 7. With this change, as long as the compiler can infer the type argument of the anonymous class, we can use the operator:
Comparator<Person> compareByHeight = new Comparator<>() {
    @Override
    public int compare(Person p1, Person p2) {
        return p2.getHeight() - p1.getHeight();
    }
}


Forbid the Underscore as an Identifier

In Java 8, Java compilers started to issue a warning on using an underscore as an identifier and an error when used in lambdas. With Java 9, an error is thrown in all cases underscores are used:
$ javac -source 9 Lambda.java 
Lambda.java:2: error: as of release 9, '_' is a keyword, and may not be used as an identifier 
    public int a(int _); 
                     ^ 
Lambda.java:4: error: '_' used as an identifier 
        t(_ -> 0); 
          ^ 
  (use of '_' as an identifier is forbidden for lambda parameters) 
2 errors 

Private Methods in Interfaces

When default methods were being added to interfaces in Java 8, private methods were being considered as well. But it was later postponed. Now with Java 9, interfaces can have private methods (static and instance) to allow non-abstract methods to share code:
interface Greeter {
    ...

    default void greetSomeone(String personName, boolean female) {
        System.out.println("Hi " + getTitle(female) + " " + personName);
    }

    default void farewellSomeone(String personName, boolean female) {
        System.out.println("Bye " + getTitle(female) + " " + personName);
    }

    private String getTitle(boolean female) {
        return female ? "Ms" : "Mr";
    }
}

New Doclet API (JEP 221)

The Doclet API, which allowed client applications to customize the output of Javadocs, underwent a re-design as defined in JEP 221. The goal was to make use of the Language Model API introduced in Java 6 as a standard of representing program elements, as well as the DocTree API to represent documentation elements in the source code and get rid of using the old language model that was part of the old Doclet API. The outcome is a migration of the old API from the com.sun.javadoc package to a new jdk.javadoc.doclet package.
Also, the standard doclet that the Javadoc tool uses by default to generate HTML documentation has been adapted to use the new API.

JShell: The Java Shell (JEP 222)

JShell is a REPL (Read-Eval-Print-Loop) tool that allows snippets of code to be run without having to place them in classes. It is similar to what exists in other JVM-based languages such as Groovy or Scala. One of the motivations behind JShell was that "the number one reason schools cite for moving away from Java as a teaching language is that other languages have a REPL and have far lower bars to an initial Hello, world! program". Meh... a bit debatable maybe. But a more convincing rationale would be to facilitate quick prototyping of new code without having to compile and run and without having to open an IDE.
In addition to the command line tool, JShell comes with an API to allow other tools to integrate JShell's functionality.
Some rules such as ending statements with semi-colons and checked exceptions are relaxed. You can even declare variables of some type that you define after declaring the variable. The class path and module path can also be changed during the session, or when starting JShell the first time (using --class-path and --module-path).
|  Welcome to JShell -- Version 9-ea
|  For an introduction type: /help intro

jshell> 1
$1 ==> 1

jshell> System.out.println("Statement without semi-colon")
Statement without semi-colon

jshell> import java.util.regex.*

jshell> boolean match = Pattern.matches("a*b", "aaab")
match ==> true

jshell> import java.io.*

jshell> FileReader fr = new FileReader("input.txt")
fr ==> java.io.FileReader@42e26948

jshell> /vars
|    int $1 = 1
|    boolean match = true
|    FileReader fr = java.io.FileReader@42e26948

jshell> /save session.txt

jshell> /open session.txt
Statement without semi-colon

jshell> /env --class-path  lib/commons-lang3-3.5.jar
|  Setting new options and restoring state.
Statement without semi-colon
Statement without semi-colon


jshell> /env
|     --class-path lib\commons-lang3-3.5.jar


jshell> import org.apache.commons.lang3.StringUtils

jshell> Foo x
|  created variable x, however, it cannot be referenced until class Foo is declared

jshell> class Foo {}
|  created class Foo
|    update replaced variable x, reset to null

jshell> /exit
|  Goodbye

New Versioning Scheme (JEP 223)

Throughout the past 20+ years, the versioning of Java releases was inconsistent and sometimes confusing. The first two major releases were JDK 1.0 and JDK 1.1. From 1.2 till 1.5, the platform was referred to as J2SE (for the Standard Edition). Starting in 1.5, the versioning changed to become Java 5, then Java 6, and so on. However, when you run java -version with an installed Java 8, the output still shows 1.8 instead of 8. The current versioning scheme for releases, introduced after Oracle acquired Sun, goes as follows:
  • For Limited Update Releases (no critical security fixes), release numbers will multiples of 20.
  • For Critical Patch Updates (fixes security vulnerabilities), the release number will be calculated by adding multiples of five to the prior Limited Update and when needed adding one to keep the resulting number odd.

Version Numbers

Starting Java 9, the versioning will be consistent with semantic versioning, and version numbers have the format $MAJOR.$MINOR.$SECURITY(.$otherpart)? where:
  • $MAJOR is the major version number and is incremented when a major version is released that typically changes the platform specification. For JDK 9, this value will be 9.
  • $MINOR is the minor version number, and is incremented for releases that contain bug fixes and enhancements to standard APIs.
  • $SECURITY is the security level, and is incremented for releases that contain critical security fixes. This version is not reset to zero when the minor version number is incremented.
  • $otherpart consists of one or more versions that can be used by JVM providers to indicate a patch with a small number of non-security fixes.

Version Strings

The version string will be the version number with some other information such as early-access release identifier or the build number:
$VNUM(-$PRE)?\+$BUILD(-$OPT)?
$VNUM-$PRE(-$OPT)?
$VNUM(+-$OPT)?
where:
  • $PRE is a pre-release identifier.
  • $BUILD is the build number
  • $OPT is optional information such as the timestamp.
For comparison, the versioning for JDK 9 using both the existing and upcoming schemes is shown in the below table:
                      Existing                New
Release Type    long           short    long           short
------------    --------------------    --------------------
Early Access    1.9.0-ea-b19    9-ea    9-ea+19        9-ea
Major           1.9.0-b100      9       9+100          9
Security #1     1.9.0_5-b20     9u5     9.0.1+20       9.0.1
Security #2     1.9.0_11-b12    9u11    9.0.2+12       9.0.2
Minor #1        1.9.0_20-b62    9u20    9.1.2+62       9.1.2
Security #3     1.9.0_25-b15    9u25    9.1.3+15       9.1.3
Security #4     1.9.0_31-b08    9u31    9.1.4+8        9.1.4
Minor #2        1.9.0_40-b45    9u40    9.2.4+45       9.2.4

The new versioning scheme is fully documented in the Runtime.Version class and version information can be accessed from it:
System.out.println(Runtime.version().toString());  // 9-ea+167
System.out.println(Runtime.version().major());     // 9
System.out.println(Runtime.version().minor());     // 0
System.out.println(Runtime.version().security());  // 0

Javadoc Search and HTML5 (JEPs 224-225)

Have you noticed anything new in the content of Javadoc pages for Java 9 API so far? Look at the top right of the main frame. That's right, there is now a search box that you can use to search for classes, methods, etc. It's hard to believe it took them so long to implement it. The pages are also in HTML5 by default instead of HTML 4.
The searching is done locally, and the things that can be searched are:
  • Modules, packages, types, and members.
  • Text that is marked with the tag @index.

New HotSpot Diagnostic Commands (JEP 228)

The following JVM diagnosis commands were added:
  • print_class_summary: print all loaded classes and their hierarchy
  • print_codegenlist: show the queue of methods to be compiled in C1 and C2 compilers
  • print_utf8pool: print string table
  • datadump_request: signal the JVM to do a data dump request for JVMTI
  • dump_codelist: print all compiled methods in code cache that are alive
  • print_codeblocks: print code cache layout and bounds
  • set_vmflag: set VM flag option using the provided value
Here is a sample test that sends a command to print the string table using the jcmd utility:
C:\Users\manouti>jcmd
6448 sun.tools.jcmd.JCmd
6392 test.ProcessHandleExample

C:\Users\manouti>jcmd ProcessHandleExample VM.stringtable
6392:
StringTable statistics:
Number of buckets       :     60013 =    480104 bytes, avg   8.000
Number of entries       :      2478 =     59472 bytes, avg  24.000
Number of literals      :      2478 =    162896 bytes, avg  65.737
Total footprint         :           =    702472 bytes
Average bucket size     :     0.041
Variance of bucket size :     0.042
Std. dev. of bucket size:     0.204
Maximum bucket size     :         3

Create PKCS12 Keystores by Default (JEP 229)

Starting Java 9, keystores are created using the PKCS12 format instead of JKS because it offers stronger cryptographic algorithms. This change is backward compatible, so applications accessing existing keystores continue to work.

Multi-Release JAR Files (JEP 238)

One of the most interesting features introduced in Java 9 is the multi-release Jar (MRJAR), which allows bundling code targeting multiple Java releases within the same Jar file. By setting Multi-Release: true in the MANIFEST.MF file, the file becomes a multi-release JAR and the Java runtime will pick the appropriate versions of classes depending on the current major version running. The structure of such a file is illustrated as follows:
jar root
  - A.class
  - B.class
  - C.class
  - D.class
  - META-INF
     - versions
        - 9
           - A.class
           - B.class
        - 10
           - A.class
  • On JDKs < 9, only the classes in the root entry are visible to the Java runtime.
  • On a JDK 9, the classes A and B will be loaded from the directory root/META-INF/versions/9, while C and D will be loaded from the base entry.
  • On a JDK 10, class A would be loaded from the directory root/META-INF/versions/10.
A multi-release JAR file allows projects to maintain different versions of their code targeting different Java platforms, while being able to distribute the code as one JAR, with a single version (e.g. Maven artifact version). This relaxes the common restriction of writing backward compatible code, and allows developers to benefit from a new language and API changes incrementally as they add new code.
This feature naturally requires modifications to some APIs used to process JAR files, such as JarFile and URLClassLoader. Also, many JDK tools have been adapted to be aware of the new format, such as java, javac, and jar.
As an example, the jar command can be used to create a multi-release JAR containing two versions of the same class compiled for both Java 8 and Java 9, albeit with a warning telling that the classes are identical:
C:\Users\manouti>jar --create --file MR.jar -C sampleproject-base demo --release 9 -C sampleproject-9 demo
Warning: entry META-INF/versions/9/demo/SampleClass.class contains a class that
is identical to an entry already in the jar

This creates an MRJAR named MR.jar with the following contents:
jar root
  - demo
     - SampleClass.class
  - META-INF
     - versions
        - 9
           - demo
              - SampleClass.class

Let us now create a class called Main that prints the URL of the SampleClass, and add it for the Java 9 version:
package demo;

import java.net.URL;

public class Main {

    public static void main(String[] args) throws Exception {
        URL url = Main.class.getClassLoader().getResource("demo/SampleClass.class");
        System.out.println(url);
    }
}

If we compile this class and re-run the JAR command, we get an error:
C:\Users\manouti>jar --create --file MR.jar -C sampleproject-base demo --release 9 -C sampleproject-9 demoentry: META-INF/versions/9/demo/Main.class, contains a new public class not found in base entries
Warning: entry META-INF/versions/9/demo/Main.java, multiple resources with same name
Warning: entry META-INF/versions/9/demo/SampleClass.class contains a class that
is identical to an entry already in the jar
invalid multi-release jar file MR.jar deleted

It turns out that the jar tool prevents adding public classes to versioned entries if they are not added to the base entries as well. This is done so that the MRJAR exposes the same public API for the different Java versions. Note that at runtime, this rule is not required. It may be only applied by tools like jar. In this particular case, the purpose of Main is to run sample code, so we can simply add a copy in the base entry. If the class were part of a newer implementation that we only need for Java 9, it could be made non-public.
To add Main to the root entry, we first need to compile it to target a pre-Java 9 release. This can be done using the new --release option of javac (see JEP 247 – Compile for Older Platform Versions):
C:\Users\manouti\sampleproject-base\demo>javac --release 8 Main.java
C:\Users\manouti\sampleproject-base\demo>cd ../..
C:\Users\manouti>jar --create --file MR.jar -C sampleproject-base demo --release 9 -C sampleproject-9 demo

Running the Main class shows that the SampleClass gets loaded from the versioned directory:
C:\Users\manouti>java --class-path MR.jar demo.Main
jar:file:/C:/Users/manouti/MR.jar!/META-INF/versions/9/demo/SampleClass.class

Remove the JVM TI hprof Agent (JEP 240)

The hprof JVM native agent was removed. Before Java 9, it could be used to dump the heap or profile the CPU. The reason it was removed is the existence of better alternatives. For example, jmap can do the heap dump, while JVisualVM can be used to profile running applications.
To demonstrate the impact of removing this agent, we can run a Java program with the hprof agent enabled (i.e. using the option -agentlib:hprof) on Java 8 and then on Java 9 (which is added on my system path):
C:\Users\manouti>D:\Dev\Java\jdk1.8.0_121\bin\java.exe -agentlib:hprof test.ProcessHandleExample
Running...
Dumping Java heap ... allocation sites ... done.

C:\Users\manouti>java -agentlib:hprof test.ProcessHandleExample
Error occurred during initialization of VM
Could not find agent library hprof on the library path, with error: Can't find dependent libraries

In all cases, this agent was not an official part of the JDK and it was rarely used by existing applications.

Remove the jhat Tool (JEP 241)

The jhat tool, which could be used to browse a heap dump in a web browser was removed, since better alternatives exist. The tool was marked experimental and subject to removal in previous releases.

Compile for Older Platform Versions (JEP 247)

Before Java 9, we used to apply -source to tell it to use the selected language specification and -target to generate a certain version of bytecode. However, this can still lead to runtime issues as the compiler will link compiled classes to platform APIs of the current version of the JDK (unless you override the boot classpath). In Java 9, these options are replaced with one simple option --release to be able to compile for an older version.
--release is equivalent to -source N -target N -bootclasspath <bootclasspath-from-N>
JDK 9 makes this possible by maintaining some signature data about APIs from old releases, specifically under $JDK_HOME/lib/ct.sym.

G1 as Default Garbage Collector (JEP 248)

Prior to Java 9, the default garbage collector was typically the Parallel GC on server VMs and the Serial GC on client ones. On Java 9, server VMs will use G1 as the default, which was introduced in Java 7. G1 is a parallel and low-pause garbage collector that works especially well for multi-core machines with big heap sizes. For an overview of the G1 collector, see http://www.oracle.com/technetwork/tutorials/tutorials-1876574.html. In addition to this feature, the Concurrent Mark Sweep (CMS) collector was deprecated.

Multi-Resolution Images (JEP 251)

A new interface MultiResolutionImage is added with a base implementation BaseMultiResolutionImage that can encapsulate several image variants with different sizes. This interface can be used to select the best image variant given certain width and height values.

Compact Strings (JEP 254)

An internal optimization is applied to the String class to reduce memory consumption. The idea is that most String objects contain characters that do not need 2 bytes to represent. The change consists of replacing the internal character array with a byte array, plus an extra byte that denotes the encoding of the byte array: either Latin-1 which takes up 1 byte, or UTF-16, which takes up 2 bytes. The String class will determine which encoding based on the content to be stored.
This is expected to reduce the size of heap memory in existing applications that rely heavily on strings, and should also reduce time spent on garbage collection. This change is internal and does not affect the external API of String and its related classes such as StringBuilder or StringBuffer.
To disable compacting strings, the option -XX:-CompactStrings can be passed to VM.

Stack-Walking API (JEP 259)

Prior to Java 9, access to the thread stack frames was limited to an internal class sun.reflect.Reflection. Specifically the method sun.reflect.Reflection::getCallerClass. Some libraries rely on this method, which is deprecated. An alternative standard API is now provided in JDK 9 via the StackWalker class, and is designed to be efficient by allowing lazy access to the stack frames. Some applications may use this API to traverse the execution stack and filter on classes. Two methods are of interest in this class:
  • public <T> T walk(Function<Stream<StackFrame>, T> function); which allow traversing a stream of stack frames for the current thread, starting from the top frame, and applying the given Function on the stream.
  • public Class<?> getCallerClass(); which returns the class that invoked the method that calls this method.
This class is thread-safe, so multiple threads can use the same instance to walk their stacks.
For example, the following prints all stack frames of the current thread:
package test;

import java.lang.StackWalker.StackFrame;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.List;
import java.util.stream.Collectors;

public class StackWalkerExample {

    public static void main(String[] args) throws NoSuchMethodException, SecurityException, IllegalAccessException, IllegalArgumentException, InvocationTargetException {
        Method fooMethod = FooHelper.class.getDeclaredMethod("foo", (Class<?>[])null);
        fooMethod.invoke(null, (Object[]) null);
    }
}

class FooHelper {
    protected static void foo() {
        BarHelper.bar();
    }
}

class BarHelper {
    protected static void bar() {
        List<StackFrame> stack = StackWalker.getInstance()
                .walk((s) -> s.collect(Collectors.toList()));
        for(StackFrame frame : stack) {
            System.out.println(frame.getClassName() + " " + frame.getLineNumber() + " " + frame.getMethodName());
        }
    }
}

Output:
test.BarHelper 26 bar
test.FooHelper 19 foo
test.StackWalkerExample 13 main
The following prints the current caller class. Note that in this case, the StackWalker needs to be created with the option RETAIN_CLASS_REFERENCE, so that Class instances are retained in the StackFrame objects. Otherwise, an exception would occur.
public class StackWalkerExample {

    public static void main(String[] args) {
        FooHelper.foo();
    }

}

class FooHelper {
    protected static void foo() {
        BarHelper.bar();
    }
}

class BarHelper {
    protected static void bar() {
        System.out.println(StackWalker.getInstance(Option.RETAIN_CLASS_REFERENCE).getCallerClass());
    }
}

Output:
class test.FooHelper
A couple of other options allow stack traces to include implementation and/or reflection frames. This may be useful for debugging purposes. For instance, the first example includes some reflection to invoke FooHelper, but the reflection methods were not shown in the output. We can add the SHOW_REFLECT_FRAMES option to the StackWalker instance upon creation so that the frames for the reflective methods are printed as well:
package test;

import java.lang.StackWalker.Option;
import java.lang.StackWalker.StackFrame;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.List;
import java.util.stream.Collectors;

public class StackWalkerExample {

    public static void main(String[] args) throws NoSuchMethodException, SecurityException, IllegalAccessException, IllegalArgumentException, InvocationTargetException {
        Method fooMethod = FooHelper.class.getDeclaredMethod("foo", (Class<?>[])null);
        fooMethod.invoke(null, (Object[]) null);
    }
}

class FooHelper {
    protected static void foo() {
        BarHelper.bar();
    }
}

class BarHelper {
    protected static void bar() {
        List<StackFrame> stack = StackWalker.getInstance(Option.SHOW_REFLECT_FRAMES)       // show reflection methods
                .walk((s) -> s.collect(Collectors.toList()));
        for(StackFrame frame : stack) {
            System.out.println(frame.getClassName() + " " + frame.getLineNumber() + " " + frame.getMethodName());
        }
    }
}

Output:
test.BarHelper 27 bar
test.FooHelper 20 foo
jdk.internal.reflect.NativeMethodAccessorImpl -2 invoke0
jdk.internal.reflect.NativeMethodAccessorImpl 62 invoke
jdk.internal.reflect.DelegatingMethodAccessorImpl 43 invoke
java.lang.reflect.Method 563 invoke
test.StackWalkerExample 14 main
Note that line numbers for some reflection methods may not be available so StackFrame.getLineNumber() may return negative values.

Encapsulate Internal APIS (JEP 260)

Types whose packages start with sun. (and some starting with com.sun.) are internal to the Java platform. The Java development team has long discouraged their use since they are not supported and vary from implementation to another. Now, most of these classes have been encapsulated in modules that don't export them for code outside the JDK. This means that existing code that relies on them will break; however this decision was based on the analysis that these APIs are very rarely, or either they have official replacements that exist prior to JDK 9. Exceptionally, the following internal APIs remained exported and placed in a separate module, and for those have supported replacements in JDK 9 they have been deprecated and may be encapsulated or removed in JDK 10:
  • sun.misc.{Signal,SignalHandler}
  • sun.misc.Unsafe
  • sun.reflect.Reflection::getCallerClass
  • sun.reflect.ReflectionFactory.newConstructorForSerialization
These APIs have been exported to public use because they are heavily used by some libraries, notably Unsafe. They are used to perform intrinsic JVM operations that are otherwise impossible to achieve.

Reactive Stream Flow API (JEP 266)

A standard set of interfaces corresponding to the reactive-streams specification have been introduced nested under the Flow class. These interfaces define a publish-subscribe mechanism that allows asynchronous flow-controlled communication between producers and consumers of data streams.
  1. Flow.Publisher: specifies a producer of data items and has one method subscribe(Flow.Subscriber subscriber) that adds a subscriber to be sent items.
  2. Flow.Subscriber: specifies a consumer of data items, and defines four methods:
    1. onSubscribe(Flow.Subscription subscription): invoked upon subscription of the subscriber by the publisher, and passes an instance of a Flow.Subscription to allow the subscriber to control the flow.
    2. onNext(T item): Invoked when a new item is received by the subscriber.
    3. onComplete(): invoked when all items have been sent by the publisher and no further items are to be received.
    4. onError(Throwable throwable): invoked when an error occurs on this subscription. No subsequent items are received.
  3. Flow.Subscription: represents a subscription and can be used by the subscriber to control the flow of data, and defines two methods:
    1. request(long n): can be invoked by the consumer to demand up to n items to be sent from the publisher. The number may be < n if the publisher finished sending all items.
    2. cancel(): invoked to cancel the subscription so that no more items are received.
  4. Flow.Processor: implements both a Publisher and a Subscriber and is used to transform data items. It can be used as a medium between a publisher and a subscriber.
Using this model of communication, the subscriber is more in control of the flow of data so that the rate of messages can be handled more efficiently. There is one additional utility class SubmissionPublisher that can be used by item generators that want to publish their data. This class implements the Flow.Publisher and AutoCloseable interfaces and can be closed to complete sending its items.

Convenience Factory Methods for Collections (JEP 269)

The interfaces List, Set and Map have been enriched for factory methods for immutable collections:
  1. 12 Overloaded of(...) factory methods for Set and List. One with a varargs parameter.
  2. 11 Overloaded of(...) factory methods for Map that take key and value arguments. Plus one that takes a varargs of Entry objects ofEntries(Entry<? extends K, ? extends V>... entries).
The returned collections are instances of nested types defined under java.util.ImmutableCollections. This class is package-private so it cannot used to check if the collection is immutable. This is left as an implementation detail of your application.

Enhancements to Streams

Apart from collections, new methods were added to java.util.stream.Stream<T>. The first two are normally intended to be used when the stream is ordered:

Stream<T> takeWhile(Predicate<? super T> predicate)

This method returns, for an ordered stream, a stream that consists of the longest prefix of elements that match the give predicate. The returned stream consists of the prefix of elements matching the predicate in that order. If the original stream is unordered, the order of elements is nondeterministic; the implementation is free to return any subset of elements that matches the predicate. However, if it happens that all elements match the predicate then the returned stream will consist of the same sequence of elements of the original (in the same order). If no element matches the predicate, an empty stream is returned.
An example that returns the first 5 integers of an ordered infinite stream:
Stream<Integer> infiniteInts = Stream.iterate(0, i -> i + 1);
infiniteInts.takeWhile(i -> i < 5).forEach(System.out::println);
Output:
0
1
2
3
4
Example with an unordered stream:
Stream<Integer> unorderedInts = Set.of(3, 4, 5, 1, 2, 0).stream();
unorderedInts.takeWhile(i -> i < 5).forEach(System.out::println);
Output:
3
2
1
0
Note that if we run this last example, it may return any other result that excludes 5.

Stream<T> dropWhile(Predicate<? super T> predicate)

This method does the opposite. It returns, for an ordered stream, a stream consisting of the elements remaining after dropping the longest prefix of elements matching the predicate. If the original stream is unordered, the behavior is indeterministic; the implementation is free to drop any subset of elements matching the predicate and return the remaining elements in the stream.
Example using an ordered stream:
Stream<Integer> finiteInts = Stream.iterate(0, i -> i < 10, i -> i + 1);  // iterate takes a seed element, a Predicate, and an UnaryOperator
finiteInts.dropWhile(i -> i < 5).forEach(System.out::println);
Output:
5
6
7
8
9
Example using an ordered stream:
Stream<Integer> unorderedInts = Set.of(3, 4, 5, 1, 2, 0).stream();
unorderedInts.dropWhile(i -> i < 5).forEach(System.out::println);
Output:
5
4

Note that if we run this last example, it may return some other stream that contains 5.

Stream<T> iterate(T seed, Predicate<? super T> hasNext, UnaryOperator<T> next)

This method returns an ordered stream that applies iteratively the predicate starting on an initial value, until the predicate returns a false with the subsequent increment specified by the UnaryOperator. It is conceptually similar to the traditional for loop:
for (T index = seed; hasNext.test(index); index = next.apply(index)) {
    ...
}

Stream<T> ofNullable(T t)

This method returns the given element if not null; otherwise an empty stream.

Enhancements to Optional

Three new methods were added to Optional:

void ifPresentOrElse(Consumer<? super T> action, Runnable emptyAction)

If a value is present, it performs the given action with the value. Otherwise, it performs the given empty-based action.

Optional<T> or(Supplier<? extends Optional<? extends T>> supplier)

If a value is present, it returns an Optional describing the value. Otherwise, it returns an Optional produced by the supplying function.

Stream<T> stream()

If a value is present, it returns a sequential Stream containing only that value. Otherwise, it returns an empty Stream.

Enhanced Deprecation (JEP 277)

The following elements have been added to the @Deprecated annotation to provide more information about the deprecation status:
1. forRemoval() If true, the deprecated API is intended to be removed in a future JDK release.
2. since() which indicated since which release this API has been deprecated.
As part of this JEP, some existing API elements have been planned for deprecation, such as legacy collections (e.g. Vector and Hashtable); some deprecated elements have been marked as for removal such as Thread.stop() and Thread.destroy().
Furthermore, a new utility jdeprscan has been added in the JDK tools to scan a JAR file or a set of classes for usage of deprecated code. Note that it only scans for deprecated code from the standard libraries.

Spin-Wait Hints (JEP 285)

This feature introduces a new method in java.lang.Thread called onSpinWait(), which allows application code to provide a hint to the JVM that it running in a spin-loop, meaning it is busy waiting for some event to occur. The JVM can benefit from this hint to execute some intrinsic code that leverages hardware platform specific instructions. For example, x86 processors can execute a PAUSE instruction to indicate spin-wait, which can improve performance.
Note that the method onSpinWait does nothing. It is simply marked as an intrinsic candidate (using the jdk.internal.HotSpotIntrinsicCandidate annotation) to indicate to the JVM that it can replace it with native code optimized for the target platform. The method is just a hint, so the JVM is free to simply do nothing. A typical scenario where this method may be used is illustrated in the docs:
class EventHandler {
    volatile boolean eventNotificationNotReceived;
    void waitForEventAndHandleIt() {
        while ( eventNotificationNotReceived ) {
            java.lang.Thread.onSpinWait();
        }
        readAndProcessEvent();
    }

    void readAndProcessEvent() {
        // Read event from some source and process it
         . . .
    }
}        

Applet API Deprecated (JEP 289)

Due to the decreasing support of Java plug-ins by web browsers, the Applet API is deprecated. However, there is no intention to remove it in the next major release, so there is no forRemoval = true in the @Deprecated.


Sunday, April 2, 2017

Five ways to reduce technical debt, rework costs in Agile, DevOps

Learn five ways to reduce technical debt and the high cost of rework in Agile and DevOps software development. Five software pros share their best practices.

Leaving some technical debt in the wake speeds software delivery, but the high interest on that debt can mean costly rework and loss of customer satisfaction. Software pros describe five ways to reduce technical debt and the problems it causes.
Technical debt, also known as code debt, is a software programming phenomenon that happens when low-quality or defective code is released in software, or when defects in software are not discovered and fixed quickly. Most often, this occurs in iterative application development when speed of release is valued more than high quality.
Letting defects go unresolved is like not paying off a loan, and the interest builds up, explained Mik Kersten, CEO of Tasktop Technologies, a software lifecycle integration tools vendor in Vancouver, B.C. "You have to pay down that technical debt," he said. Otherwise, a part of the application portfolio is a dead end, and it's going to sink the company with overtime costs needed to do maintenance.
Here are five dos and don'ts for turning the tables on technical debt.
 1. Do define best practices for DevOps and adhere to them.
Automating every application lifecycle management process is as important as using automated development tools when it comes to reducing technical debt, advised Michael Irigoyen, software engineer at Accusoft, based in Tampa, Fla., at DeveloperWeek 2017 in San Francisco. "Automate the end-to-end process," he said.
Unfortunately, tooling gets more attention than craftsmanship in software development today, noted Matt Heusser, principal consultant for Excelon Development. Too many software organizations today are buying tools and just dropping them in the development pipeline, rather than creating foundation processes. So, when they end up with a lot of buggy software, they blame the tool.
Enforcing best practices can stop people from taking process shortcuts -- like cutting corners writing code -- that lead to mistakes, said Christopher Tozzi, analyst for Fixate IO in Livermore, Calif. "Doing things the quick-and-dirty way [can create] quick-and-dirty code snippets, which can add up to big technical debt across a large program."
To improve first-time quality and reduce the regression rate, Heusser said DevOps teams must facilitate collaboration between dev and test, improve communication, and develop better coding and testing skills. This can't be done without developing a component architecture and the ability to deploy different components. "It means coming up with meaningful examples in the requirements with something like the three amigos," he said.
Investing in process improvements can be very valuable. At Accusoft, Irigoyen said, regularly scheduled, two-day hackathons give developers time to work on solutions to their own and their peers' process problems. Among other innovations, these sessions have produced methods and tools that help reduce technical debt, such as a log overview and analyzer, a programmed drone that reacts to build failures and a CSS style reference website for UI developers.
 2. Do automate all nonvalue work.
Reduce technical debt by streamlining activities that add time and cost to development, but do not add value to the product, Tasktop's Kersten said. Examples can include meeting compliance requirements, doing reports on activities and making handoffs of work from one person to another.
Nonvalue work increases as companies adopt Agile and iterative development. "When you go from nine-month release cycles to four-week iterations, your people have to do the same nonvalue work that they would normally do every nine months in four weeks," Kersten said. Automating all those nonvalue activities is critical to reducing technical debt and speeding DevOps processes.
 3. Don't let the flaws in software fester.
"The later in software development that a defect is identified or found, the more expensive it is to resolve or remediate the issue," said Theresa Lanowitz, founder of voke inc., which has researched the cost of technical debt and rework in Agile software development and is based in Minden, Nev.
Over 30 years ago, the cost of fixing bugs in production was found to be 1,000 times the cost if it were found in the requirements phase, according to a study by Barry Boehm and Victor Basili. In 2008, an IBM System Science Institute report estimated the cost of discovering software defects after release is 30 times more than catching them before release.
Every defect creates and carries baggage that increases the cost of repair over time. Heusser noted that fixing something right away is faster than getting a customer report, trying to figure it out, debugging, fixing and retesting.
"The other big problem with deferring fixes is the incremental, death by a thousand cuts that happens as the defect tracker fills up," Heusser said. "When a new bug is reported, people have to check the defect tracker to see if it was identified, but marked 'fix-later.' The open defect list begins to look big."
Tozzi pointed out that fixing flaws in production software can’t be done without attracting unwanted attention. Critical updates have to be released, which takes a long time and requires informing users.  Also, bottlenecks can occur -- for example, not all users will install updates. In a number of ways, a bug that makes it into production can hurt the provider's reputation.
4. Don't rush through the requirements phase of iterative development.
It's a popular myth that gathering the right requirements is easier in iterative development, Lanowitz said. The Agile development methodology has pushed the defining application requirements to later in the lifecycle. That poor requirements elicitation is a primary cause of rework remains true to this day, she said.
"People think that developing stuff in smaller pieces now means there are fewer and more obvious requirements," Lanowitz said. Instead, users are even more demanding of feature releases, as opposed to monolithic app releases. They use the feature right away and can decide if it works well quickly. Automating testing is a must-have way to reduce technical debt.
5. Do automate testing first to reduce technical debt.
Without automated testing, it's impossible to reduce technical debt and rework in Agile and DevOps projects, experts agreed.
"Automate all the traditional types of tests," Lanowitz said. "It eliminates the overall total number of defects, reduces your test cycle and reduces your overall software cycle."
Businesses' survival today is questionable if leaders don't understand test automation's role in reducing technical debt, experts cited above agreed. Tasktop's Kersten concluded: "Those companies that don't start automating now will get disrupted by the more lean startups."

DevOps success calls for quality

A company that allows technical debt to enable speedy software releases creates a jaded DevOps' culture, Heusser said. "New features are rewarded, and there are not clear consequences for bad code, [if] it doesn't slow you down. The net result is the code base continues to get worse, and there is a great deal of baggage added, which slows down forward progress.
Source: Five ways to reduce technical debt, rework costs in Agile, DevOps

Friday, March 31, 2017

Java: Interface vs. Abstract Class


How to characterize concrete classes using abstract classes and interfaces in Java.

  · Java Zone
A class is named a concrete class when it has a name and implements every remaining method that is declared along the class hierarchy. Along with the hierarchy, its supertypes become a more general representation of the acting domain, and finally goes beyond that border and ends with the fact that almost everything is an object. A supertype can be either a concrete class that is apparently not final, an abstract class or even one or more interfaces. Combining a super or an abstract class with some interfaces is also a wide acknowledged way for characterizing a concrete class. Polymorphism is given when more than one class implements a particular method differently based on a class’s nature. In this article, the focus is on two of three kinds of supertypes they distinguish from their essential characteristics. Both an interface as well as an abstract class can be instantiated in a manner of an anonymous class.

Abstract Class

Typically, an abstract class implements common behaviors and member variables across any concrete class, and its method might have already specified an interface. The distinguishable behavior is gained through declaring abstract methods, which need to be implemented in a specific class. The abstract class can inherit features from another concrete or abstract class, and can enrich further behavior while adding interfaces. In Java, such tangible implementations can be explicitly emphasized with the annotation @Override that indicates a deviation of manner (docs.oracle.org, n.d.). The polymorphism stops at that point, where a concrete implementation of a method becomes final. As methods in an abstract class can also be private, it makes such a class most appropriate for encapsulating private methods while breaking down the complexity of shared methods into smaller pieces. (Martin, 2009). An abstract class is ultimately very close to a concrete implementation.

Interface

Since Java 1.8, an interface can implement default methods to provide a general behavior (Panka, 2016). Consequently, both an abstract class and an interface approach each other regarding their features. While member variables of an interface are explicitly public and final, an abstract class does not sanction their members about access modifiers. Moreover, there is a small difference with a significant effect regarding possible access modifiers for a default method in interfaces. A default method is always public and is in contrast to an abstract method that accepts either default, protected, or public as an access modifier. Whatever an interface defines, it can be used anywhere and may enrich an object with specific behaviors. An interface allows only inheritance from another interface. Both an abstract class and an interface can implement static methods. (docs.oracle.com, n.d.). Another type of interfaces are such without any declared method and are fully accepted as a marker interface. They merely indicate the compliance from a developer that, for example, a class is serializable (java.io.Serializable) or cloneable (java.lang.Cloneable) (mrbool.com, n.d.).

Example 1: Interface vs. Abstract Class

This comparison emphasizes the advantage of an abstract class over an interface focused on the calculation of the angle between two straight lines. However, both an interface and an abstract class can be used for that purpose while the first does not go in line with Martin’s suggestions about clean code.
public interface TwoDimensional {
    double PI = 3.141579; //is public and final
    default double getAngle(TwoDimensional a, TwoDimensional b) /* is public */{
        double alpha = 0.0;
        //do complex calculation here
        //modulus
        //scalar
        //to degrees
        return alpha;
    }
    int getX(); //is public
    int getY();
}

public abstract class AbstractTwoDimensional {
    public final double PI = 3.141579;
    private int x;
    private int y;
    public final double getAngle(AbstractTwoDimensional other) {
        double a = calcModulus(x, y);
        double b = calcModulus(other.getX(), other.getY());
        double s = scalar(other.getX(), other.getY());
        return toDegrees(s, a, b);
    }
    private double toDegrees(double s, double a, double b) {
        //compute
        return 0;
    }
    private double calcModulus(int x, int y) {
        //compute...
        return 0;
    }
    private double scalar(int x2, int y2) {
        //compute...
        return 0;
    }
    abstract int getX();
    abstract int getY();
    //some other abstract methods…
}

Example 2: Combination of Interface and Abstract Class

This example is very close to the prior one, but is distinguished by a mix of interface and abstract classes. They both characterize concrete classes.
public interface TwoDimensional {
    Double PI = 3.141579;
    double getAngle(TwoDimensional other);
    int getX();
    int getY();
}

public abstract class AbstractTwoDimensional implements TwoDimensional {    
    private int x;
    private int y;
    @Override
    public final double getAngle(TwoDimensional other) {
        double a = calcModulus(x, y);
        double b = calcModulus(other.getX(), other.getY());
        double s = scalar(other.getX(), other.getY());
        return toDegrees(s, a, b);
    }
    @Override
    public final int getX() {
        return x;
    }
    @Override
    public final int getY() {
        return y;
    }
    protected void setX(int x) {
        this.x = x;
    }
    protected void setY(int y) {
        this.y = y;
    }
    private double toDegrees(double s, double a, double b) {
        //compute
        return 0;
    }
    private double calcModulus(int x, int y) {
        //compute...
        return 0;
    }
    private double scalar(int x2, int y2) {
        //compute...
        return 0;
    }
}

public class MutableLine extends AbstractTwoDimensional {
    //some specific things…
    public MutableLine(ImmutableLine line) {
        //extract data and init
    }
}

public final class ImmutableLine extends AbstractTwoDimensional {
    //some specific things...
    public ImmutableLine(MutableLine line) {
        //extract data
    }
    @Override
    public final void setX(int x) {
        //throw an appropriate exception
    }
    @Override
    public final void setY(int y) {
        //throw an appropriate exception
    }
}

References

docs.oracle.org (n.d.) Anonymous Classes. Oracle. Available from: https://docs.oracle.com/javase/tutorial/java/javaOO/anonymousclasses.html. [9 February 2017].
docs.oracle.org (n.d.) Overriding and Hiding Methods. Oracle. Available from: https://docs.oracle.com/javase/tutorial/java/IandI/override.html. [10 February 2017].
Martin, R. C. (2009) Clean Code: A Handbook of Agile Software Craftsmanship. Prentice Hall. [Kindle]. [11 February 2017].
Panka, J. (2016) Java 8 Interface Changes – static method, default method. JournalDev. Available from: http://www.journaldev.com/2752/java-8-interface-changes-static-method-default-method. [9 February 2017].
docs.oracle.com (n.d.) Lesson: Interfaces and Inheritance. Available from: https://docs.oracle.com/javase/tutorial/java/IandI. [10 February 2017].
mrbool.com (n.d.) What is Marker interface in Java? Available from: http://mrbool.com/what-is-marker-interface-in-java/28557. [11 February 2017].

Source: Java: Interface vs. Abstract Class