Modern Java/JVM Build Practices

Modern Java/JVM Build Practices

Modern Java/JVM Build Practices is an article on modern Java/JVM projects with sample builds for Gradle and Maven, and a focus on hygiene and best build practices. Shift problems left: Find issues earlier in your development cycle, before they happen in production.

Try out the Gradle and Maven builds:

$ ./gradlew build
# Output ommitted
$ ./mvnw verify
# Output omitted

If you have comments or improvements, or find issues, please submit a GitHub issue in this repo .

NB — This is a living document, updated to stay fresh.

Contributing

Please file issues, or contribute PRs! I'd love a conversation with you.

TOC

• Introduction
• You and your project
• Getting your project started
• The JDK
• Use Gradle or Maven
• Setup your CI
• Keep local consistent with CI
• Keep build current
• Generate code
• Use linting
• Use static code analysis
• Shift security left
• Leverage unit testing and coverage
• Use mutation testing
• Use integration testing
• Going further

Introduction

Hi! I want you to have awesome builds 🟢 . If you're on a Java or a JVM project (Kotlin, Scala, Clojure, JRuby, et al), this article is for you.

My purpose is to highlight and provide guidance for building modern Java/JVM projects with Gradle or Maven. This article focuses on Java, but most points apply to any JVM language build (linting is an example exception).

This project has these goals:

• Starter build scripts for Modern Java/JVM builds in Gradle and Maven, helpful for new or existing projects
• Quick solutions for raising project quality and security in your build
• Shift problems to the left ("to the left" meaning earlier in the development cycle). You'll get feedback earlier while still having a fast local build. Time spent fixing issues locally is better than waiting on CI to fail, or worse, for production to fail
• The article focus is on Gradle and Maven: these are the most used build tools for Java/JVM projects

I want to help with the question: I am at Day 1 on my project ("day 0" for those pedants like myself ☺ ): How do I start with a build that supports real coding? And if I'm at "Day X" (X > 1), how do I improve my current build?

For Java/JVM projects, use Gradle or Maven. The article doesn't cover alternative build tools: solid data shows Gradle or Maven are the build tools for most folks. Unless you find yourself in a complex monorepo culture (Google, etc), or there are mandates from above, you need to select one of Gradle or Maven. However, for projects not using Gradle or Maven, you will still find improvements for your build herein (though details will differ).

This article aids you in spinning up a new Gradle or Maven project immediately for a Java/JVM project, or aiding you in improving your existing build.

For new projects, you may find Spring Initializr, mn from Micronaut, or JHipster, among many other project starters, more to your liking: they provide you with starter Gradle or Maven scripts specific for those frameworks. That's great! This article should still help you improve your build beyond "getting started". You should pick and choose build features to add to your starter project, whatever makes sense for your project.

The top goal of this article: Make people awesome (that means you). This project is based on lots of experience and experiments with Gradle and Maven builds, and shares with you lessons learned.

You and your project

There are simple ways to make your project great. Some goals to strive for:

• Visitors and new developers get off to a quick start, and can understand what the build does (if they are interested)
• Users of your project trust it—the build does what it says on the tin—, and they feel safe relying on your project
• You don't get peppered with questions that are answered "in the source" —because not everyone wants to read the source, and you'd rather be coding than answering questions ☺
• Coding should feel easy. You solve real problems, and do not spend overmuch much time on build details: your build supports you
• Your code passes "smell tests": no simple complaints, and you are proud of what others see. Hey! You're a professional, and it shows. (This is one of my personal fears as a programmer)

Hopefully this article and the sample build scripts help you!

Getting your project started

To get a project off to a good start, consider these items. Even for existing projects, you an address these as you go along or while refurbishing an existing project:

• Team agreement comes first. Make sure everyone is onboard and clear on what build standards are, and understands—at least as an outline—what the build does for them
• Provide a good README.md. This saves you a ton of time in the long run. This is your most important step. A good resource is Yegor's Elegant READMEs
  • Intelligent laziness is a virtue. Time invested in good documentation pays off
  • A good README.md answers visitors questions, so you don't spend time answering trivial questions, and explains/justifies your project to others. Fight Conway's Law with communication!
• Pick Gradle or Maven, and use only one. This project provides both to demonstrate equivalent builds for each. See Use Gradle or Maven for more discussion
• Use build wrappers committed into the project root. These run Gradle or Maven, and coders should always invoke ./gradlew or ./mvnw (use shell aliases if these grow tiresome to type)
  • Build wrappers are shell scripts to run Gradle or Maven. The wrapper takes care of downloading needed tools without getting in the way. New contributors and developers can start right away; they do not need to install more software
  • For Gradle, use ./gradlew (part of Gradle)
  • For Maven, use ./mvnw (in progress with Apache to bundle as part of Maven)
• Always run CI on push to a shared repository. It's a sad panda when someone is excited about their commit, and then the commit breaks the other developers
  • In CI, use caches for dependency downloads; this speeds up the feedback cycle from CI (see below)
  • When sensible, move code quality and security checks into local builds before changes hit CI (see below)
• Pick a common code style, and stay consistent; update tooling to complain on style violations
  • The team should agree on a common code style, eg, SUN, Google, et al
  • See Use linting

Tips

• Consider using client-side Git hooks for pre-push to run a full, clean, local build. This helps ensure "oopsies" from going to CI where they impact everyone. The options are broad. Try web searches on:

  • "gradle install git hooks"
  • "maven install git hooks"

  This article presently has no specific recommendations on choices of plugin or approach

TODOs

• Fill out an article section on Git setup, and automated hooks

The JDK

For any Java/JVM project, the first decision is which version of Java (the JDK) to use? Some guidelines:

• Update from Java 8 and older: These are no longer supported unless one buys paid support contracts from Oracle
• Java 11 is the current LTS ("long-term support") version

In this project, you'll see the choice of Java 11 as this is the version to recommend in production.

In general, you will find that AdoptOpenJDK is a go-to choice for obtaining the JDK.

Managing your Java environment

One of the best tools for managing your Java environment in projects is jEnv. It supports both "global" (meaning you, the user) and "project" choices (particular to a directory and its children) in which JDK installation to use. You may notice the .java-version file: this is a per-project file for jEnv to pick the project Java version.

For those on Windows, you may need to use WSL2 to use jEnv.

There are many ways to install the JDK, most are platform-dependent. In general, your team will be better off using a "managed" approach, rather than with each person using binary installers. Popular choices include:

• Apt and friends for Linux or WSL
• Homebrew for Mac
• SDKMAN for multiple platforms

Use Gradle or Maven

The choice between Gradle and Maven depends on your team, your broader ecosystem, and your project needs. In summary:

• Gradle — written in Goovy or Kotlin; dynamic, imperative, and mutable; requires debugging your build on occasion, but less verbose than Maven's XML. Use of "parent Gradle" projects is possible but challenging. You can locally extend your build script either inline with build code, with project plugins, or with plugins from a separate project (perhaps shared across project for your team). If interested in custom plugins, read more here
• Maven — written in XML; declarative and immutable; verbose but specific; it either works or not. Use of "parent Maven" projects is simple with built-in support. You can locally extend your build with plugins from a separate project (perhaps shared across project for your team). If interested in custom plugins, read more here

This article offers no preference between Gradle or Maven. You need to decide.

Projects using Ant should migrate. It is true that Ant is well-maintained (the latest version dates from September 2020). However, you will spend much effort in providing modern build tooling, and effort in migrating is repaid by much smaller work in integrating modern tools. Data point: consider the number of Stackoverflow posts providing Gradle or Maven answers to those for Ant. Consider Ant builds no longer well-supported, and a form of Tech Debt.

Throughout when covering both Gradle and Maven, Gradle will be discussed first, then Maven. This is no expressing a preference! It is neutral alphabetical order.

Tips

• The sample Gradle and Maven build scripts often specify specific versions of the tooling, separate from the plugin versions. This is intentional. You should be able to update the latest tool version even when the plugin has not yet caught up
• Gradle uses advanced terminal control, so you cannot always see what is happening. To view Gradle steps plainly when debugging your build, use:

  $ ./gradlew <your tasks> | cat
  

  or save the output to a file:

  $ ./gradlew <your tasks> | tee -o some-file
  

• Maven colorizes output, but does not use terminal control to overwrite output

Setup your CI

Your CI is your "source of truth" for successful builds. Your goal: Everyone trusts a "green" CI build is solid.

When using GitHub, a simple starting point is ci.yml. (GitLabs is similar, but as this project is hosted in GitHub, there is not a simple means to demonstrate CI at GitLabs). This sample GitHub workflow builds with Gradle, and then with Maven.

If you use GitLab, read about the equivalent in GitLab CI/CD, or for Jenkins in Pipeline.

Keep local consistent with CI

Setup local CI

Batect is a cool tool from Charles Korn. With some setup, it runs your build in a "CI-like" local environment via Docker. This is one of your first lines of defence against "it runs on my box".

You would not run Batect in your CI pipeline itself: use GitHub Actions (or GitLab equivalent). Batect is for your local build.

See batect.yml to configure. For this project, there are demonstration targets:

$ ./batect build-gradle
# output ommitted
$ ./batect build-maven
# output ommitted

Tips

• A simple way in CI to disable ASCII control sequences from colorizing or Gradle's overwriting of lines (the control sequences can make for hard-to-read CI logs) is to use the environment setting:

  TERM=dumb
  

  For example, with Gradle, this will log all the build steps without attempting to overwrite earlier steps with later ones
• If you encounter issues with Gradle and Batect, try stopping the local Gradle daemons before running Batect:

  $ ./gradlew --stop
  $ ./batect <your Batect arguments>
  

TODOs

• Improve Gradle Docker mounting. batect.yml mounts all of ~/.gradle. This is confusing for PIDs of daemons, etc, and other mounted elements which are particular to your local computer: Docker essentially runs as a separate box from our local computer (a separate Linux kernel instance), and Gradle does not always cope with this situation well

Keep build current

An important part of build hygiene is keeping your build system, plugins, and dependencies up to date. This might be simply to address bug fixes (including bugs you weren't aware of), or might be mission-critical security fixes. The best policy is: Stay current. Others will have found—reported problems—, and 3^rd-parties will have addressed these. Leverage the power of Linus' Law ("given enough eyeballs, all bugs are shallow").

Keep plugins and dependencies up-to-date

• Gradle
• Maven
• Team agreement about releases only, or if non-release plugins and dependencies are acceptable

Example use which shows outdated plugins and dependencies, but does not modify any project files:

$ ./gradlew dependencyUpdates
# output ommitted
$ ./mvnw versions:display-property-updates
# output ommitted

In this project, version numbers for Gradle are kept in gradle.properties, and for Maven in the POM.

More on Gradle version numbers

Your simplest approach to Gradle is to keep everything in build.gradle.
Even this unfortunately still requires a settings.gradle to define the project artifact name, and leaves duplicate version numbers for related dependencies scattered through build.gradle.

Another approach is to rely on a Gradle plugin such as that from Spring Boot to manage dependencies for you. This unfortunately does not help with plugins at all, nor with dependencies that Spring Boot does not know about.

This project uses a 3-file solution for Gradle versioning:

• gradle.properties is the sole source of truth for version numbers, both plugins and dependencies
• settings.gradle configures plugin versions using the properties
• build.gradle uses plugins without needing version numbers, and dependencies refer to their property versions

So to adjust a version, edit gradle.properties. To see this approach in action for dependencies, try:

$ grep junitVersion gradle.properties setttings.gradle build.gradle
gradle.properties:junitVersion=5.7.0
build.gradle:    testImplementation "org.junit.jupiter:junit-jupiter:$junitVersion"
build.gradle:    testImplementation "org.junit.jupiter:junit-jupiter-params:$junitVersion"

To update Gradle:

$ $EDITOR gradle.properties  # Change gradleWrapperVersion property
$ ./gradlew wrapper  # Update
$ ./gradlew wrapper  # Confirm

With Gradle, there is no "right" solution for hygienic versioning.

Tips

• See the bottom of build.gradle for an example of customizing "new" versions reported by the Gradle dependencyUpdates task
• The equivalent Maven approach for controlling the definition of "new" is to use Version number rules
• With the Gradle plugin, you can program your build to fail if dependencies are outdated. Read at Configuration option to fail build if stuff is out of date for details

Generate code

When sensible, prefer to generate rather than write code. Here's why:

• Intelligent laziness is a virtue
• Tools always work, unless they have bugs, and you can fix bugs.
  Programmers make typos, and fixing typos is a challenge when not obvious. Worse are thinkos; code generation does not "think", so is immune to this problem
• Generated code does need code review, only the source input for generation needs review, and this is usually shorter and easier to understand.
  Your hand-written code needs review
• Generated code is usually ignored by tooling such as linting or code coverage (and there are simple workarounds when this is not the case). Your hand-written code needs tooling to shift problems left

Note that many features for which in Java one would use code generation (eg, Lombok's @Getter or @ToString), can be built-in language features in other languages such as Kotlin (eg, properties or data classes).

Lombok

Lombok is by far the most popular tool in Java for code generation. Lombok is an annotation processor, that is, a library ( jar) which cooperates with the Java compiler. (An introductory guide to annotations and annotation processors is a good article if you'd like to read more on how annotation processing works.)

Lombok covers many common use cases, and does not have runtime dependencies, and there are plugins for popular IDEs to understand Lombok's code generation, and tooling integration such as provided by JaCoCo code coverage (see below).

Leverage Lombok to tweak code coverage

Be sparing in disabling code coverage! JaCoCo knows about Lombok's @Generated, and will ignore annotated code.

A typical use is for the main() method in a framework such as Spring Boot or Micronaut. For a command-line program, you will want to test your main().

Lombok configuration

Configure Lombok in src/lombok.config rather than the project root or a separate config directory. At a minimum:

config.stopBubbling = true
lombok.addLombokGeneratedAnnotation = true
lombok.anyConstructor.addConstructorProperties = true
lombok.extern.findbugs.addSuppressFBWarnings = true

Lines:

1. stopBubbling tells Lombok that there are no more configuration files higher in the directory tree
2. addLombokGeneratedAnnotation helps JaCoCo ignore code generated by Lombok
3. addConstructorProperties helps JSON/XML frameworks such as Jackson (this may not be relevant for your project, but is generally harmless, so the benefit comes for free)
4. addSuppressFBWarnings helps SpotBugs ignore code generated by Lombok

Use linting

"Linting" is static code analysis with an eye towards style and dodgy code constructs. The term derives from early UNIX.

Linting for modern languages is simple: the compiler complains on your behalf. This is the case, for example, Golang. Having common team agreements on style and formatting is a boon for avoiding bikeshedding, and aids in:

• Reading a code base, relying on a similar style throughout
• Code reviews, focusing on substantive over superficial changes
• Merging code, avoiding trivial or irrelevant conflicts

Code style and formatting are entirely a matter of team discussion and agreement. In Java, there is no recommended style, and javac is good at parsing almost anything thrown at it. However, humans reading code are not as well-equipped.

Pick a team style, stick to it, and enforce it with tooling.

With Java, one needs to rely on external tooling for linting. The most popular choice is:

• CheckStyle
  • Gradle plugin
  • Maven plugin

However, unlike built-in solutions, Checkstyle will not auto-format code for you.

The demonstration projects assume checkstyle configuration at config/checkstyle/checkstyle.xml.
This is the default location for Gradle, and configured for Maven in this project.

The Checkstyle configuration used is stock sun_checks.xml with the addition of support for @SuppressWarnings(checkstyle:...).

Tips

• If you use Google Java coding conventions, consider Spotless which can autoformat your code.

Use static code analysis

Important in your build is static code analysis. This is analysis of your source and compiled bytecode which finds known issues ranging among other things:

• Idioms that your team finds poor or hard to read
• Dangerous anti-patterns (eg, missing null checks)
• Insecure code (see Shift security left)
• Outdated code use (eg, Java 5 patterns better expressed with Java 11 improvements)

The demonstration builds use these to help you:

• PMD
• SpotBugs

TODOs

• CPD for Gradle -- see https://github.com/aaschmid/gradle-cpd-plugin. CPD works for Maven

Shift security left

• Find Security Bugs — a plugin for SpotBugs
• DependencyCheck

Use checksums and signatures: verify what your project downloads! When you publish for consumption by others: provide MD5 (checksum) files in your upload! Be a good netizen.

• For Gradle, read more at Verifying dependencies
• For Maven, always run with the --strict-checksums (or -C) flag. See Maven Artifact Checksums - What? for more information. This is easy to forget about at the local command line. An alias helps:

  $ alias mvnw=`./mvnw --strict-checksums`
  

  However, for CI, this is easy! The Batect configuration on this project says:

  build-maven:
    description: Build and test with Maven
    run:
      container: build-env
      command: ./mvnw --strict-checksums clean verify

  and the GitHub action says:

  - name: Build and test with Maven
    run: ./mvnw --strict-checksums verify

  (Batect and GitHub Actions are discussed both above.)

Tips

• With GitHub actions, consider adding a tool such as Dependabot
• Unfortunately, the Gradle ecosystem is not a mature as the Maven one in this regard. For example, if you enable checksum verifications in Gradle, many or most of your plugin and dependency downloads fail

TODOs

• How to automate the -C (checksum) flag in Maven? See Maven Artifact Checksums - What?

Leverage unit testing and coverage

• JaCoCo
• "Ratchet" to fail build when coverage drops
• Fluent assertions -- lots of options in this area
  • AssertJ — -- solid choice
  • Built assertions from Junit make is difficult for developers to distinguish "actual" values from "expected" values. This is a limitation from Java as it lacks named parameters

Unit testing and code coverage are foundations for code quality. Your build should help you with these as much as possible.

Plugins:

• For Gradle this is part of the "java" plugin
• For Maven, use the Maven Surefire Plugin

(See suggestion : Ignore the generated code for a Lombok/PITest issue.)

To see the coverage report (on passed or failed coverage), open:

• For Gradle, build/reports/jacoco/test/html/index.html
• For Maven, target/site/jacoco/index.html

Tips

• See discussion on Lombok how to _ sparingly_ leverage the @Generated annotation for marking code that JaCoCo should ignore
• Discuss with your team the concept of a "coverage ratchet". This means, once a baseline coverage percentage is agreed to, the build configuration will only raise this value, not lower it. This is fairly simple to do by periodically examining the JaCoCo report, and raising the build coverage percentage over time to match improvements in the report
• Unfortunately neither Gradle's nor Maven's JaCoCo plugin will fail your build when coverage rises! This would be helpful for supporting the coverage ratchet

Use mutation testing

Unit testing is great for testing your production code. But have you thought about testing your unit tests? What that means is, how are you sure your tests really check what you meant them to? Fortunately, there is an automated way to do just that, no code from you required, only some build configuration.

Mutation testing is a simple concept: Go "break" some production code, and see if any unit tests fail. Production bytecode is changed during the build— for example, an if (x) is changed to if (!x)—, and the unit tests run. With good code coverage, there should now be a failing unit test.

The best option for Java/JVM mutation testing is PITest. It is under active development, does some rather clever things with the production bytecode, and has Gradle and Maven plugins. The main drawback is that PITest is noisy, so there will be more build output than you might expect.

After running a build using PITest, to see the mutation report (on passed or failed mutation coverage), open:

• For Gradle, build/reports/pitest/index.html
• For Maven, target/pit-reports/index.html

Tips

• Without further configuration, PITest defaults to mutating classes using your project group as the package base. Example: Set the project group to "demo" for either Gradle or Maven if your classes are underneath the "demo.*" package namespace, otherwise PITest may complain that there are no classes to mutate, or no unit tests to run

Use integration testing

Here the project says "integration testing". Your team may call it by another name. This means bringing up your application, possibly with fakes, stubs, mocks, spies, dummies, or doubles for external dependencies (databases, other services, etc), and running tests against high-level functionality, but not starting up external dependencies themselves (ie, Docker, or manual comman-line steps).
Think of CI: what are called here "integration tests" are those which do not need your CI to provide other services.

An example is testing STDOUT and STDERR for a command-line application. (If you are in Spring Framework/Boot-land, use controller tests for your REST services.)

Unlike src/main/java and src/test/java, there is no generally agreed convention for where to put integration tests. This project keeps all tests regardless of type in src/test/java for simplicity of presentation, naming integration tests with "*IT.java". A more sophisticated approach may make sense for your project

If you'd like to keep your integration tests in a separate source root from unit tests, consider these plugins:

• For Gradle, use Gradle TestSets Plugin
• For Maven, use the Maven Failsafe Plugin

Caution: This project duplicates ApplicationIT.java and ApplicationTest.java reflecting the split in philosophy between Gradle and Maven for integration tests. Clearly in a production project, you would have only one of these.

Tips

• Failsafe shares the version number with Surefire. This project uses a shared maven-testing-plugins.version property
• Baeldung has a good introduction article on Maven Failsafe
• There are alternatives to the "test pyramid" perspective. Consider swiss cheese if it makes more sense for your project. The build techniques still apply

Going further

Can you do more to improve your build, and shift problems left (before they hit CI)? Of course! Below are some topics to discuss with your team about making them part of the local build.

The Test Pyramid

TODO: Small, right-side, floating image of the test pyramid

What is the "Test Pyramid"? This is an important conceptual framework for validating your project at multiple levels of interaction. Canonical resources describing the test pyramid include:

• TestPyramid
• The Practical Test Pyramid

As you move your testing "to the left" (helping local builds cover more concerns), you'll want to enhance your build with more testing at different levels of interaction. These are not covered in this article, so research is needed.

NB — What this article calls "integration tests" may have a different name for your team.

Use automated live testing when appropriate

"Live testing" here means spinning up a database or other remote service for local tests, and not using fakes, stubs, mocks, spies, dummies, or doubles . In these tests, your project calls on real external dependencies, albeit dependencies spun up locally rather than in production or another environment. These might be call "out of process" tests.

This is a complex topic. Some potentially useful resources to pull into your build:

• Flyway — Version your schema in production, and version your test data
• LocalStack — Local testing for AWS services
• TestContainers — Local Docker for real database instances, or any Docker-provided service

Use contract testing when appropriate

Depending on your program, you may want additional testing specific to circumstances. For example, with REST services and Spring Cloud, consider:

• Consumer Driven Contracts

There are many options in this area. Find the choices which work best for you and your project.

Provide User Journey tests when applicable

Another dimension to consider for local testing: User Journey tests.

• Why test the user journey?