Javadoc-NG – javadoc rewritten from scratch
The classic javadoc tool has been written in the 90's. Many things have changed since then, but javadoc and the sites it generates are still a bit old-fashioned.
Javadoc-NG is an attempt to create a modern version of javadoc, based on modern technology and libraries as its building blocks. It is intended to be easy to use and to customize and it should integrate seamlessly with other parts of a project's documentation.
tl;dr show me how it looks!
If you want to try it out with your own code or a library of your choice, jump right to the usage section.
What's the problem with the old javadoc?
It's a bit fuzzy, but using a Javadoc site just feels like going back in time a few decades. For example many Javadoc deployments still use the good old frameset like the official JDK8 docs do. Starting with JDK9, the frameset is gone, but it has not been replaced with anything useful. It does have a search feature now, but it is not very forgiving concerning typos, so it's better to know precisely what you're looking for to get useful results. It is also sometimes broken, for example try to click any of the type or member search result in these docs from the Error Prone project and get a 404 error. Many libraries do not have the search feature enabled though, because they are written to be compatible with the still popular Java 8 and hence use the old JDK8's javadoc tool chain for generating their documentation. Often you find yourself clicking into the "All Classes" frame and using the browser's own search feature to locate a class – or head back to Google and use that to find the class you're looking for.
While many things can be criticized in terms of UI and UX, I think one important point is that Javadocs cannot be easily integrated seamlessly into a project's homepage. Many projects use a custom stylesheet to change some colors like the Gradle docs do, but that's usually it. I have only seen a true integration into the navigation of the project site as a whole on the Android documentation. I don't know what they are doing to create that documentation, but I assume Google has created some internal tool for that. Unfortunately whatever they created there is not available for others to use to document their own libraries.
Another problem is that javadoc is not a lightweight tool – it is part of the JDK and relies on a lot of internal APIs. It is difficult to hack the documentation output, add a little extra functionality or remove unwanted features. Best thing you can do is parse the generated HTML and try to manipulate the output on that layer, which does not lead anywhere when making anything but trivial adjustments. When you look at HTTP APIs being documented, they are often integrated right into the content section of a project's website. Nobody does that with Javadocs, although in my opinion it would totally make sense.
Why start from scratch?
While some argue that rewriting something from scratch is something you should never do, there's also people who believe there are not only reasons against, but also in favor of rewriting something from scratch.
The javadoc tool has generally been designed with the idea in mind that others might want to change the output of the tool, which is why it is possible to implement your own Doclet and pass that to the tool. While that interface looks relatively simple at first sight, when trying to change the output of the Javadoc standard doclet, you'll discover that you really need to dig through an internal package with about 99 files and 22565 lines of code that is really cumbersome after 25 years of evolution. For one the project started with the language features from Java 1.0. The language has evolved drastically since then, however javadoc has not been refactored radically to make use of the new features that could help structure it better. Also, there were no libraries available to the developers at that time, to make their life easier except what was in the JDK itself. To name just one example, there was probably no HTML library available so the doclet comes with its own HTML generator engine.
What is the stack of javadoc-ng?
On the backend side, it uses JavaParser, a popular javacc-based parser, for understanding Java source files and extracting information such as available packages, classes, constructors, methods, fields etc. from it. On top of that it uses an ANTLR grammar to make sense of Javadoc comments including the embedded HTML code and tags such as
Typically, a project or library that is being documented depends on a number of third party libraries and of course the JRE libraries. In order to create proper cross-reference links to library classes and methods, it is necessary to build the namespace of types and members from those dependencies. The asm bytecode processor is used for this task.
It builds on top of jsoup to generate HTML code that can be served using an embedded Jetty or packaged as a WAR file for deployment on any standard application server such as Tomcat or Glassfish. The ability to generate static sites consisting of plain files only, like the classic javadoc tool does, is also a goal, however preference is currently given to an application server based deployment.
On the frontend side, it uses Bootstrap 4 to create an elegant user interface. Reading software documentation on mobile devices might be a corner case, but Bootstrap also provides us with a framework that is well-suited for building a UI that works well on desktop screens of varying sizes.
What do I gain by using javadoc-ng?
- Get a modern, decent layout
- Enable search for projects that don't have that feature enabled / still use an old JDK's javadoc (pre Java 9)
- Enable fuzzy search using ngrams, get results even with lazy typing (for example, search for spitter in Guava and still find Splitter)
- Enable source code view for projects that do not enable that in their docs
- Code highlighting in source code view (Compare classic to javadoc-ng)
- The development has really just begun, so you can shape the features it supports and get your documentation to look the way you want it to. We're looking for any open source project interested in this and we're happy to let your needs guide the next steps of development.
How does it work?
First download javadoc-ng-0.0.1.jar.
You'll need Java 9 or later to execute the Jar file, I have tested with Java 11.
Running a web server locally from the CLI
Assuming you have downloaded a Guava source jar to
/tmp/guava.jar, then you can run:
java -jar javadoc-ng-0.0.1.jar run-server \ --input /tmp/guava.jar --title "Guava 29.0"
After parsing the source files contained in the Jar file, this output should appear:
Please visit http://localhost:8080
You can then visit http://localhost:8080 in your browser to browse the documentation.
If you have a project on your disk just point the tool to a directory instead of a jar file and it will pick up all
*.java files recursively and build the documentation for that:
java -jar javadoc-ng-0.0.1.jar run-server \ --input /tmp/guava --title "Guava 29.0"
Even better, you can just pass a Maven coordinate to the tool and let it fetch the Jar artifact for you:
java -jar javadoc-ng-0.0.1.jar run-server \ --coordinates com.google.guava:guava:29.0-jre \ --title "Guava 29.0"
You can pass a
--port argument to start the server on a different port:
java -jar javadoc-ng-0.0.1.jar run-server \ --input /tmp/guava --title "Guava 29.0" \ --port 9000
Building a WAR for deployment on an application server
You can create a WAR archive for deployment on an application server like this:
java -jar javadoc-ng-0.0.1.jar create-war \ --input /tmp/guava.jar --title "Guava 29.0" \ --output /tmp/guava.war
You can deploy that to your application server or just download the jetty-runner and use it to run the application like this:
java -jar /home/z/jetty/jetty-runner-9.4.31.v20200723.jar \ /tmp/guava.war
Jetty also accepts a
--port argument, so you can start it on a different port, too:
java -jar /home/z/jetty/jetty-runner-9.4.31.v20200723.jar \ --port 9000 /tmp/guava.war
As with the
run-server task, you can specify a directory or a Maven coordinate instead of jar file as input:
java -jar javadoc-ng-0.0.1.jar create-war \ --coordinates com.google.guava:guava:29.0-jre \ --title "Guava 29.0"
Additional configuration options
run-server and the
create-war tasks support a number of additional arguments. Run either command with a
--help option to see a list of options and their explanation.
Here's a list of options:
--title: the title to display in the top left corner of the navigation bar
--color: e.g. '#b93221', a color to use for the header and footer
--dark: specify this option if you're using a dark color above. It will turn the color of the font used in header and footer from black to white
--classic: a base URL to use for the
Classiclink available on types to link back to the original documentation. E.g. use 'https://guava.dev/releases/29.0-jre/api/docs/' for Guava.
--external: use this (multiple times) to specify additional JAR files to use for resolving types and members of dependencies of your project or library