If you've never heard about
Doclet,
this is a JDK
library that enables you to write your own variant of the
javadoc tool
that comes bundled with the JDK.
This tool is for generating HTML pages from class-, field- and method- /** comments */
that were written in Java sources by developers.
Doclet has been reworked for modular Java, and its use was
documented newly.
Mind that some
Java 8 documentation
still refers to the old version (which says a lot about javadoc popularity).
The best time of Doclet was before
annotations
were introduced with Java 1.6 in 2007,
as it was used as replacement for annotations.
The interest in documentation never was great in developer circles,
being called a
"sweet code smell",
and everybody was happy that he/she could believe
in self-explanatory source code
(which I regard to be a myth, but myths are more popular than reality).
Nevertheless we all want autocomplete-tooltips in our IDE,
showing us the documentation of the API that we need to call
(because there is very few self-explanatory code in this world:-).
However, if you are interested in writing a Doclet, here is
a description of how you can implement and launch it in your IDE's debugger.
A Doclet normally is launched by -doclet
and -docletpath
options in command line of the javadoc
tool,
but it's not possible to debug it then, so we need a solution.
Overview
What we need to implement:
-
A static Java launcher of the
javadoc
tool, with
the options to use our doclet.
This class can then be started in "Run" or "Debug" mode of your IDE.
-
The
Doclet
interface.
The resulting class can be used with the
-doclet
option in command line of the javadoc tool.
-
The
ElementVisitor
interface that is accepted by the "elements" returned from
DocletEnvironment.html#getSpecifiedElements()
in the run()
implementation of the Doclet
interface.
In this article you will find just an abstraction of that visitor (called ModelBuilder
),
because it is up to you to decide what your doclet should do.
Launcher and Doclet
I implemented a parameterizable launcher and the Doclet
interface in just one class.
This must be run from another class implementing a runnable main()
method.
You find an example at the end of this chapter.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80 | import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import javax.lang.model.SourceVersion;
import javax.lang.model.element.Element;
import javax.tools.DocumentationTool;
import javax.tools.ToolProvider;
import jdk.javadoc.doclet.Doclet;
import jdk.javadoc.doclet.DocletEnvironment;
import jdk.javadoc.doclet.Reporter;
/**
* A javadoc launcher that implements the Doclet interface.
*/
public class DocletLauncher implements Doclet
{
/**
* Launcher that calls javadoc.
* @param docletClass the doclet class to use, value for "-doclet" option.
* @param DOCLET_CLASSPATH the path where the doclet resides, value for "-docletpath".
* @param SOURCE_BASEPATH the path where the sources reside, prefix for all sourceFiles.
* @param sourceFiles Java sources to process.
*/
public static void startDoclet(
String DOCLET_CLASSPATH,
String SOURCE_BASEPATH,
String[] sourceFiles)
{
// for every file, call javadoc
for (String sourceFile : sourceFiles) {
if (sourceFile.endsWith(".java") == false) {
if (sourceFile.contains("."))
sourceFile = sourceFile.replace('.', '/');
sourceFile = sourceFile+".java";
}
// use a custom stream to get file names and line numbers
final String[] docletArguments = new String[] {
"-doclet", DocletLauncher.class.getName(),
"-docletpath", DOCLET_CLASSPATH,
SOURCE_BASEPATH+"/"+sourceFile
};
final DocumentationTool javaDoc = ToolProvider.getSystemDocumentationTool();
javaDoc.run(null, null, null, docletArguments); // in, out, err
}
}
/** Called by javadoc for source visitation. */
@Override
public boolean run(DocletEnvironment environment) {
for (Element element : environment.getSpecifiedElements()) {
final ModelBuilder modelBuilder = new ModelBuilder(environment.getDocTrees());
element.accept(modelBuilder, null);
}
return true;
}
@Override
public void init(Locale locale, Reporter reporter) {
}
@Override
public String getName() {
return getClass().getSimpleName();
}
@Override
public Set<? extends Option> getSupportedOptions() {
return Collections.emptySet();
}
@Override
public SourceVersion getSupportedSourceVersion() {
return SourceVersion.latest();
}
}
|
The startDoclet()
method on line 28 can be run from any
class main()
with parameters according to your development environment (IDE).
For a simple Eclipse Java project (non-Maven), the DOCLET_CLASSPATH would be bin
,
for an Eclipse Maven module, target/classes
.
The SOURCE_BASEPATH must point to some path where the Java source files to analyze are below.
Mind that you need to add all compile-dependencies to your CLASSPATH when
scanning sources outside the module where your doclet resides,
because javadoc
uses the javac
compiler!
On line 34, all source files given as arguments are looped
and passed to the javadoc tool, one by one.
On line 36, the arguments are converted into file system paths
when given as fully-qualified class names.
Line 42 builds together the arguments to be passed to javadoc.
Every source file is placed under the given SOURCE_BASEPATH.
Line 47 and 48 finally run the javadoc tool.
This will trigger, for every source file in loop, the run() method on line 54.
Mind that we skip from static usage to the Doclet instance here.
The run()
method on line 55 loops all "specified elements" in given source file.
Mind that if you have a public inner interface or class in your source file, that will be
a separate "specified element", although it also will be contained in the main element.
The ModelBuilder
on line 56 is not a JDK class, it is the
ElementVisitor
implementation specific to the doclet we provide here.
You can see its abstraction below.
The DocTrees
object that is passed to it is a utility needed to
retrieve comments, file paths and line numbers of the scanned source.
All other methods do not matter very much, they were not called by javadoc
for a single Java file.
Example for a class that calls the launcher, which you would run in debug-mode from your IDE:
public class StartDoclet
{
public static void main(String[] args) {
final String DOCLET_CLASSPATH = "bin"; // or "target/classes"
final String SOURCE_BASEPATH = "src"; // or "src/main/java"
//final String SOURCE_BASEPATH = "../my_module/src/main/java/";
DocletLauncher.startDoclet(
DOCLET_CLASSPATH,
SOURCE_BASEPATH,
args);
}
}
If you run StartDoclet
from another module than where the source
to scan is, you would pass ../my_module/src/main/java/
as SOURCE_BASEPATH,
in case the source is in a parallel Maven-module called "my_module".
It is important to also take that other project and all its dependencies into the CLASSPATH
when running the StartDoclet
, because javadoc
uses the javac
compiler,
and that would fail when trying to compile some source without having its dependencies.
For Eclipse, you can do that on the "Dependencies" tab in "Run/Debug Configuration".
Alternatively you can place a StartDoclet
class in every module where you intend to scan source.
In that case, each of these modules must refer to the DocletLauncher module,
thus you would revert the dependency.
Element Visitor
Writing the Doclet was easy, here comes the hard part.
It took me some time to find out all the details of the doclet API,
for example it was really hard to get the source's file path, or the line number of some element.
The class below is an abstraction of what you need to implement for Java source visitation.
It translates the quite complex doclet visitation to a simpler, supporting inner class nesting.
All public visitXXX
methods belong to the Doclet
interface,
all protected abstract visitXXX
methods need to be overridden
by a concrete ModelBuilder
that would produce HTML code.
Do not confuse these two method groups.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161 | import java.util.HashSet;
import java.util.List;
import java.util.Set;
import javax.lang.model.element.Element;
import javax.lang.model.element.ElementVisitor;
import javax.lang.model.element.ExecutableElement;
import javax.lang.model.element.PackageElement;
import javax.lang.model.element.TypeElement;
import javax.lang.model.element.TypeParameterElement;
import javax.lang.model.element.VariableElement;
import javax.tools.JavaFileObject;
import com.sun.source.doctree.DocCommentTree;
import com.sun.source.doctree.DocTree;
import com.sun.source.tree.CompilationUnitTree;
import com.sun.source.tree.LineMap;
import com.sun.source.util.DocSourcePositions;
import com.sun.source.util.DocTrees;
import com.sun.source.util.TreePath;
/**
* To be overridden for interpretation of one Java source file.
*/
public abstract class AbstractModelBuilder implements ElementVisitor<Void,Object>
{
protected final DocTrees docTrees;
private final Set<Integer> lineNumbers = new HashSet<>();
public AbstractModelBuilder(DocTrees docTrees) {
this.docTrees = docTrees;
}
// START interface ElementVisitor
@Override
public final Void visitType(TypeElement element, Object parameter) {
if (lineNumbers.isEmpty())
if (visitFile(filePath(element)) == false)
return null;
final int line = fileLine(element);
lineNumbers.add(line); // avoids receiving no-arg constructor created by compiler
final String className = getName(element);
visitClass(className, line, getComment(element), element);
// to achieve nesting of inner classes, explicitly loop enclosed elements
for (Element enclosed : element.getEnclosedElements())
enclosed.accept(this, parameter);
visitEndOfClass(className, element); // terminate current inner class
return null;
}
@Override
public final Void visitVariable(VariableElement element, Object parameter) {
visitFieldOrMethod(element, false);
return null;
}
@Override
public final Void visitExecutable(ExecutableElement element, Object parameter) {
visitFieldOrMethod(element, true);
return null;
}
@Override
public final Void visit(Element e, Object p) {
return null;
}
@Override
public final Void visitPackage(PackageElement e, Object p) {
return null;
}
@Override
public final Void visitTypeParameter(TypeParameterElement e, Object p) {
return null;
}
@Override
public final Void visitUnknown(Element e, Object p) {
return null;
}
// END interface ElementVisitor
// START to be implemented by sub-classes
/**
* Sub-classes can process given source file path. By default this returns true.
* @return true when processing of this file should continue, false to ignore it.
*/
protected boolean visitFile(String filePath) {
return true;
}
/** Sub-classes must process given top-level or inner class. */
protected abstract void visitClass(String name, int line, String comment, TypeElement e);
/** Sub-classes must process given field. */
protected abstract void visitField(String name, int line, String comment, VariableElement e);
/** Sub-classes must process given method. */
protected abstract void visitMethod(String name, int line, String comment, ExecutableElement e);
/** The current class ends now. */
protected abstract void visitEndOfClass(String name, TypeElement e);
// END to be implemented by sub-classes
private final String filePath(TypeElement e) {
final CompilationUnitTree compilationUnit = docTrees.getPath(e).getCompilationUnit();
final JavaFileObject fileObject = compilationUnit.getSourceFile();
return fileObject.getName();
}
private void visitFieldOrMethod(Element e, boolean isMethod) {
final int line = fileLine(e);
// avoid multiple visitations due to explicitly looping enclosed elements
if (lineNumbers.contains(line) == false) {
lineNumbers.add(line);
if (isMethod)
visitMethod(getName(e), line, getComment(e), (ExecutableElement) e);
else
visitField(getName(e), line, getComment(e), (VariableElement) e);
}
}
private String getName(Element e) {
return (e instanceof TypeElement) ? e.toString() : e.getSimpleName().toString();
}
private String getComment(Element e) {
final DocCommentTree docCommentTree = docTrees.getDocCommentTree(e);
if (docCommentTree == null)
return null;
final StringBuilder sb = new StringBuilder();
final List<? extends DocTree> fullBody = docCommentTree.getFullBody();
for (final DocTree commentFragment : fullBody) {
final String text = commentFragment.toString().trim();
if (sb.length() <= 0)
sb.append(text);
else
sb.append(" "+text);
}
final String result = sb.toString();
return result.isBlank() ? null : result;
}
private final int fileLine(Element e) {
final TreePath path = docTrees.getPath(e);
final CompilationUnitTree compilationUnit = docTrees.getPath(e).getCompilationUnit();
final LineMap lineMap = compilationUnit.getLineMap();
final DocSourcePositions spos = docTrees.getSourcePositions();
final long startPos = spos.getStartPosition(compilationUnit, path.getLeaf());
return (int) lineMap.getLineNumber(startPos);
}
}
|
Besides the java.lang.model
imports we also have com.sun.source
now
(don't ask me if this is a "legal" import), needed for DocTrees
helpers.
Implementing ElementVisitor<Void,Object>
on line 25
means that the visit-methods won't return
anything (Void), and their second parameter will be of type Object
(actually this is not used here, it would be the second parameter
you pass to accept()
in DocletLauncher
).
The constructor on line 30 stores the DocTrees
utility, coming from the caller's
DocletEnvironment
.
Line 37 - 84 contain all implementations of the doclet ElementVisitor
interface.
Only three of them do something, because AbstractModelBuilder
just catches classes,
(visitType()
, also all inner classes go here),
fields (visitVariable()
) and methods (visitExecutable()
).
The first visitType()
call also exposes the source file path.
The visitType()
implementation explicitly iterates over
the enclosed elements on line 49. This technique requires to avoid duplicate calls,
because all elements would be visited anyway also without this explicit loop.
By storing line numbers we avoid duplicate calls, and achieve nesting of inner classes.
The visitEndOfClass()
call on line 52 notifies about exiting an inner class.
Both visitVariable()
and visitExecutable()
also need to comply with the line number management.
To do this once-and-only-once, both call the same procedure on line 119, with a flag.
All other methods from line 70 - 84 are not called when scanning a single Java source file.
Line 94 - 108 contain the methods you need to implement when you want to use this class
for your doclet. Maybe you should add data types, and a list of parameters to method visiting.
You can retrieve such informations from the Element
derivations
VariableElement
and ExecutableElement
.
On line 113 the internals start.
It looks like we need to know about "compilation unit" to get the source file path on line 114,
and the line numbers on line 153.
Both make use of the DocTrees
utlitity that was given to constructor.
Line 132 shows how to get the name of a class, field or method.
All can be done with "simple name".
The toString()
method should not carry application semantic, but here it was done so.
(Not the only inconsistency, why was field called "variable", or method "executable"?)
For getting the comment of some element on line 135
you would need a dedicated DocTreeVisitor
that you can pass
to every single DocTree
item in list.
I tried to make it simpler, again with the infamous toString()
call.
Complex comments may not look good, and @tags
are not contained.
Resume
I hope this helps in case you need to write a doclet.
The doclet API was made for experts, not for the masses.
For me it was a frustrating experience, because
I just wanted to achieve some source folding as it can be seen in my
recent articles,
and it took me days to come to an end.
Let's hope that javadoc
will become more popular again in the future, we need it!