State of jigsaw: Description of module file format?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

State of jigsaw: Description of module file format?

Reinier Zwitserloot
I noticed this document:
http://cr.openjdk.java.net/~mr/jigsaw/notes/jigsaw-big-picture-01

Does not cover the module file format. A friend asked if the module format
(insofar as what is planned right now) includes room to put the source (and
possibly, javadoc, license, etc) in the module to ease the ability of tools
like an IDE to offer utility like popping up javadoc, letting you view the
source of a library call, etc.

... and, I can't remember if it does or not.

 --Reinier Zwitserloot
Reply | Threaded
Open this post in threaded view
|

Re: State of jigsaw: Description of module file format?

mark.reinhold
2012/1/23 23:38 -0800, [hidden email]:
> I noticed this document:
> http://cr.openjdk.java.net/~mr/jigsaw/notes/jigsaw-big-picture-01
>
> Does not cover the module file format.

No, not yet.  The Jigsaw project page links to a (slightly out-of-date)
description, here:

  http://cr.openjdk.java.net/~mr/jigsaw/notes/module-file-format/

There is at present no provision for source code, javadoc, or license
text.

- Mark
Reply | Threaded
Open this post in threaded view
|

Re: State of jigsaw: Description of module file format?

Reinier Zwitserloot
Would it be possible to consider adding such a feature?

Three ways to go that come to mind:

A) Create a new section type for 'source', 'javadoc' and for 'license'.
Define the content of license to just be the license text. 'source' and
'javadoc' are analogous to the resources section, with SubSectionFileHeader
describing the files using the same path restrictions as class files, i.e.
the javadoc of my/package/name/MyClass.class is found in the javadoc
section, in the SubSection with path my/package/name/MyClass.html. The
source is in the 'source' section, SubSection my/package/name/MyClass.java.
Existing tooling (incl. netbeans, intelliJ, and eclipse) manage to figure
this out once you provide them 3 separate archive files, one with javadoc,
one with source, and one with class files, using the same model, so
presumably this should be enough.

B) Create a new section type for 'extra data'. Extra data sections have no
subsections, and the data associated with it starts with a
newline-terminated key string, such as "source" or "license", followed by
some data. Presumably for licenses this data will be the raw license text,
and for source files it will be an archive file stream, presumably in zip
format. This method is a little easier on the module description file
format and is easier to expand on later, but it requires 2 very different
processors for handling class data vs. source/javadoc data for tools.

C) Hack the functionality into the 'resources' section, using some accepted
initial path (src/ and api/ for example).



While option A requires some additional work in the jigsaw spec, it does
appear to offer some unique advantages:

* Tooling can use the same code they already use to read class data to read
javadoc/src.

* Any module file format consumer (including the JVM) can stream past the
javadoc/src or even jump right past if the module source is random access
by skipping the entire section; all that needs to be read to do this is the
section header (section type + size).

* No risk of different pseudo-standards as every IDE attempts to define
their own magic prefix path in the resource section for source/javadoc.

* Trivial to write size optimizer that strips sections out that aren't
required to run the module. It's not that much harder to write such a tool
in any of the other scenarios, but if 'src/' and 'api/' files in the
resource section are merely strongly assumed to be irrelevant but never
officially defined as such, this tool would not be entirely safe to use
(there's always a chance the app itself does need these files and picked an
unfortunate path name to store these resources into).



Looking at the buid side of existing module-esque systems, such as ivy and
maven, indicates there's a need for such a feature (for example, to publish
to maven-central, source and javadoc has to be there, and all 3 (production
jar, javadoc zip, and source zip) are as a rule available for any project
there.

>From a more practical standpoint, being able to see the API documentation
when using a library, or jump into the source either to just review how a
third party library's code works or to allow the debugger to jump into a
method call is a rather useful aspect, and all the 3 major IDEs have ways
to associate javadoc/source with a library dependency in order to
accomplish these things. It would be nice if a single module file is all
you need to download and link into your IDE or debugger without the hassle
of having to also ship the source and javadoc separately.

 --Reinier Zwitserloot



On Tue, Jan 24, 2012 at 17:07, <[hidden email]> wrote:

> 2012/1/23 23:38 -0800, [hidden email]:
> > I noticed this document:
> > http://cr.openjdk.java.net/~mr/jigsaw/notes/jigsaw-big-picture-01
> >
> > Does not cover the module file format.
>
> No, not yet.  The Jigsaw project page links to a (slightly out-of-date)
> description, here:
>
>  http://cr.openjdk.java.net/~mr/jigsaw/notes/module-file-format/
>
> There is at present no provision for source code, javadoc, or license
> text.
>
> - Mark
>
Reply | Threaded
Open this post in threaded view
|

Source/Javadoc links (was: Description of module file format?)

Jesse Glick
On 01/27/2012 10:49 AM, Reinier Zwitserloot wrote:
> Trivial to write size optimizer that strips sections out that aren't
> required to run the module.

For example, for deployment to an embedded device, or via JNLP or applet. I do not see any compelling reason to strip out optional sections for SE desktop deployment -
disks are cheap these days. I agree that a well-defined location is critical, and including this information right in the jmod rather than a separate artifact is
desirable for simplicity.

(OSGi defines the OSGI-OPT/ ZIP path prefix which it guarantees can be safely stripped out, but does not define more than that as far as I know.)

> being able to see the API documentation
> when using a library, or jump into the source either to just review how a
> third party library's code works or to allow the debugger to jump into a
> method call is a rather useful aspect

Yes, this is absolutely crucial for developer productivity when using third-party libraries - or even your own libraries that you just happen to be referencing by
well-known version in the repository, rather than via collocation in a source checkout. I work on Maven support in NetBeans and I can say that automatic association of
sources and Javadoc with a dependency (downloading them on demand in that case) is a key feature, one of the primary "pain points" when using something like plain Ant
without a repository concept.

In my experience sources are much more important than Javadoc, so if package size is a concern dropping Javadoc but keeping sources would be reasonable. An IDE can
essentially reconstruct Javadoc from source files during code completion anyway - and formatted appropriately for that tool rather than a web browser. You can always
publish HTML Javadoc on your project site but once users start really writing code this is not likely to be the preferred medium. There are of course some subtleties in
producing on-the-fly documentation:

1. The javadoc tool lets you decide which packages & classes, and which access levels, to document. I think this is superseded by the exports list of the module.

2. The tool lets you enable or disable display of @author and @version and the like. But an IDE (or other source-consuming tool) can just have a personal preference for
displaying these.

3. HTML generation lets you link to other Javadoc sets. Probably superseded by the requires clause of the module: if sources are available for other libraries (or HTML
Javadoc has been associated by some other means), then a tool can trivially link to them itself.

4. Custom doclets are probably inappropriate for general-purpose consumption, especially by a tool which may have its own specialized way of displaying library
documentation to begin with.

5. Custom taglets are a more interesting case, since these would actually be specific to certain projects. How widely used are these? To what extent are these superseded
by @Documented annotations? When they are not trivially replaced with an annotation, does it make sense to declare required taglets in module-info.java, so that they
would both be run automatically by javadoc with special parameters, and interpretable by IDEs which wish to support this feature (running the taglet in a sandbox)? Custom
-tag looks to be superseded by annotations.

6. Some libraries include custom content under doc-files/ subdirs, generally HTML pages and maybe some images. I suppose these could be preserved as is in the source
section of a jmod.

7. Documentation written in a language other than English is uncommon but a consideration. If the only documentation - that written in source files itself - is in another
language, then to reconstruct summary sentences properly you need to know to use the right BreakIterator. (If the primary documentation is in English but translated
documentation is available separately, then probably nothing special needs to happen in the jmod: an IDE user can explicitly associate the translated documentation with
the library.)