Review request: jar tool to transform a plain jar file to a modular jar

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Review request: jar tool to transform a plain jar file to a modular jar

Mandy Chung
I implemented the support in the jar tool to transform a plain JAR file
to a modular JAR file [1]:
    http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/modularize-jar/

Simple example:
To modularize an existing jar file (two ways - see below):
   $ jar uf astro.jar -module org.astro@2.0
   $ jar uf greetings.jar -C modules/com.greetings module-info.class
   $ java -jar greetings.jar   # launch in legacy mode

Installing a modular JAR file in the module library
     $ jmod install greetings.jar astro.jar -L mlib

Run in module mode
     $ java -L mlib -m com.greetings

Details:
The jar tool provides a new option to transform a plain old JAR file
into a modular JAR:
    -module {module-name@version}
       Generate META-INF/module-info.class entry in the JAR file [a]

It is designed to handle simple cases that:
    * requires the default Java platform module [b]
    * exports all public types
    * main entry point as specified in the manifest or in the 'e'
      flag, if any
    * infers the dependencies from 'Class-Path' attribute if it is
      a modular JAR file, and the specific module name of
      that version will be added in the module dependency [c].

The jar tool will determine if the inputfiles include a
"module-info.class" entry that indicates that it is a modular
application.  If exists, it will create a modular JAR by copying
the module-info.class file in the META-INF directory [d].
The "-module" option is not required in this case.

Initially I propose to use the GNU-style option (--module) but
it should really be a separate project to have the existing
command-line tools to support GNU-style options that deserves
its own JEP [2].  I like the option be a long-name option rather
than the traditional single letter option which I don't see such
inconsistency is an issue or find a good letter for it :)

Since the generated module-info.class is typically fairly simple,
it doesn't seem really necessary to output the module-info.java
and so didn't implement that.

Implementation: I use the classfile library that Jon Gibbons wrote
to generate the module-info.class.  Since Jon will be updating
javac and classfile per the module-info spec [3], I modified
ModuleExport_attribute and ModuleRequires_attribute classes in
a simple way but the right change will come when they are updated
per the proposed spec.

Footnotes:
[a] jarfile name tends to be unnamed or contains '-' character.
     The module name will not be inferred from the jarfile name.
[b] Private and internal APIs in the jdk are not exported.
     If the JAR file depends on any of these APIs and running
     in module mode, this will result in a runtime error.
[c] If the path listed in the 'Class-Path' attribute is not a
     modular JAR, it will simply issue a warning and continue.
     Module dependencies may be incomplete in this case.
[d] If an application has been modularized, module-info.java that
     describes the module definition is placed in the root of the
     source tree and compiled in the same destination directory as
     its classes.

BTW, I'm working on adding a new regression test for this new
functionality. It looks like I have to write a better framework
to cover different test cases and do the validation that may take
some time.

Thanks
Mandy

[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2011-April/001251.html
[2] http://openjdk.java.net/jeps
[3] http://openjdk.java.net/projects/jigsaw/doc/topics/modules-classfile.html

Reply | Threaded
Open this post in threaded view
|

Re: Review request: jar tool to transform a plain jar file to a modular jar

Mandy Chung
  The jigsaw team discussed the option name and the location of the
module metadata in the jar file.   We decide to follow the single letter
convention for the option name and will use letter 'I' which is the
closest one to specify a "module ID".

When module-info.class exists in the inputfiles, if the metadata is in
META-INF/module-info.class, there will be two copies of
module-info.class (module-info.class and META-INF/module-info.class) in
the jar file.  It'd be error-prone for tools that manipulates the module
metadata [1].  We decide to keep "module-info.class" as the path for the
module metadata in a modular jar file as it's in the modulepath.

I'll send out a new webrev once I update the implementation.

Mandy
[1]
http://openjdk.java.net/projects/jigsaw/doc/draft-java-module-system-requirements-12#manipulable-module-metadata

On 08/25/11 18:20, Mandy Chung wrote:

> I implemented the support in the jar tool to transform a plain JAR file
> to a modular JAR file [1]:
>    http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/modularize-jar/
>
> Simple example:
> To modularize an existing jar file (two ways - see below):
>   $ jar uf astro.jar -module org.astro@2.0
>   $ jar uf greetings.jar -C modules/com.greetings module-info.class
>   $ java -jar greetings.jar   # launch in legacy mode
>
> Installing a modular JAR file in the module library
>     $ jmod install greetings.jar astro.jar -L mlib
>
> Run in module mode
>     $ java -L mlib -m com.greetings
>
> Details:
> The jar tool provides a new option to transform a plain old JAR file
> into a modular JAR:
>    -module {module-name@version}
>       Generate META-INF/module-info.class entry in the JAR file [a]
>
> It is designed to handle simple cases that:
>    * requires the default Java platform module [b]
>    * exports all public types
>    * main entry point as specified in the manifest or in the 'e'
>      flag, if any
>    * infers the dependencies from 'Class-Path' attribute if it is
>      a modular JAR file, and the specific module name of
>      that version will be added in the module dependency [c].
>
> The jar tool will determine if the inputfiles include a
> "module-info.class" entry that indicates that it is a modular
> application.  If exists, it will create a modular JAR by copying
> the module-info.class file in the META-INF directory [d].
> The "-module" option is not required in this case.
>
> Initially I propose to use the GNU-style option (--module) but
> it should really be a separate project to have the existing
> command-line tools to support GNU-style options that deserves
> its own JEP [2].  I like the option be a long-name option rather
> than the traditional single letter option which I don't see such
> inconsistency is an issue or find a good letter for it :)
>
> Since the generated module-info.class is typically fairly simple,
> it doesn't seem really necessary to output the module-info.java
> and so didn't implement that.
>
> Implementation: I use the classfile library that Jon Gibbons wrote
> to generate the module-info.class.  Since Jon will be updating
> javac and classfile per the module-info spec [3], I modified
> ModuleExport_attribute and ModuleRequires_attribute classes in
> a simple way but the right change will come when they are updated
> per the proposed spec.
>
> Footnotes:
> [a] jarfile name tends to be unnamed or contains '-' character.
>     The module name will not be inferred from the jarfile name.
> [b] Private and internal APIs in the jdk are not exported.
>     If the JAR file depends on any of these APIs and running
>     in module mode, this will result in a runtime error.
> [c] If the path listed in the 'Class-Path' attribute is not a
>     modular JAR, it will simply issue a warning and continue.
>     Module dependencies may be incomplete in this case.
> [d] If an application has been modularized, module-info.java that
>     describes the module definition is placed in the root of the
>     source tree and compiled in the same destination directory as
>     its classes.
>
> BTW, I'm working on adding a new regression test for this new
> functionality. It looks like I have to write a better framework
> to cover different test cases and do the validation that may take
> some time.
>
> Thanks
> Mandy
>
> [1]
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2011-April/001251.html
> [2] http://openjdk.java.net/jeps
> [3]
> http://openjdk.java.net/projects/jigsaw/doc/topics/modules-classfile.html
>

Reply | Threaded
Open this post in threaded view
|

Re: Review request: jar tool to transform a plain jar file to a modular jar

Mandy Chung
  Updated webrev:
     http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/modularize-jar.01/

To modularize an existing jar file:
1. use 'I' flag to specify the module name

   $ jar ufI astro.jar org.astro@2.0

   $ jar tvf astro.jar
       0 Tue Aug 30 16:15:04 PDT 2011 META-INF/
     74 Tue Aug 30 16:15:04 PDT 2011 META-INF/MANIFEST.MF
       0 Tue Aug 30 16:15:00 PDT 2011 org/
       0 Tue Aug 30 16:15:00 PDT 2011 org/astro/
  276 Tue Aug 30 16:15:00 PDT 2011 org/astro/World.class
   172 Tue Aug 30 16:21:26 PDT 2011 module-info.class<---- newly generated

2. add module-info.class to the jar file as an input file

   $ jar uf greetings.jar -C modules/com.greetings module-info.class

Installing a modular JAR file in the module library
     $ jmod install greetings.jar astro.jar -L mlib

Run in module mode
     $ java -L mlib -m com.greetings

Details:
The jar tool provides a new option to transform a plain old JAR file
into a modular JAR:
    -I {module-name@version}
       Generate "module-info.class" entry in the JAR file [a]

It is designed to handle simple cases that:
    * requires the default Java platform module [b]
    * exports all public types
    * main entry point as specified in the manifest or in the 'e'
      flag, if any
    * infers the dependencies from 'Class-Path' attribute if it is
      a modular JAR file, and the specific module name of
      that version will be added in the module dependency [c].

The jar tool will determine if the inputfiles include a
"module-info.class" entry that indicates that it is a modular
application [d].  If exists, 'I' flag cannot be specified.

Implementation: I use the classfile library that Jon Gibbons wrote
to generate the module-info.class.  Since Jon will be updating
javac and classfile per the module-info spec [3], I modified
ModuleExport_attribute and ModuleRequires_attribute classes in
a simple way but the right change will come when they are updated
per the proposed spec.

Footnotes:
[a] jarfile name tends to be unnamed or contains '-' character.
     The module name will not be inferred from the jarfile name.
[b] Private and internal APIs in the jdk are not exported.
     If the JAR file depends on any of these APIs and running
     in module mode, this will result in a runtime error.
[c] If the path listed in the 'Class-Path' attribute is not a
     modular JAR, it will simply issue a warning and continue.
     Module dependencies may be incomplete in this case.
[d] If an application has been modularized, module-info.java that
     describes the module definition is placed in the root of the
     source tree and compiled in the same destination directory as
     its classes.

Mandy
[1]
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2011-August/001452.html
Reply | Threaded
Open this post in threaded view
|

Re: Review request: jar tool to transform a plain jar file to a modular jar

Xueming Shen
Comments on the sun.tools.jar.Main

(1) #197-#201
      Shouldn't the "equals" be the "ignore case equals"?

(2) #202-#214
      The code might be better(?) organized as
       if (moduled != null) {
           if (mif != nuill) {
               ...
               return false;
           }
           ....
       }

(3) It might be better to move the addModuleRequires() functionality into
       ModuleInfo implementation? in which it can simply take a Manifest
object
       and take whatever info it might need from the manifest to create the
       new ModuleInfo object. The sun.tools.jar.Main might not want to
know the
       details of how to transfer the necessary from a Manifest to a
ModuleInfo.
       For example, later you might find you would need an additional
attribute
       from a existing Manifest, ideally you should be able to simply
update the
       ModuleInfo implementation, without touching the
sun.tools.jar.Main (the
       minfo.write(zos) will take care of the writing...)

(4) With the migration of new try-resource in getJarPath(), the code
might look
      much better by simply do

      if (man == null)
          return files;
      ...
      if (value == null)
           return files

       And we probably don't need check if (rt != null)
       new JarFile() should never return null, it will through an
IOException if it
       fails.

-Sherman



On 08/30/2011 04:29 PM, Mandy Chung wrote:

>  Updated webrev:
>     http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/modularize-jar.01/
>
> To modularize an existing jar file:
> 1. use 'I' flag to specify the module name
>
>   $ jar ufI astro.jar org.astro@2.0
>
>   $ jar tvf astro.jar
>       0 Tue Aug 30 16:15:04 PDT 2011 META-INF/
>     74 Tue Aug 30 16:15:04 PDT 2011 META-INF/MANIFEST.MF
>       0 Tue Aug 30 16:15:00 PDT 2011 org/
>       0 Tue Aug 30 16:15:00 PDT 2011 org/astro/
>  276 Tue Aug 30 16:15:00 PDT 2011 org/astro/World.class
>   172 Tue Aug 30 16:21:26 PDT 2011 module-info.class<---- newly generated
>
> 2. add module-info.class to the jar file as an input file
>
>   $ jar uf greetings.jar -C modules/com.greetings module-info.class
>
> Installing a modular JAR file in the module library
>     $ jmod install greetings.jar astro.jar -L mlib
>
> Run in module mode
>     $ java -L mlib -m com.greetings
>
> Details:
> The jar tool provides a new option to transform a plain old JAR file
> into a modular JAR:
>    -I {module-name@version}
>       Generate "module-info.class" entry in the JAR file [a]
>
> It is designed to handle simple cases that:
>    * requires the default Java platform module [b]
>    * exports all public types
>    * main entry point as specified in the manifest or in the 'e'
>      flag, if any
>    * infers the dependencies from 'Class-Path' attribute if it is
>      a modular JAR file, and the specific module name of
>      that version will be added in the module dependency [c].
>
> The jar tool will determine if the inputfiles include a
> "module-info.class" entry that indicates that it is a modular
> application [d].  If exists, 'I' flag cannot be specified.
>
> Implementation: I use the classfile library that Jon Gibbons wrote
> to generate the module-info.class.  Since Jon will be updating
> javac and classfile per the module-info spec [3], I modified
> ModuleExport_attribute and ModuleRequires_attribute classes in
> a simple way but the right change will come when they are updated
> per the proposed spec.
>
> Footnotes:
> [a] jarfile name tends to be unnamed or contains '-' character.
>     The module name will not be inferred from the jarfile name.
> [b] Private and internal APIs in the jdk are not exported.
>     If the JAR file depends on any of these APIs and running
>     in module mode, this will result in a runtime error.
> [c] If the path listed in the 'Class-Path' attribute is not a
>     modular JAR, it will simply issue a warning and continue.
>     Module dependencies may be incomplete in this case.
> [d] If an application has been modularized, module-info.java that
>     describes the module definition is placed in the root of the
>     source tree and compiled in the same destination directory as
>     its classes.
>
> Mandy
> [1]
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2011-August/001452.html

Reply | Threaded
Open this post in threaded view
|

Re: Review request: jar tool to transform a plain jar file to a modular jar

Alan Bateman
In reply to this post by Mandy Chung
Mandy Chung wrote:
>  Updated webrev:
>     http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/modularize-jar.01/
I see Sherman has done a detailed review on the sun.tools.jar changes.

I'm looking at additions to java.util.jar.JarFile and I'm wondering if
the class or method descriptions should having wording to explain that a
JAR file can have a legacy manifest and also a module-info. I also
wonder if we will need to update the JAR specification too. Minor
comment is that the new methods should have @since 1.8.

A passing comment on SimpleLibrary lines 980-988 is that they could be
replaced with Files.write(is, md.toPath().resolve("info"));

The sun.tools.jar.ModuleInfo class looks very useful and I'm sure we
will have other places that need to generate module-info classes. Would
it make sense to put it in org.openjdk.jigsaw? If so then it would be
good to have it support requires optional too.

-Alan.
Reply | Threaded
Open this post in threaded view
|

Re: Review request: jar tool to transform a plain jar file to a modularjar

Michael Bien
In reply to this post by Mandy Chung
  any chance to add shortcuts to those common execution commands?

$ java mlib com.greetings
as shortcut for
$ java -L mlib -m com.greetings

or even
$ java mlib
if there is such thing like a default main method.

modules have great potential to make java commandline friendly.

regards,
-michael

On 08/26/2011 03:20 AM, Mandy Chung wrote:

> I implemented the support in the jar tool to transform a plain JAR file
> to a modular JAR file [1]:
>    http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/modularize-jar/
>
> Simple example:
> To modularize an existing jar file (two ways - see below):
>   $ jar uf astro.jar -module org.astro@2.0
>   $ jar uf greetings.jar -C modules/com.greetings module-info.class
>   $ java -jar greetings.jar   # launch in legacy mode
>
> Installing a modular JAR file in the module library
>     $ jmod install greetings.jar astro.jar -L mlib
>
> Run in module mode
>     $ java -L mlib -m com.greetings
>
> Details:
> The jar tool provides a new option to transform a plain old JAR file
> into a modular JAR:
>    -module {module-name@version}
>       Generate META-INF/module-info.class entry in the JAR file [a]
>
> It is designed to handle simple cases that:
>    * requires the default Java platform module [b]
>    * exports all public types
>    * main entry point as specified in the manifest or in the 'e'
>      flag, if any
>    * infers the dependencies from 'Class-Path' attribute if it is
>      a modular JAR file, and the specific module name of
>      that version will be added in the module dependency [c].
>
> The jar tool will determine if the inputfiles include a
> "module-info.class" entry that indicates that it is a modular
> application.  If exists, it will create a modular JAR by copying
> the module-info.class file in the META-INF directory [d].
> The "-module" option is not required in this case.
>
> Initially I propose to use the GNU-style option (--module) but
> it should really be a separate project to have the existing
> command-line tools to support GNU-style options that deserves
> its own JEP [2].  I like the option be a long-name option rather
> than the traditional single letter option which I don't see such
> inconsistency is an issue or find a good letter for it :)
>
> Since the generated module-info.class is typically fairly simple,
> it doesn't seem really necessary to output the module-info.java
> and so didn't implement that.
>
> Implementation: I use the classfile library that Jon Gibbons wrote
> to generate the module-info.class.  Since Jon will be updating
> javac and classfile per the module-info spec [3], I modified
> ModuleExport_attribute and ModuleRequires_attribute classes in
> a simple way but the right change will come when they are updated
> per the proposed spec.
>
> Footnotes:
> [a] jarfile name tends to be unnamed or contains '-' character.
>     The module name will not be inferred from the jarfile name.
> [b] Private and internal APIs in the jdk are not exported.
>     If the JAR file depends on any of these APIs and running
>     in module mode, this will result in a runtime error.
> [c] If the path listed in the 'Class-Path' attribute is not a
>     modular JAR, it will simply issue a warning and continue.
>     Module dependencies may be incomplete in this case.
> [d] If an application has been modularized, module-info.java that
>     describes the module definition is placed in the root of the
>     source tree and compiled in the same destination directory as
>     its classes.
>
> BTW, I'm working on adding a new regression test for this new
> functionality. It looks like I have to write a better framework
> to cover different test cases and do the validation that may take
> some time.
>
> Thanks
> Mandy
>
> [1]
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2011-April/001251.html
> [2] http://openjdk.java.net/jeps
> [3]
> http://openjdk.java.net/projects/jigsaw/doc/topics/modules-classfile.html

Reply | Threaded
Open this post in threaded view
|

Re: Review request: jar tool to transform a plain jar file to a modular jar

Mandy Chung
In reply to this post by Xueming Shen
  Sherman, Alan,

Thanks for the review.  I did some further cleanup and here is an
updated webrev:
    http://cr.openjdk.java.net/~mchung/jigsaw/webrevs/modularize-jar.02/

My comments inlined below.

On 09/12/11 6:06, Alan Bateman wrote:
>
> I'm looking at additions to java.util.jar.JarFile and I'm wondering if
> the class or method descriptions should having wording to explain that
> a JAR file can have a legacy manifest and also a module-info. I also
> wonder if we will need to update the JAR specification too.

Yes, the JAR spec needs to be updated but the JAR spec is in the pubs
repo.  I prefer to defer to update the docs later before we integrate to
JDK 8 and track it in the bug database for the time being.

> Minor comment is that the new methods should have @since 1.8.

Fixed.
>
> A passing comment on SimpleLibrary lines 980-988 is that they could be
> replaced with Files.write(is, md.toPath().resolve("info"));
>

Thanks for the suggestion.  I think you meant Files.copy.

> The sun.tools.jar.ModuleInfo class looks very useful and I'm sure we
> will have other places that need to generate module-info classes.
> Would it make sense to put it in org.openjdk.jigsaw? If so then it
> would be good to have it support requires optional too.

org.openjdk.jigsaw.** is included in the base module while
sun.tools.jar.ModuleInfo is for tools to use and it depends on
com.sun.tools.classfile library.  Sorry I initially also thought that
org.openjdk.jigsaw might be an appropriate package to put this class but
it's not.

I propose to leave it in sun.tools.jar package for now and move it to an
appropriate package later.  I have modified this class to support
require optional and other modifiers.

On 08/31/11 12:37, Xueming Shen wrote:
> Comments on the sun.tools.jar.Main
>
> (1) #197-#201
>      Shouldn't the "equals" be the "ignore case equals"?

module-info.class is a jar entry whose name is case-sensitive.  On the
other hand, there is a bug in L628 that should use equals instead of
equalsIgnoreCase method.  I fixed that in the new webrev.

>
> (2) #202-#214
>      The code might be better(?) organized as
>       if (moduled != null) {
>           if (mif != nuill) {
>               ...
>               return false;
>           }
>           ....
>       }
>

Did some additional cleanup too (see the webrev).

> (3) It might be better to move the addModuleRequires() functionality into
>       ModuleInfo implementation? in which it can simply take a
> Manifest object
>       and take whatever info it might need from the manifest to create
> the
>       new ModuleInfo object. The sun.tools.jar.Main might not want to
> know the
>       details of how to transfer the necessary from a Manifest to a
> ModuleInfo.
>       For example, later you might find you would need an additional
> attribute
>       from a existing Manifest, ideally you should be able to simply
> update the
>       ModuleInfo implementation, without touching the
> sun.tools.jar.Main (the
>       minfo.write(zos) will take care of the writing...)
>

In fact, I'd like to keep sun.tools.jar.ModuleInfo not to couple with
Manifest.  It's a helper class to generate module-info.class with the
given module metadata.

> (4) With the migration of new try-resource in getJarPath(), the code
> might look
>      much better by simply do
>
>      if (man == null)
>          return files;
>      ...
>      if (value == null)
>           return files
>

I tried to retain the existing structure except those which bug me :)
I folded the three if-statements into 1.

>       And we probably don't need check if (rt != null)
>       new JarFile() should never return null, it will through an
> IOException if it
>       fails.
>

Removed the check.

Thanks
Mandy
Reply | Threaded
Open this post in threaded view
|

Re: Review request: jar tool to transform a plain jar file to a modularjar

Mandy Chung
In reply to this post by Michael Bien
  On 09/12/11 07:15, Michael Bien wrote:
>  any chance to add shortcuts to those common execution commands?
>
> $ java mlib com.greetings
> as shortcut for
> $ java -L mlib -m com.greetings
>

BTW, you don't need the -L option when the module is in the system
module library.

IMO, I don't think "java -m com.greetings" is not command-line
friendly.  "-m" option is a way to tell the launcher to run the
application in module mode.  With "-m" option, it'll launch the
application in legacy mode as today.

> or even
> $ java mlib
> if there is such thing like a default main method.
>

I think you meant the default module.  This only saves to type the
module name for one single module in a module library.

> modules have great potential to make java commandline friendly.
>

It already does as you no longer need the classpath in modules :)

Mandy
Reply | Threaded
Open this post in threaded view
|

Re: Review request: jar tool to transform a plain jar file to a modular jar

Alan Bateman
In reply to this post by Mandy Chung
Mandy Chung wrote:
>
> Yes, the JAR spec needs to be updated but the JAR spec is in the pubs
> repo.  I prefer to defer to update the docs later before we integrate
> to JDK 8 and track it in the bug database for the time being.
You're right, we just need to remember to go back to it later.

>
>> Minor comment is that the new methods should have @since 1.8.
>
> Fixed.
MODULEINFO_NAME will need this too.


>>
>> A passing comment on SimpleLibrary lines 980-988 is that they could
>> be replaced with Files.write(is, md.toPath().resolve("info"));
>>
>
> Thanks for the suggestion.  I think you meant Files.copy.
Sorry yes, I meant Files.copy.

>
>> The sun.tools.jar.ModuleInfo class looks very useful and I'm sure we
>> will have other places that need to generate module-info classes.
>> Would it make sense to put it in org.openjdk.jigsaw? If so then it
>> would be good to have it support requires optional too.
>
> org.openjdk.jigsaw.** is included in the base module while
> sun.tools.jar.ModuleInfo is for tools to use and it depends on
> com.sun.tools.classfile library.  Sorry I initially also thought that
> org.openjdk.jigsaw might be an appropriate package to put this class
> but it's not.
Okay for now but I think we're going to need this in other places soon
to avoid duplicating the code.

-Alan.