Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

Martijn Verburg
Hi All,

Stephen's comments are in his blog post: http://blog.joda.org/2018/03/j
pms-negative-benefits.html

He does raise some interesting points.  Perhaps it's worth spending some
time before 11 is out to see if there is a combination of Maven and JPMS
changes / clarifications that can address some of these pain points.

If Stephen is missing something, then that's an area we could document
(and/or publicise better if already documented.  That is something I and
others can help to address through the usual JUG Leaders / Champions / Info
Q etc channels.

@Stephen are you able to share the source code?  It may be that there is
another
path that could be explored to mitigate some of the challenges you faced.

@Robert - I've been thinking about the difficulty of getting enough
engineering time for Maven committers to add support for major features
like JPMS.  I'll start a separate thread with you on that.

Cheers,
Martijn
Reply | Threaded
Open this post in threaded view
|

Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

David Lloyd
On Fri, Mar 23, 2018 at 5:19 AM, Martijn Verburg
<[hidden email]> wrote:

> Hi All,
>
> Stephen's comments are in his blog post: http://blog.joda.org/2018/03/j
> pms-negative-benefits.html
>
> He does raise some interesting points.  Perhaps it's worth spending some
> time before 11 is out to see if there is a combination of Maven and JPMS
> changes / clarifications that can address some of these pain points.
>
> If Stephen is missing something, then that's an area we could document
> (and/or publicise better if already documented.  That is something I and
> others can help to address through the usual JUG Leaders / Champions / Info
> Q etc channels.
>
> @Stephen are you able to share the source code?  It may be that there is
> another
> path that could be explored to mitigate some of the challenges you faced.
>
> @Robert - I've been thinking about the difficulty of getting enough
> engineering time for Maven committers to add support for major features
> like JPMS.  I'll start a separate thread with you on that.

Case in point: I've had
https://github.com/apache/maven-compiler-plugin/pull/1 sitting out
there since January.  This commit is a key to what is at least a
somewhat workable strategy for MR JARs and multi-JVM build and testing
for us.  Robert has said that the change is acceptable, but indicated
that there may be a resource problem in terms of getting the plugin to
a releasable state.

I (and others I work with) are more than willing to contribute what
extra time we have to fixing or improving things in Maven and its
plugins, and even in the JDK itself, but the other side of that is
that the fixes do need to be accepted in a timely manner.  With Java
11 just around the corner, the time is now.

--
- DML
Reply | Threaded
Open this post in threaded view
|

Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

Stephen Colebourne
In reply to this post by Martijn Verburg
On 23 March 2018 at 10:19, Martijn Verburg <[hidden email]> wrote:
> Stephen's comments are in his blog post:
> http://blog.joda.org/2018/03/jpms-negative-benefits.html

Firstly, I want to emphasise that my goal is effectively that of a
retrospective, to examine what hasn't worked so well and to improve
things from where we are. The jigsaw team did a great job in getting
the feature out at all - I want to make sure it is used rather than
ignored or used as a blocker to progressing the JDK.


> @Stephen are you able to share the source code?

The source code is all public:
https://github.com/ThreeTen/threeten-extra
https://github.com/JodaOrg/joda-parent
https://github.com/JodaOrg/joda-convert
https://github.com/JodaOrg/joda-beans

While some of the problems are tool-based, the most fundamental issues
are about JPMS itself.

At the heart of the problem is the split between class-path and
module-path. Since this split has happened now, and can't "unhappen"
what is needed is a way to make it easier to manage (fix the problems
the split has created).

As it stands, a library developer cannot control whether they run on
the class-path or module-path. This increases the bug-surface of the
library, and requires testing in two different environments (which is
not widely known). Without being able to insist that a library is on
the module-path it is also clear that the benefits of strong
encapsulation and reliable configuration don't apply to library
consumers.

My belief is that a way needs to be found for a library author to
insist that their library is run as a named module. There are probably
a number of ways this could be achieved - I'm interested in whether
the change makes things better, not what the specific change is.

One approach is to say that modular jar files are always treated as
named modules:

- a modular jar on the class-path is treated as being named, not part
of the unnamed module It is therefore encapsulated, but depends on the
unnamed module (class-path), so does not have full reliable
configuration. There is the potential for some incompatibility with
this change where code that uses the modular jar now can't access the
encapsulated packages, but this is a Good thing (as the library author
specifically coded for that encapsulation). Any incompatibilities are
smaller than JPMS has already caused, and could be managed with the
existing command line flags. I would hope this does not require a JVM
spec change to achieve.

A second approach is to say that the module author must mark modules strict:

- a module author could mark a module as "strict" in module-info.java
so that it is not permitted to be on the class-path. This does not
force the whole application to move to the module-path - only strict
modules and their dependencies would need to move. At runtime, if a
strict module is found on the class-path an error occurs. I suspect
this requires a JVM spec change, so may be harder.

There may well be more approaches. My gut feeling is that the first
approach is the better one, being relatively simple and implementable,
but of course I may be wrong.

Ultimately, the requirement is that library authors who go to the
effort of adding module-info.java should see some benefits in doing
so, where today they only have increased cost through running as both
an unnamed and named module.

thanks
Stephen
Reply | Threaded
Open this post in threaded view
|

Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

Remi Forax
----- Mail original -----
> De: "Stephen Colebourne" <[hidden email]>
> À: "jigsaw-dev" <[hidden email]>
> Envoyé: Vendredi 23 Mars 2018 13:51:06
> Objet: Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

> On 23 March 2018 at 10:19, Martijn Verburg <[hidden email]> wrote:
>> Stephen's comments are in his blog post:
>> http://blog.joda.org/2018/03/jpms-negative-benefits.html
>
> Firstly, I want to emphasise that my goal is effectively that of a
> retrospective, to examine what hasn't worked so well and to improve
> things from where we are. The jigsaw team did a great job in getting
> the feature out at all - I want to make sure it is used rather than
> ignored or used as a blocker to progressing the JDK.
>
>
>> @Stephen are you able to share the source code?
>
> The source code is all public:
> https://github.com/ThreeTen/threeten-extra
> https://github.com/JodaOrg/joda-parent
> https://github.com/JodaOrg/joda-convert
> https://github.com/JodaOrg/joda-beans
>
> While some of the problems are tool-based, the most fundamental issues
> are about JPMS itself.
>
> At the heart of the problem is the split between class-path and
> module-path. Since this split has happened now, and can't "unhappen"
> what is needed is a way to make it easier to manage (fix the problems
> the split has created).
>
> As it stands, a library developer cannot control whether they run on
> the class-path or module-path. This increases the bug-surface of the
> library, and requires testing in two different environments (which is
> not widely known). Without being able to insist that a library is on
> the module-path it is also clear that the benefits of strong
> encapsulation and reliable configuration don't apply to library
> consumers.
>
> My belief is that a way needs to be found for a library author to
> insist that their library is run as a named module. There are probably
> a number of ways this could be achieved - I'm interested in whether
> the change makes things better, not what the specific change is.
>
> One approach is to say that modular jar files are always treated as
> named modules:
>
> - a modular jar on the class-path is treated as being named, not part
> of the unnamed module It is therefore encapsulated, but depends on the
> unnamed module (class-path), so does not have full reliable
> configuration. There is the potential for some incompatibility with
> this change where code that uses the modular jar now can't access the
> encapsulated packages, but this is a Good thing (as the library author
> specifically coded for that encapsulation). Any incompatibilities are
> smaller than JPMS has already caused, and could be managed with the
> existing command line flags. I would hope this does not require a JVM
> spec change to achieve.
>
> A second approach is to say that the module author must mark modules strict:
>
> - a module author could mark a module as "strict" in module-info.java
> so that it is not permitted to be on the class-path. This does not
> force the whole application to move to the module-path - only strict
> modules and their dependencies would need to move. At runtime, if a
> strict module is found on the class-path an error occurs. I suspect
> this requires a JVM spec change, so may be harder.
>
> There may well be more approaches. My gut feeling is that the first
> approach is the better one, being relatively simple and implementable,
> but of course I may be wrong.
>
> Ultimately, the requirement is that library authors who go to the
> effort of adding module-info.java should see some benefits in doing
> so, where today they only have increased cost through running as both
> an unnamed and named module.
>
> thanks
> Stephen

You can check dynamically if a class is in the unamed module (classpath) or not and emit an error message.

This is exactly what the VM does when the class version is wrong.

Rémi

Reply | Threaded
Open this post in threaded view
|

Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

David Lloyd
In reply to this post by Stephen Colebourne
On Fri, Mar 23, 2018 at 7:51 AM, Stephen Colebourne
<[hidden email]> wrote:

> One approach is to say that modular jar files are always treated as
> named modules:
>
> - a modular jar on the class-path is treated as being named, not part
> of the unnamed module It is therefore encapsulated, but depends on the
> unnamed module (class-path), so does not have full reliable
> configuration. There is the potential for some incompatibility with
> this change where code that uses the modular jar now can't access the
> encapsulated packages, but this is a Good thing (as the library author
> specifically coded for that encapsulation). Any incompatibilities are
> smaller than JPMS has already caused, and could be managed with the
> existing command line flags. I would hope this does not require a JVM
> spec change to achieve.

This would cause problems for containers which do not use JPMS yet
(which is to say: containers); such frameworks may behave poorly
(having an expectation that they were loaded as JPMS modules) or fail
to load (if there is some kind of hypothetical enforcement at a JVM
level).

> A second approach is to say that the module author must mark modules strict:
>
> - a module author could mark a module as "strict" in module-info.java
> so that it is not permitted to be on the class-path. This does not
> force the whole application to move to the module-path - only strict
> modules and their dependencies would need to move. At runtime, if a
> strict module is found on the class-path an error occurs. I suspect
> this requires a JVM spec change, so may be harder.

I don't think this would resolve the container problem, though it
would allow containers to refuse to load such JARs.  I'm not sure
that's a net improvement for users though.

> Ultimately, the requirement is that library authors who go to the
> effort of adding module-info.java should see some benefits in doing
> so, where today they only have increased cost through running as both
> an unnamed and named module.

I agree, we've effectively doubled the testing effort for frameworks
due to JPMS, and tripled in the case where frameworks must include MR
JAR supplements for 8 vs 9+.  In a word, this sucks.

--
- DML
Reply | Threaded
Open this post in threaded view
|

Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

Jason Greene
In reply to this post by Stephen Colebourne

> On Mar 23, 2018, at 7:51 AM, Stephen Colebourne <[hidden email]> wrote:
>
> Without being able to insist that a library is on
> the module-path it is also clear that the benefits of strong
> encapsulation and reliable configuration don't apply to library
> consumers.

Why would a framework author ever want to cut off 99% of their users that are not using JPMS, most of which are completely happy with the arguably more flexible classpath? I think the only way you solve this problem is if you resurrect (and extend) some of the original goals to support common unification and bridging across runtime models. Then the industry as a whole could be pushing adoption.

-Jason
Reply | Threaded
Open this post in threaded view
|

Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

Stephen Colebourne
In reply to this post by David Lloyd
On 23 March 2018 at 13:17, David Lloyd <[hidden email]> wrote:

> On Fri, Mar 23, 2018 at 7:51 AM, Stephen Colebourne
> <[hidden email]> wrote:
>> One approach is to say that modular jar files are always treated as
>> named modules:
>>
>> - a modular jar on the class-path is treated as being named, not part
>> of the unnamed module It is therefore encapsulated, but depends on the
>> unnamed module (class-path), so does not have full reliable
>> configuration. There is the potential for some incompatibility with
>> this change where code that uses the modular jar now can't access the
>> encapsulated packages, but this is a Good thing (as the library author
>> specifically coded for that encapsulation). Any incompatibilities are
>> smaller than JPMS has already caused, and could be managed with the
>> existing command line flags. I would hope this does not require a JVM
>> spec change to achieve.
>
> This would cause problems for containers which do not use JPMS yet
> (which is to say: containers); such frameworks may behave poorly
> (having an expectation that they were loaded as JPMS modules) or fail
> to load (if there is some kind of hypothetical enforcement at a JVM
> level).

The classes in the modular jar file would still be in the same
classloader (because same classloader module loading is the standard
strategy). Other than encapsulating internal packages, the container
really shouldn't see any changes should it? At some point containers
will support JPMS anyway.

Stephen
Reply | Threaded
Open this post in threaded view
|

Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

David Lloyd
On Fri, Mar 23, 2018 at 8:49 AM, Stephen Colebourne
<[hidden email]> wrote:

> On 23 March 2018 at 13:17, David Lloyd <[hidden email]> wrote:
>> On Fri, Mar 23, 2018 at 7:51 AM, Stephen Colebourne
>> <[hidden email]> wrote:
>>> One approach is to say that modular jar files are always treated as
>>> named modules:
>>>
>>> - a modular jar on the class-path is treated as being named, not part
>>> of the unnamed module It is therefore encapsulated, but depends on the
>>> unnamed module (class-path), so does not have full reliable
>>> configuration. There is the potential for some incompatibility with
>>> this change where code that uses the modular jar now can't access the
>>> encapsulated packages, but this is a Good thing (as the library author
>>> specifically coded for that encapsulation). Any incompatibilities are
>>> smaller than JPMS has already caused, and could be managed with the
>>> existing command line flags. I would hope this does not require a JVM
>>> spec change to achieve.
>>
>> This would cause problems for containers which do not use JPMS yet
>> (which is to say: containers); such frameworks may behave poorly
>> (having an expectation that they were loaded as JPMS modules) or fail
>> to load (if there is some kind of hypothetical enforcement at a JVM
>> level).
>
> The classes in the modular jar file would still be in the same
> classloader (because same classloader module loading is the standard
> strategy). Other than encapsulating internal packages, the container
> really shouldn't see any changes should it? At some point containers
> will support JPMS anyway.

The container won't see any changes, but the framework will if the
container is not creating a JPMS module for it.

At some point, containers _may_ support JPMS.  However, this seems to
be turning into quite a technical challenge in practice.  And until
there are some implementations supporting JPMS, there will be no specs
for supporting JPMS (at least, unless the world has gone mad, which
perhaps it has).

--
- DML