Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Plugins
Hi all,

I might have spoken too soon in my last October email [1]. Ever since,
I've
been having a hell of a time trying to get Gradle's so-called
„shaded“ API jar
to play nicely with the other modules in my JPMS-build-enabling Gradle
plugin [2]

For some reason, Gradle's generated artifact redundantly duplicates all
the
packages of two modules from the JDK: javax.xml and jdk.xml.dom.

Any other modules in the module path that read packages in either of
those two system
modules breaks the JPMS with your classic split package error:

----
$ java -ea src\eg\UnPatchable.java

Exception in thread "main" java.lang.module.ResolutionException:
Modules gradle.api and java.xml export package javax.xml.transform to
module
 com.github.javaparser.symbolsolver.core
        at
java.base/java.lang.module.Resolver.resolveFail(Resolver.java:885)
        at
java.base/java.lang.module.Resolver.failTwoSuppliers(Resolver.java:797)
        at
java.base/java.lang.module.Resolver.checkExportSuppliers(Resolver.java:718)
        at java.base/java.lang.module.Resolver.finish(Resolver.java:362)
        at
java.base/java.lang.module.Configuration.<init>(Configuration.java:141)
        at
java.base/java.lang.module.Configuration.resolve(Configuration.java:424)
        at
java.base/java.lang.module.Configuration.resolve(Configuration.java:256)
        at eg.UnPatchable.main(UnPatchable.java:21)

----

Or the same failure but with a different message:

----
$ java -ea --module-path lib src\eg\UnPatchable.java

Error occurred during initialization of boot layer
 java.lang.module.ResolutionException: Module gradle.api contains
package
 org.xml.sax, module java.xml exports package org.xml.sax to gradle.api
----

What the subject of this email is about, is that even attempting to
patch the Gradle API jar does not fix the split package problem:

----
$ java -ea --module-path lib --add-modules ALL-MODULE-PATH \
 --patch-module java.xml=lib\gradle-api-6.0-rc-2.jar \
 --patch-module jdk.xml.dom=lib\gradle-api-6.0-rc-2.jar; \
 --patch-module gradle.api=com.github.javaparser.symbolsolver.core
src\eg\UnPatchable.java

Error occurred during initialization of boot layer
 java.lang.module.ResolutionException: Modules gradle.api and java.xml
export
 package javax.xml.validation to module
com.github.javaparser.symbolsolver.core
----

Those errors are reproducible with these steps:

1. Download, unzip and cd into the attached example project [3]
2. From a terminal, enter the commands I entered in the snippets above

I'm hoping that it's not as bad as I think it is. Hopefully, it's
nothing worse than I just haven't learned what the right command is to
fix this.

Please, can jigsaw-dev shed some light on this so far intractable
problem? TIA.

----
[1] http://bit.ly/014298
[2] http://bit.ly/mrJar
[3] http://bit.ly/014309zip


Reply | Threaded
Open this post in threaded view
|

Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Alan Bateman
On 12/11/2019 02:30, Plugins wrote:

> Hi all,
>
> I might have spoken too soon in my last October email [1]. Ever since,
> I've
> been having a hell of a time trying to get Gradle's so-called
> „shaded“ API jar
> to play nicely with the other modules in my JPMS-build-enabling Gradle
> plugin [2]
>
> For some reason, Gradle's generated artifact redundantly duplicates all
> the
> packages of two modules from the JDK: javax.xml and jdk.xml.dom.
The Gradle devs will know but I assume these are coming from the
xml-apis-${version}.jar (Apache Xerces project). It's not clear, to me
anyway, why these API classes are still being distributed in 2019. The
XML processing API was added to Java SE 1.4, the XML stream API came a
bit later. The XML APIs started out in Java SE as "standalone
technologies" where they were in the JDK but could be upgraded by
deploying a newer version with the endorsed standards override
mechanism. It's possible, and probable, that developers have been
deploying xml-apis on the class path and got lucky that it didn't cause
problems. Anyway, none of this makes sense since Java SE 9 as both
JSR-206 and JSR-173 ceased to be standalone technologies (see the final
MR for both JSRs), and mechanism to upgrade them (the endorsed standards
override mechanism) has been removed.

-Alan.
Reply | Threaded
Open this post in threaded view
|

RE: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Plugins
In reply to this post by Plugins

Hi Alan,

> ...why these API classes are still being distributed in 2019...
> ...none of this makes sense since Java SE 9 as both
> JSR-206 and JSR-173 ceased to be standalone technologies...

You've put what I was thinking all along, into the perfect words there.
Thanks, Alan.

If you can answer a few questions for me please Alan, you could help
salvage the little sanity I have left after wrestling with this:

1. What's the correct command to patch javax.xml and jdk.xml.dom with
Gradle's API JAR?
2. If it is actually impossible to patch either of them with Gradle,
what's your best guess for why that might be?
3. How can I _exclude_ javax.xml and jdk.xml.dom from the module graph?
    • I have already tried --limit-modules, but haven't had any luck

In the meantime, I will try to muster up the will to report  the issue
to the Gradle devs.

TIA

-------- Original Message --------
Subject: Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is
At Least Technically Possible - Somehow
From: Alan Bateman <[hidden email]>
Date: Tue, November 12, 2019 1:04 am
To: Plugins <[hidden email]>, [hidden email]

On 12/11/2019 02:30, Plugins wrote:

> Hi all,
>
> I might have spoken too soon in my last October email [1]. Ever since,
> I've
> been having a hell of a time trying to get Gradle's so-called
> „shaded“ API jar
> to play nicely with the other modules in my JPMS-build-enabling Gradle
> plugin [2]
>
> For some reason, Gradle's generated artifact redundantly duplicates all
> the
> packages of two modules from the JDK: javax.xml and jdk.xml.dom.
The Gradle devs will know but I assume these are coming from the
xml-apis-${version}.jar (Apache Xerces project). It's not clear, to me
anyway, why these API classes are still being distributed in 2019. The
XML processing API was added to Java SE 1.4, the XML stream API came a
bit later. The XML APIs started out in Java SE as "standalone
technologies" where they were in the JDK but could be upgraded by
deploying a newer version with the endorsed standards override
mechanism. It's possible, and probable, that developers have been
deploying xml-apis on the class path and got lucky that it didn't cause
problems. Anyway, none of this makes sense since Java SE 9 as both
JSR-206 and JSR-173 ceased to be standalone technologies (see the final
MR for both JSRs), and mechanism to upgrade them (the endorsed standards

override mechanism) has been removed.

-Alan.
Reply | Threaded
Open this post in threaded view
|

Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Alan Bateman
On 12/11/2019 09:19, Plugins wrote:

> :
>
>
> If you can answer a few questions for me please Alan, you could help
> salvage the little sanity I have left after wrestling with this:
>
> 1. What's the correct command to patch javax.xml and jdk.xml.dom with
> Gradle's API JAR?
> 2. If it is actually impossible to patch either of them with Gradle,
> what's your best guess for why that might be?
> 3. How can I _exclude_ javax.xml and jdk.xml.dom from the module graph?
>      • I have already tried --limit-modules, but haven't had any luck
>
> In the meantime, I will try to muster up the will to report  the issue
> to the Gradle devs.
>
You can't use --patch-module to augment both java.xml and jdk.xml.dom
with the classes/resources from gradle-api-6.0-rc-2.jar as that will
create a split package issue. You might get away with `--limit-modules
java.se` and patching java.xml but it would be really odd to have
classes for Gradle, SLF4J, JodaTime, and more in the java.xml module. I
think the only sane approach is to deconstruct gradle-api-6.0-rc-2.jar
so that the components it contains can be used as automatic modules.

I'm not sure what to say about xml-apis. I assume it ends up in
gradle-api-6.0-rc-2.jar because something depends on it. I can't think
why anything might be depending on this relic in 2019. It might be
useful track down why it is included in the first place.

-Alan

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=536928
Reply | Threaded
Open this post in threaded view
|

RE: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Plugins
In reply to this post by Plugins

> ...You might get away with `--limit-modules
> java.se` and patching java.xml...

Awesome! I will give that a shot. Many thanks!

> ...it would be really odd to have classes for Gradle,
> SLF4J, JodaTime, and more in the java.xml module...


I take your advice from JEP 261's „Patching module content“ section
to heart: „The --patch-module option is intended only for testing and
debugging. Its use in production settings is strongly discouraged“ [1]

My reasons for patching is only to allow temporary access to the classes
under test by unit tests. Since those are conventionally in the same
package but in two different Gradle source sets.


> ...It might be useful track down why it is included in the first place...

I agree. I will see what I can find out about that.

Thanks for the Eclipse bug link too, Alan. Super informative!


[1] http://bit.ly/JEP261Patch

-------- Original Message --------
Subject: Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is
At Least Technically Possible - Somehow
From: Alan Bateman <[hidden email]>
Date: Tue, November 12, 2019 4:50 am
To: Plugins <[hidden email]>, [hidden email]

On 12/11/2019 09:19, Plugins wrote:

> :
>
>
> If you can answer a few questions for me please Alan, you could help
> salvage the little sanity I have left after wrestling with this:
>
> 1. What's the correct command to patch javax.xml and jdk.xml.dom with
> Gradle's API JAR?
> 2. If it is actually impossible to patch either of them with Gradle,
> what's your best guess for why that might be?
> 3. How can I _exclude_ javax.xml and jdk.xml.dom from the module graph?
> • I have already tried --limit-modules, but haven't had any luck
>
> In the meantime, I will try to muster up the will to report the issue
> to the Gradle devs.
>
You can't use --patch-module to augment both java.xml and jdk.xml.dom
with the classes/resources from gradle-api-6.0-rc-2.jar as that will
create a split package issue. You might get away with `--limit-modules
java.se` and patching java.xml but it would be really odd to have
classes for Gradle, SLF4J, JodaTime, and more in the java.xml module. I
think the only sane approach is to deconstruct gradle-api-6.0-rc-2.jar
so that the components it contains can be used as automatic modules.

I'm not sure what to say about xml-apis. I assume it ends up in
gradle-api-6.0-rc-2.jar because something depends on it. I can't think
why anything might be depending on this relic in 2019. It might be
useful track down why it is included in the first place.

-Alan

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=536928
Reply | Threaded
Open this post in threaded view
|

RE: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Plugins
In reply to this post by Plugins
> ...You might get away with `--limit-modules
> java.se` and patching java.xml...

„--limit-modules java.se“ wasn't enough. To get the java launcher
past some „error: ...“ messages, I needed to add a few more modules:
„--limit-modules java.se,jdk.compiler,jdk.zipfs“.

But then I was back to the same ResolutionException blocker that had
originally caused me to lose the will to live:

$ java -ea --limit-modules java.se,jdk.compiler,jdk.zipfs --patch-module
java.xml=lib\gradle-api-6.0-rc-2.jar src\eg\UnPatchable.java
Exception in thread "main" java.lang.module.ResolutionException: Module
gradle.api contains package org.w3c.dom.views, module java.xml exports
package org.w3c.dom.views to gradle.api
        at
java.base/java.lang.module.Resolver.resolveFail(Resolver.java:885)
        at
java.base/java.lang.module.Resolver.failTwoSuppliers(Resolver.java:789)
        at
java.base/java.lang.module.Resolver.checkExportSuppliers(Resolver.java:737)
        at java.base/java.lang.module.Resolver.finish(Resolver.java:362)
        at
java.base/java.lang.module.Configuration.<init>(Configuration.java:141)
        at
java.base/java.lang.module.Configuration.resolve(Configuration.java:424)
        at
java.base/java.lang.module.Configuration.resolve(Configuration.java:256)
        at eg.UnPatchable.main(UnPatchable.java:24)

It really is impossible to patch Gradle's API JAR, it looks like :(
Please tell me I'm wrong?


> ...I'm not sure what to say about xml-apis. I assume it ends up in
> gradle-api-6.0-rc-2.jar because something depends on it. I can't think
> why anything might be depending on this relic in 2019. It might be
> useful track down why it is included in the first place.

Gradle makes heavy use a lot of stuff in the javax.xml.* and org.w3c.*
packages internally [1] And, again, this shaded jar we're talking about
is generated on demand — IFF a Gradle plugin author happens to have
applied a specific internal Gradle plugin that is only ever applied to
assist in the development of custom Gradle plugins. So this api jar is
not actually „distributed“ in the same sense that other artifacts
are normally „distributed“ — through a public repository, say.

I can't say with 100% certainty, but I _think_ everything contained in
this shaded JAR is expected to never show up on the classpath(s) of end
users of my plugin for example.

So I think you're right, Alan. Those xml-apis dependencies in that
shaded JAR are probably just one of a ton of other rebundled internal
dependencies that Gradle itself needs to support the plugin development
use case.

----

[1] http://bit.ly/xmlapisHits



-------- Original Message --------
Subject: RE: Seeking Assurance That Patching gradle-api-6.{...}.jar Is
At Least Technically Possible - Somehow
From: "Plugins" <[hidden email]>
Date: Tue, November 12, 2019 6:30 am
To: "Alan Bateman" <[hidden email]>,
[hidden email]


> ...You might get away with `--limit-modules
> java.se` and patching java.xml...

Awesome! I will give that a shot. Many thanks!

> ...it would be really odd to have classes for Gradle,
> SLF4J, JodaTime, and more in the java.xml module...


I take your advice from JEP 261's „Patching module content“ section
to heart: „The --patch-module option is intended only for testing and
debugging. Its use in production settings is strongly discouraged“ [1]

My reasons for patching is only to allow temporary access to the classes
under test by unit tests. Since those are conventionally in the same
package but in two different Gradle source sets.


> ...It might be useful track down why it is included in the first place...

I agree. I will see what I can find out about that.

Thanks for the Eclipse bug link too, Alan. Super informative!


[1] http://bit.ly/JEP261Patch

-------- Original Message --------
Subject: Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is
At Least Technically Possible - Somehow
From: Alan Bateman <[hidden email]>
Date: Tue, November 12, 2019 4:50 am
To: Plugins <[hidden email]>, [hidden email]

On 12/11/2019 09:19, Plugins wrote:

> :
>
>
> If you can answer a few questions for me please Alan, you could help
> salvage the little sanity I have left after wrestling with this:
>
> 1. What's the correct command to patch javax.xml and jdk.xml.dom with
> Gradle's API JAR?
> 2. If it is actually impossible to patch either of them with Gradle,
> what's your best guess for why that might be?
> 3. How can I _exclude_ javax.xml and jdk.xml.dom from the module graph?
> • I have already tried --limit-modules, but haven't had any luck
>
> In the meantime, I will try to muster up the will to report the issue
> to the Gradle devs.
>
You can't use --patch-module to augment both java.xml and jdk.xml.dom
with the classes/resources from gradle-api-6.0-rc-2.jar as that will
create a split package issue. You might get away with `--limit-modules
java.se` and patching java.xml but it would be really odd to have
classes for Gradle, SLF4J, JodaTime, and more in the java.xml module. I
think the only sane approach is to deconstruct gradle-api-6.0-rc-2.jar
so that the components it contains can be used as automatic modules.

I'm not sure what to say about xml-apis. I assume it ends up in
gradle-api-6.0-rc-2.jar because something depends on it. I can't think
why anything might be depending on this relic in 2019. It might be
useful track down why it is included in the first place.

-Alan

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=536928
Reply | Threaded
Open this post in threaded view
|

Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Alan Bateman
On 12/11/2019 23:45, Plugins wrote:

> Gradle makes heavy use a lot of stuff in the javax.xml.* and org.w3c.*
> packages internally [1] And, again, this shaded jar we're talking about
> is generated on demand — IFF a Gradle plugin author happens to have
> applied a specific internal Gradle plugin that is only ever applied to
> assist in the development of custom Gradle plugins. So this api jar is
> not actually „distributed“ in the same sense that other artifacts
> are normally „distributed“ — through a public repository, say.
>
> I can't say with 100% certainty, but I _think_ everything contained in
> this shaded JAR is expected to never show up on the classpath(s) of end
> users of my plugin for example.
>
> So I think you're right, Alan. Those xml-apis dependencies in that
> shaded JAR are probably just one of a ton of other rebundled internal
> dependencies that Gradle itself needs to support the plugin development
> use case.
>
I've no doubt that Gradle is making use of the XML processing and W3C
DOM APIs but they are provided by the JDK so I don't know why Gradle 6.0
includes xml-apis-1.4.01.jar in its distribution. If you run with
`-Xlog:class+load` then I assume none of the javax.xml.* and org.w3c.*
classes are loaded from this JAR file, this goes for JDK 8 and older too.

I took a quite look at xml-apis-1.4.01.jar and it looks like a copy of
the XML API classes (not the implementation) from a very old JDK or JAXP
release. The class files v45.3 = JDK 1.0.2 :-) The class files seems to
have been compiled in Dec 2009 and I suspect were compiled from the JAXP
API classes that were in Java SE 6 (one of the classes defines a method
that was added in Java SE 6). So my guess is that this JAR file has been
obsolete for a long time. It may have been used for deployments on JDK 5
or older that could upgrade the JDK but want to use newer versions of
the XML APIs. Alternatively, maybe it was useful for the period when the
JDK didn't include the all of the W3C API.

Hopefully someone from Gradle can comment on this issue, my guess is
that xml-apis is not needed.

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Dawid Weiss
Hi Alan,

> I took a quite look at xml-apis-1.4.01.jar and it looks like a copy of
> the XML API classes (not the implementation) from a very old JDK or JAXP
> release. The class files v45.3 = JDK 1.0.2 :-)

Just FYI: the latest Apache Xerces release (that's June, 2018) still
depends on this jar...

http://repo1.maven.org/maven2/xerces/xercesImpl/2.12.0/xercesImpl-2.12.0.pom

Dawid
Reply | Threaded
Open this post in threaded view
|

Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Alan Bateman
On 14/11/2019 10:07, Dawid Weiss wrote:
> :
> Just FYI: the latest Apache Xerces release (that's June, 2018) still
> depends on this jar...
>
> http://repo1.maven.org/maven2/xerces/xercesImpl/2.12.0/xercesImpl-2.12.0.pom
>
A release in 2018 suggests it may still be maintained.  Do you, or
anyone else, have cycles to reach out to this project to help them? It's
not clear to me why they would want developers to downgrade the XML API
to where it was in Java SE 6.

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Dawid Weiss
They seem to be aware about it -- see this issue, for example:
https://issues.apache.org/jira/browse/XERCESJ-1689

Dawid

On Thu, Nov 14, 2019 at 12:33 PM Alan Bateman <[hidden email]> wrote:

>
> On 14/11/2019 10:07, Dawid Weiss wrote:
> > :
> > Just FYI: the latest Apache Xerces release (that's June, 2018) still
> > depends on this jar...
> >
> > http://repo1.maven.org/maven2/xerces/xercesImpl/2.12.0/xercesImpl-2.12.0.pom
> >
> A release in 2018 suggests it may still be maintained.  Do you, or
> anyone else, have cycles to reach out to this project to help them? It's
> not clear to me why they would want developers to downgrade the XML API
> to where it was in Java SE 6.
>
> -Alan
Reply | Threaded
Open this post in threaded view
|

Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Alan Bateman
On 14/11/2019 17:52, Dawid Weiss wrote:
> They seem to be aware about it -- see this issue, for example:
> https://issues.apache.org/jira/browse/XERCESJ-1689
>
Thanks but that looks to be xercesImpl.jar. The main troublemaker seems
to be xml-apis.jar as it has a copy of the JAXP APIs from JDK 6 or older.

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: Seeking Assurance That Patching gradle-api-6.{...}.jar Is At Least Technically Possible - Somehow

Dawid Weiss
I realize this. However xercesImpl has a POM dependency on those
xml-apis -- this dependency brings this JAR in for other projects
(like mine). I have no idea who published xml-apis [1] -- this is
indeed ancient code. Sigh.

Dawid

[1] http://repo1.maven.org/maven2/xml-apis/xml-apis/

On Thu, Nov 14, 2019 at 7:51 PM Alan Bateman <[hidden email]> wrote:
>
> On 14/11/2019 17:52, Dawid Weiss wrote:
> > They seem to be aware about it -- see this issue, for example:
> > https://issues.apache.org/jira/browse/XERCESJ-1689
> >
> Thanks but that looks to be xercesImpl.jar. The main troublemaker seems
> to be xml-apis.jar as it has a copy of the JAXP APIs from JDK 6 or older.
>
> -Alan