excluding transitive module

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

excluding transitive module

Jochen Theodorou
Hi all,

I am wondering if there is a solution purely in the module-info for this:

* Project requires Library1 and Library2
* SomeLibrary requires SharedApi
* OtherLibrary requires SharedApiImpl

The problem is, that it will not compile because SharedApi and
SharedApiImpl implement the same packages but not the same logic. One is
just an API, while the other is an implementation of the API.

Scenario 1:

What would seem to be logic to me is to exclude SharedApi, but does
SomeLibrary then need its requires to be changed to SharedApiImpl? How
would I do that if I cannot change SomeLibrary?

Scenario 2:

Let us assume I could change the module-info of SomeLibrary. How would I
express that Shared API is compilation only? Is that what "requires
SomeLibrary for compilation" would be for?

bye Jochen
Reply | Threaded
Open this post in threaded view
|

Re: excluding transitive module

Alan Bateman
On 14/04/2020 09:24, Jochen Theodorou wrote:

> Hi all,
>
> I am wondering if there is a solution purely in the module-info for this:
>
> * Project requires Library1 and Library2
> * SomeLibrary requires SharedApi
> * OtherLibrary requires SharedApiImpl
>
> The problem is, that it will not compile because SharedApi and
> SharedApiImpl implement the same packages but not the same logic. One is
> just an API, while the other is an implementation of the API.
This seems futile without first cleaning up the architectural issues.

How does SharedApi locate the implementation? Is this configurable or
does it always assume it's in the same run-time package as itself? If
the API and implementation (or default implementation) have to be in the
same run-time package then it leads to SharedApi and SharedApiImpl
needing to be combined into one module, not two.

Why is OtherLibrary requiring SharedApiImpl? This suggests that
SharedApiImpl is more than an implementation, does it have an additional
implementation specific API that OtherLibrary make use of?

If you decide to combine the API and default implementation into the one
module then it will look something like this:

module api {
     exports api;
     uses spi;
}

The default implementation will be non-public classes in the same
package as the public API classes. The `uses spi` is stand in for
whatever the service type that some other implementation can provide
(assume it is indeed pluggable, why else would they be separated in the
first place?).

If you don't want to combine the API and implementation into the same
module then you'll have to move the implementation to another package so
you have something like the following fully encapsulated implementation:

module impl {
     requires api;
     provides spi with impl;
}

Different run-time packages means they can't use package-privates. If
there is an implementation-specific/extension API then impl will need to
export that package. There will be presumably be api types in the impl
specific API so the requires would change to `requires transitive api`.
OtherImpl will `requires impl`.

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: excluding transitive module

Jochen Theodorou
On 14.04.20 11:09, Alan Bateman wrote:

> On 14/04/2020 09:24, Jochen Theodorou wrote:
>> Hi all,
>>
>> I am wondering if there is a solution purely in the module-info for this:
>>
>> * Project requires Library1 and Library2
>> * SomeLibrary requires SharedApi
>> * OtherLibrary requires SharedApiImpl
>>
>> The problem is, that it will not compile because SharedApi and
>> SharedApiImpl implement the same packages but not the same logic. One is
>> just an API, while the other is an implementation of the API.
> This seems futile without first cleaning up the architectural issues.
>
> How does SharedApi locate the implementation?

It does not, there is no SPI. It just implements the interfaces

> Is this configurable or
> does it always assume it's in the same run-time package as itself?

actually it does not even assume somebody else may have it. So yes.

> If
> the API and implementation (or default implementation) have to be in the
> same run-time package then it leads to SharedApi and SharedApiImpl
> needing to be combined into one module, not two.

so the best way would be to change SomeLibrary to require
SharedApiImpl... which means changing the library after the fact...
which is a tad difficult

> Why is OtherLibrary requiring SharedApiImpl? This suggests that
> SharedApiImpl is more than an implementation, does it have an additional
> implementation specific API that OtherLibrary make use of?

There can be different implementations (it is just no an SPI
architecture). There is one that then builds on top of SharedAPI, but it
is not the implementation I need.

> If you decide to combine the API and default implementation into the one
> module then it will look something like this:
>
> module api {
>      exports api;
>      uses spi;
> }
>
> The default implementation will be non-public classes in the same
> package as the public API classes. The `uses spi` is stand in for
> whatever the service type that some other implementation can provide
> (assume it is indeed pluggable, why else would they be separated in the
> first place?).
>
> If you don't want to combine the API and implementation into the same
> module then you'll have to move the implementation to another package so
> you have something like the following fully encapsulated implementation:
>
> module impl {
>      requires api;
>      provides spi with impl;
> }
>
> Different run-time packages means they can't use package-privates. If
> there is an implementation-specific/extension API then impl will need to
> export that package. There will be presumably be api types in the impl
> specific API so the requires would change to `requires transitive api`.
> OtherImpl will `requires impl`.

I see.

bye Jochen


Reply | Threaded
Open this post in threaded view
|

Re: excluding transitive module

Alex Buckley
On 4/14/2020 3:12 AM, Jochen Theodorou wrote:

> On 14.04.20 11:09, Alan Bateman wrote:
>> On 14/04/2020 09:24, Jochen Theodorou wrote:
>>> Hi all,
>>>
>>> I am wondering if there is a solution purely in the module-info for
>>> this:
>>>
>>> * Project requires Library1 and Library2
>>> * SomeLibrary requires SharedApi
>>> * OtherLibrary requires SharedApiImpl
>>>
>>> The problem is, that it will not compile because SharedApi and
>>> SharedApiImpl implement the same packages but not the same logic. One is
>>> just an API, while the other is an implementation of the API.

>> How does SharedApi locate the implementation?
>
> It does not, there is no SPI. It just implements the interfaces

>> Why is OtherLibrary requiring SharedApiImpl? This suggests that
>> SharedApiImpl is more than an implementation, does it have an additional
>> implementation specific API that OtherLibrary make use of?
>
> There can be different implementations (it is just no an SPI
> architecture). There is one that then builds on top of SharedAPI, but it
> is not the implementation I need.

All the talk of "implements the same packages" and "an implementation of
the API" and "just implements the interfaces" is besides the point.
There are no services here. All the module system has to work with are
the packages exported by SharedApi and SharedApiImpl. It's fine for
Library1 to require SharedApi, and for Library2 to require
SharedApiImpl, but if the exported packages overlap, then it's not fine
for Project to require, indirectly, both SharedApi (via Library1) and
SharedApiImpl (via Library2). That is, SharedApi and SharedApiImpl are
not composable. If you had put them both on the classpath in JDK 8, then
you would have had a broken system. If you want to compose an
application that includes both, then one of them has to change.

There is no need to speak of "transitive" modules because `requires
transitive` has not entered the picture.

Alex
Reply | Threaded
Open this post in threaded view
|

Re: excluding transitive module

Remi Forax
In reply to this post by Jochen Theodorou
Hi Jochen,
JPMS has no notion of of API and implementation of the same jar. It's a concept of your build tool and not something JPMS knows.

The notion of compilation dependencies and runtime dependencies is not a concept of JPMS but a concept of your build tools.

In term of JPMS, if you want to be able to use a jar at compile time and another one at runtime,
both should have the same module name, in fact they should have exactly the same module descriptor (the same module-info).

So in term of dependency,
- for,Maven/Gradle, the compile dependency will use SharedAPI and the runtime dependency SharedApiImpl
- for JPMS, both have the same module-info and if you want to depend on that module, just require it.

From the Maven Central POV, obviously, you have two jars so they can not have the same arfifact ID (coordinate), but they contains the same module-info.class.

regards,
Rémi

----- Mail original -----
> De: "Jochen Theodorou" <[hidden email]>
> À: "jigsaw-dev" <[hidden email]>
> Envoyé: Mardi 14 Avril 2020 10:24:34
> Objet: excluding transitive module

> Hi all,
>
> I am wondering if there is a solution purely in the module-info for this:
>
> * Project requires Library1 and Library2
> * SomeLibrary requires SharedApi
> * OtherLibrary requires SharedApiImpl
>
> The problem is, that it will not compile because SharedApi and
> SharedApiImpl implement the same packages but not the same logic. One is
> just an API, while the other is an implementation of the API.
>
> Scenario 1:
>
> What would seem to be logic to me is to exclude SharedApi, but does
> SomeLibrary then need its requires to be changed to SharedApiImpl? How
> would I do that if I cannot change SomeLibrary?
>
> Scenario 2:
>
> Let us assume I could change the module-info of SomeLibrary. How would I
> express that Shared API is compilation only? Is that what "requires
> SomeLibrary for compilation" would be for?
>
> bye Jochen
Reply | Threaded
Open this post in threaded view
|

Re: excluding transitive module

Jochen Theodorou
On 14.04.20 20:20, Remi Forax wrote:

> Hi Jochen,
> JPMS has no notion of of API and implementation of the same jar. It's a concept of your build tool and not something JPMS knows.
>
> The notion of compilation dependencies and runtime dependencies is not a concept of JPMS but a concept of your build tools.
>
> In term of JPMS, if you want to be able to use a jar at compile time and another one at runtime,
> both should have the same module name, in fact they should have exactly the same module descriptor (the same module-info).
>
> So in term of dependency,
> - for,Maven/Gradle, the compile dependency will use SharedAPI and the runtime dependency SharedApiImpl
> - for JPMS, both have the same module-info and if you want to depend on that module, just require it.
>
>  From the Maven Central POV, obviously, you have two jars so they can not have the same arfifact ID (coordinate), but they contains the same module-info.class.

If the case is api and implementation and the same module name, then
using the build tool to exclude the api dependency works.

If the module name is not the same I have not found a solution. It is a
problem here because of shared packages.

I found this being a problem btw with a lot of jee libraries.

Of course I can make my project require whatever, but if it is libraries
I do not have under control things get a bit difficult without changing
them. But changing them is not an action really well supported by any
build tool I know.

bye Jochen
Reply | Threaded
Open this post in threaded view
|

Re: excluding transitive module

Jochen Theodorou
In reply to this post by Alex Buckley
On 14.04.20 19:38, Alex Buckley wrote:
[...]
> It's fine for
> Library1 to require SharedApi, and for Library2 to require
> SharedApiImpl, but if the exported packages overlap, then it's not fine
> for Project to require, indirectly, both SharedApi (via Library1) and
> SharedApiImpl (via Library2). That is, SharedApi and SharedApiImpl are
> not composable.

And what do you do when you are in that situation?

> If you had put them both on the classpath in JDK 8, then
> you would have had a broken system. If you want to compose an
> application that includes both, then one of them has to change.
> There is no need to speak of "transitive" modules because `requires
> transitive` has not entered the picture.

Without the module system I can just exclude one and be fine. With
module system I get into a split package problem adn have not found a
real solution yet

bye Jochen

Reply | Threaded
Open this post in threaded view
|

Re: excluding transitive module

Alex Buckley
On 4/15/2020 12:00 AM, Jochen Theodorou wrote:

> On 14.04.20 19:38, Alex Buckley wrote:
> [...]
>> It's fine for
>> Library1 to require SharedApi, and for Library2 to require
>> SharedApiImpl, but if the exported packages overlap, then it's not fine
>> for Project to require, indirectly, both SharedApi (via Library1) and
>> SharedApiImpl (via Library2). That is, SharedApi and SharedApiImpl are
>> not composable.
>
> And what do you do when you are in that situation?
>
>> If you had put them both on the classpath in JDK 8, then
>> you would have had a broken system. If you want to compose an
>> application that includes both, then one of them has to change.
>> There is no need to speak of "transitive" modules because `requires
>> transitive` has not entered the picture.
>
> Without the module system I can just exclude one and be fine. With
> module system I get into a split package problem adn have not found a
> real solution yet

In the JDK 8 world:  If SharedApi and SharedApiImpl have different
names, then (as you implied to Remi) Maven is going to treat them
independently and put both on the classpath. Yes, you can realize that
their content overlaps and exclude one of them in your POM, but *every*
*single* *user* has to travel that road independently! If a user doesn't
realize, and doesn't exclude, then their classpath has split packages
and their application is broken. I think it's outrageous that the
developers of SharedApi and SharedApiImpl imposed this tax on *every*
*single* *user* and no-one held them to account.

Post JDK 8:  Only when SharedApi and SharedApiImpl are modularized does
the presence of overlapping exports become clear, and prevent an
application from being composed unsafely.

There is nothing you can do in the module-info.java of your application
to express that its module graph indirectly requires the uncomposable
modules M,N,O and that any `requires M` and `requires N` clauses should
be overridden with `requires O`. This would be a maintenance nightmare.
Mail their developers and tell them what they've done. Every library
including its own copy of a standard API is a bug, not a feature.

(If a library is explicitly written to be loaded in its own class
loader, such as in the OSGi environment, then things are different:
private copies of APIs are OK, up to a point. However, the libraries you
are using are plainly from the JDK 8 classpath era and have been lightly
modularized -- enough for each vendor to let their module be required
and jlinked, but not enough to address the bigger architectural issue of
reuse / composability.)

Alex
Reply | Threaded
Open this post in threaded view
|

Re: excluding transitive module

Remi Forax
In reply to this post by Jochen Theodorou
----- Mail original -----
> De: "Jochen Theodorou" <[hidden email]>
> À: "jigsaw-dev" <[hidden email]>
> Envoyé: Mercredi 15 Avril 2020 08:55:00
> Objet: Re: excluding transitive module

> On 14.04.20 20:20, Remi Forax wrote:
>> Hi Jochen,
>> JPMS has no notion of of API and implementation of the same jar. It's a concept
>> of your build tool and not something JPMS knows.
>>
>> The notion of compilation dependencies and runtime dependencies is not a concept
>> of JPMS but a concept of your build tools.
>>
>> In term of JPMS, if you want to be able to use a jar at compile time and another
>> one at runtime,
>> both should have the same module name, in fact they should have exactly the same
>> module descriptor (the same module-info).
>>
>> So in term of dependency,
>> - for,Maven/Gradle, the compile dependency will use SharedAPI and the runtime
>> dependency SharedApiImpl
>> - for JPMS, both have the same module-info and if you want to depend on that
>> module, just require it.
>>
>>  From the Maven Central POV, obviously, you have two jars so they can not have
>>  the same arfifact ID (coordinate), but they contains the same
>>  module-info.class.
>
> If the case is api and implementation and the same module name, then
> using the build tool to exclude the api dependency works.
>
> If the module name is not the same I have not found a solution. It is a
> problem here because of shared packages.

as Alex said, this should be fixed by the maintainers of the jars, not you.

>
> I found this being a problem btw with a lot of jee libraries.
>
> Of course I can make my project require whatever, but if it is libraries
> I do not have under control things get a bit difficult without changing
> them. But changing them is not an action really well supported by any
> build tool I know.

For the Jakarta EE libraries, the main issue is that there are not many jars that has a module-info, so the default strategy to name the automatic module with the jar name fail here because an api an its implementations should have the same name.
Injecting the right module-info until your pull request is accepted is an escape hatch.

>
> bye Jochen

Rémi