Patterns for libraries to access private state of user modules

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Patterns for libraries to access private state of user modules

Gunnar Morling
Hi,

Libraries such as Hibernate often need to access private state of classes
from other modules (e.g. when using field access for JPA entities).

Assuming that such library and the user's JAR are both provided as JPMS
(named) modules, it had been my understanding so far [1], that the library
should allow to pass in a MethodHandles.Lookup instance which has private
access to the user's classes. This could be passed via a parameter during
the library bootstrap, retrieved via a callback or similar.

But discussing this with Alex Buckley at Devoxx Belgium last week, he
suggested that this may not be the best level of abstraction. User code
should never have to deal with low-level APIs such as Lookup, instead some
sort of utility method should be provided by the library. It's not quite
clear to me how this would look like, though. I can't see how a utility
method in my library could obtain the Lookup, as MethodHandles#lookup() is
caller-sensitive and must be invoked from within the module that wishes to
grant private access to the library.

So I'm curious whether there are any recommendations around this. Having
some kind of documented API best practices for library authors in the same
situation as me would be very helpful.

I could also help to write that up, provided there's an agreed on pattern
to do this.

Thanks,

--Gunnar

[1]
http://in.relation.to/2017/04/11/accessing-private-state-of-java-9-modules/#can-you-handle-the-var
Reply | Threaded
Open this post in threaded view
|

Re: Patterns for libraries to access private state of user modules

David Lloyd
On Mon, Nov 13, 2017 at 10:03 AM, Gunnar Morling <[hidden email]> wrote:
> Hi,
>
> Libraries such as Hibernate often need to access private state of classes
> from other modules (e.g. when using field access for JPA entities).
>
> [...] I can't see how a utility
> method in my library could obtain the Lookup, as MethodHandles#lookup() is
> caller-sensitive and must be invoked from within the module that wishes to
> grant private access to the library.

My understanding is that you should be using
MethodHandlers.privateLookupIn(userClazz, lookup()); and that the user
module should be "open" to you (i.e. they have to opt in to granting
reflection access).  Part of this was tied up in the discussion around
the need for standardized module names for spec modules, so a user can
choose the right module name:

  opens com.mycompany.dao to java.spec.jpa;

Then you only need a Lookup from the "java.spec.jpa" module to create
the private lookup in the user class.

--
- DML
Reply | Threaded
Open this post in threaded view
|

Re: Patterns for libraries to access private state of user modules

Alan Bateman
On 13/11/2017 16:54, David Lloyd wrote:

> :
> My understanding is that you should be using
> MethodHandlers.privateLookupIn(userClazz, lookup()); and that the user
> module should be "open" to you (i.e. they have to opt in to granting
> reflection access).  Part of this was tied up in the discussion around
> the need for standardized module names for spec modules, so a user can
> choose the right module name:
>
>    opens com.mycompany.dao to java.spec.jpa;
>
> Then you only need a Lookup from the "java.spec.jpa" module to create
> the private lookup in the user class.
>
Yes, this is one approach. The application module will require the JPA
module and at the same time open the packages with classes that have
annotations on private members to the JPA module. If JPA is pluggable
(which I think it is) then the code doing the deep reflection may be in
an JPA implementation module, in which case the JPA API module may have
to open the packages to the JPA implementation module. JAXB is one
example doing this already and it would be good to exercise it with
other libraries to ensure that it is feasible.

The other approach is to have the application module provide the Lookup
as a capability. For some libraries, Guice for example, there is
explicit initialization which could be the place to provide the Lookup
object. If you there is no explicit initialization then coercion may be
an option, maybe at build time or run-time. Coercion is of course much
easier if there is an app server or container using module layers as it
can open any package in any module in the layer to other modules.

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: Patterns for libraries to access private state of user modules

Gunnar Morling
2017-11-13 20:18 GMT+01:00 Alan Bateman <[hidden email]>:

> On 13/11/2017 16:54, David Lloyd wrote:
>
>> :
>> My understanding is that you should be using
>> MethodHandlers.privateLookupIn(userClazz, lookup()); and that the user
>> module should be "open" to you (i.e. they have to opt in to granting
>> reflection access).  Part of this was tied up in the discussion around
>> the need for standardized module names for spec modules, so a user can
>> choose the right module name:
>>
>>    opens com.mycompany.dao to java.spec.jpa;
>>
>> Then you only need a Lookup from the "java.spec.jpa" module to create
>> the private lookup in the user class.
>>
>> Yes, this is one approach. The application module will require the JPA
> module and at the same time open the packages with classes that have
> annotations on private members to the JPA module. If JPA is pluggable
> (which I think it is) then the code doing the deep reflection may be in an
> JPA implementation module, in which case the JPA API module may have to
> open the packages to the JPA implementation module.


How would that look like like exactly? I.e. how could the JPA API module
"pass on" the open packages of the user's module to the JPA implementation
module?

I can see how it'd work if the user's module would be an entirely open
module. But it seems desirable to limit it to open up only specific
packages to specific modules (the JPA provider). The user could open to a
specific implementation module of course, but that'd come at the cost of
reduced portability.

JAXB is one example doing this already and it would be good to exercise it
> with other libraries to ensure that it is feasible.
>

Is this happening in the JAXB reference implementation? Would you perhaps
have any pointers to specifics so I could take a look?

>
> The other approach is to have the application module provide the Lookup as
> a capability. For some libraries, Guice for example, there is explicit
> initialization which could be the place to provide the Lookup object.


Yes, what I like about it is that it makes the need for private access more
apparent to the user, if e.g. the library bootstrap is designed in a way
that a Lookup object is mandatory (as opposed to the requirement for
opening up the user's module/packages, which essentially can only be
documented or things will fail at runtime). Although in reality it'd likely
have to be multiple Lookup objects, one for each module of the user
containing entities or similar. It'll be interesting how to collect those
Lookups so they can be passed to the library without making them available
to other modules, too.

But in respect to my initial question you seem to suggest indeed that the
user would have to pass a literal Lookup object which they'd have obtained
themselves. So indeed the user code would have to deal with the specifics
of Lookup objects.


> If you there is no explicit initialization then coercion may be an option,
> maybe at build time or run-time. Coercion is of course much easier if there
> is an app server or container using module layers as it can open any
> package in any module in the layer to other modules.


Right, I think containers have different ways for making this work, be it
via modification of module descriptors, injection of some helper code into
the user's module etc. My original question here was primarily focused on
usage under plain SE and envisioned patterns there.

-Alan
>
Reply | Threaded
Open this post in threaded view
|

Re: Patterns for libraries to access private state of user modules

Alan Bateman
On 14/11/2017 17:07, Gunnar Morling wrote:

>
>
> :
>
>     JAXB is one example doing this already and it would be good to
>     exercise it with other libraries to ensure that it is feasible.
>
>
> Is this happening in the JAXB reference implementation? Would you
> perhaps have any pointers to specifics so I could take a look?
Yes, in javax.xml.bind.ContextFinder where it uses its
delegateAddOpensToImplModule method to open the packages to the
implementation module.


>
> Yes, what I like about it is that it makes the need for private access
> more apparent to the user, if e.g. the library bootstrap is designed
> in a way that a Lookup object is mandatory (as opposed to the
> requirement for opening up the user's module/packages, which
> essentially can only be documented or things will fail at runtime).
> Although in reality it'd likely have to be multiple Lookup objects,
> one for each module of the user containing entities or similar. It'll
> be interesting how to collect those Lookups so they can be passed to
> the library without making them available to other modules, too.
If the library needs to reflect into several non-open modules then it
would minimally need a Lookup with MODULE access for each one.

>
> But in respect to my initial question you seem to suggest indeed that
> the user would have to pass a literal Lookup object which they'd have
> obtained themselves. So indeed the user code would have to deal with
> the specifics of Lookup objects.
Right, either explicitly or oerced by some means.

-Alan