ClassLoader.getResources vs Module.getResourceAsStream

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

ClassLoader.getResources vs Module.getResourceAsStream

Michał Zegan
Hello.
When reading docs for jdk9 and jdk10 it seems that those methods work in
a bit different way when it goes to encapsulation:
Module.getResourceAsStream will retrieve the resource without a problem
if a package is opened to the caller module, probably including the fact
that it will find a resource when the calling module is the same as one
represented by the module object.
But, ClassLoader.getResources and other resource methods seem to require
unconditional package open.
Why? I don't quite understand that distinction.
Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Alan Bateman
On 14/07/2018 14:00, Michał Zegan wrote:

> Hello.
> When reading docs for jdk9 and jdk10 it seems that those methods work in
> a bit different way when it goes to encapsulation:
> Module.getResourceAsStream will retrieve the resource without a problem
> if a package is opened to the caller module, probably including the fact
> that it will find a resource when the calling module is the same as one
> represented by the module object.
> But, ClassLoader.getResources and other resource methods seem to require
> unconditional package open.
> Why? I don't quite understand that distinction.
ClassLoaders, especially in a delegation chain, have no notion of "who"
is trying to locate the resource. The ClassLoader.getResourceXXX methods
are also not final. All told, the ClassLoader.getResourceXXX cannot
reliably support qualified opens so this is why they are specified to
only locate resources in modules when the package is open to all modules.

The general guideline is to use Class or Module getResourceXXX when you
want to locate a resource in your own module or another specific module.
Use ClassLoader.getResourceXXX when you want to search the class path.
If you follow that then it makes it a lot easier to migrate existing
code to modules.

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Michał Zegan
What is then a recommendation for searching for all resources with name
x that i can access? Something like load all configuration including
some kind of extensions. I cannot list resources easily.

W dniu 14.07.2018 o 17:31, Alan Bateman pisze:

> On 14/07/2018 14:00, Michał Zegan wrote:
>> Hello.
>> When reading docs for jdk9 and jdk10 it seems that those methods work in
>> a bit different way when it goes to encapsulation:
>> Module.getResourceAsStream will retrieve the resource without a problem
>> if a package is opened to the caller module, probably including the fact
>> that it will find a resource when the calling module is the same as one
>> represented by the module object.
>> But, ClassLoader.getResources and other resource methods seem to require
>> unconditional package open.
>> Why? I don't quite understand that distinction.
> ClassLoaders, especially in a delegation chain, have no notion of "who"
> is trying to locate the resource. The ClassLoader.getResourceXXX methods
> are also not final. All told, the ClassLoader.getResourceXXX cannot
> reliably support qualified opens so this is why they are specified to
> only locate resources in modules when the package is open to all modules.
>
> The general guideline is to use Class or Module getResourceXXX when you
> want to locate a resource in your own module or another specific module.
> Use ClassLoader.getResourceXXX when you want to search the class path.
> If you follow that then it makes it a lot easier to migrate existing
> code to modules.
>
> -Alan

Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Alan Bateman
On 14/07/2018 16:38, Michał Zegan wrote:
> What is then a recommendation for searching for all resources with name
> x that i can access? Something like load all configuration including
> some kind of extensions. I cannot list resources easily.
>
Services is the cleaner way to do this kind of thing, esp. if you have
flexibility to change the code and move away from legacy ClassLoader
getResources.

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Michał Zegan
It is a completely new code. It is not modularized for now because some
dependencies are not modularized, but I want it to be compatible to ease
later modularization.
My actual goal is to load xml files defining and configuring some
factories, from all modules that contain them. Not sure how would you do
that with services?

W dniu 14.07.2018 o 17:53, Alan Bateman pisze:

> On 14/07/2018 16:38, Michał Zegan wrote:
>> What is then a recommendation for searching for all resources with name
>> x that i can access? Something like load all configuration including
>> some kind of extensions. I cannot list resources easily.
>>
> Services is the cleaner way to do this kind of thing, esp. if you have
> flexibility to change the code and move away from legacy ClassLoader
> getResources.
>
> -Alan

Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Sander Mak
In that case you'd expose the factories through the services mechanism (`provides com.acme.api.MyWidgetFactory with com.acme.factories.XmlBasedWidgetFactory` in the module descriptor). Or, if you must expose the XML itself to the outside world rather than the factories, you can create a service that offers the XML as String or InputStream. Each module then has an implementation reading its own encapsulated XML. The former approach is preferable IMO.

Note that you can also use the services mechanism without module descriptors by placing text files in META-INF/services (https://docs.oracle.com/javase/tutorial/ext/basics/spi.html). However, if your code isn't a reusable library itself, you can also choose to modularize your own code and depend on non-modularized dependencies through their automatic module name (http://branchandbound.net/blog/java/2017/12/automatic-module-name/).

Sander

On 14 Jul 2018, at 17:58, Michał Zegan <[hidden email]<mailto:[hidden email]>> wrote:

It is a completely new code. It is not modularized for now because some
dependencies are not modularized, but I want it to be compatible to ease
later modularization.
My actual goal is to load xml files defining and configuring some
factories, from all modules that contain them. Not sure how would you do
that with services?

W dniu 14.07.2018 o 17:53, Alan Bateman pisze:
On 14/07/2018 16:38, Michał Zegan wrote:
What is then a recommendation for searching for all resources with name
x that i can access? Something like load all configuration including
some kind of extensions. I cannot list resources easily.

Services is the cleaner way to do this kind of thing, esp. if you have
flexibility to change the code and move away from legacy ClassLoader
getResources.

-Alan


Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Remi Forax


----- Mail original -----
> De: "Sander Mak" <[hidden email]>
> À: "jigsaw-dev" <[hidden email]>
> Envoyé: Lundi 16 Juillet 2018 08:52:41
> Objet: Re: ClassLoader.getResources vs Module.getResourceAsStream

> In that case you'd expose the factories through the services mechanism
> (`provides com.acme.api.MyWidgetFactory with
> com.acme.factories.XmlBasedWidgetFactory` in the module descriptor). Or, if you
> must expose the XML itself to the outside world rather than the factories, you
> can create a service that offers the XML as String or InputStream. Each module
> then has an implementation reading its own encapsulated XML. The former
> approach is preferable IMO.
>
> Note that you can also use the services mechanism without module descriptors by
> placing text files in META-INF/services
> (https://docs.oracle.com/javase/tutorial/ext/basics/spi.html). However, if your
> code isn't a reusable library itself, you can also choose to modularize your
> own code and depend on non-modularized dependencies through their automatic
> module name
> (http://branchandbound.net/blog/java/2017/12/automatic-module-name/).
>

You have to do both, each service declared in a module-info MUST be declared in META-INF/services too because you can use a modular jar in the classpath as a plain old jar.

Alan, we should patch jar to warn when there is a module-info.class that declare services with no corresponding META-INF/services.

> Sander
>

Rémi

> On 14 Jul 2018, at 17:58, Michał Zegan
> <[hidden email]<mailto:[hidden email]>> wrote:
>
> It is a completely new code. It is not modularized for now because some
> dependencies are not modularized, but I want it to be compatible to ease
> later modularization.
> My actual goal is to load xml files defining and configuring some
> factories, from all modules that contain them. Not sure how would you do
> that with services?
>
> W dniu 14.07.2018 o 17:53, Alan Bateman pisze:
> On 14/07/2018 16:38, Michał Zegan wrote:
> What is then a recommendation for searching for all resources with name
> x that i can access? Something like load all configuration including
> some kind of extensions. I cannot list resources easily.
>
> Services is the cleaner way to do this kind of thing, esp. if you have
> flexibility to change the code and move away from legacy ClassLoader
> getResources.
>
> -Alan
Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Stephen Colebourne
In reply to this post by Alan Bateman
In my experience (as I've written before), ClassLoader.getResources is
perhaps the biggest pain point I've experienced in trying to move
beyond Java 8. The method seems to have been very widely used, and IMO
was considered to be preferred over Class.getResourceXxx. And it is
very confusing to use once modules come into play.

There is no simple replacement for the ability to search for resources
across jar files. Module.getResourceXxx cannot be used as it only
returns one resource from the unamed module, when there could be many
(and the docs are no particularly clear on what they do). This is very
inconvenient. Returning a stream instead of a URL also typically
involves wider code change to adopt. In addition, many projects are
still on Java 8, so can't use java.lang.Module anyway.

ServiceLoader is completely the wrong solution for config files. Its
far too heavyweight.

My solution (which I think is pretty horrible) has been to move config
files to be under META-INF.

Old location:
 org/joda/convert
New location:
 META-INF/org/joda/convert

Its backwards incompatible to downstream users, but at least it works
with ClassLoader.getResources

Given there are no good solutions to normal coding problems, I can't
help feeling that Jigsaw didn't get resource access quite right.

Stephen


On 14 July 2018 at 16:31, Alan Bateman <[hidden email]> wrote:

> On 14/07/2018 14:00, Michał Zegan wrote:
>>
>> Hello.
>> When reading docs for jdk9 and jdk10 it seems that those methods work in
>> a bit different way when it goes to encapsulation:
>> Module.getResourceAsStream will retrieve the resource without a problem
>> if a package is opened to the caller module, probably including the fact
>> that it will find a resource when the calling module is the same as one
>> represented by the module object.
>> But, ClassLoader.getResources and other resource methods seem to require
>> unconditional package open.
>> Why? I don't quite understand that distinction.
>
> ClassLoaders, especially in a delegation chain, have no notion of "who" is
> trying to locate the resource. The ClassLoader.getResourceXXX methods are
> also not final. All told, the ClassLoader.getResourceXXX cannot reliably
> support qualified opens so this is why they are specified to only locate
> resources in modules when the package is open to all modules.
>
> The general guideline is to use Class or Module getResourceXXX when you want
> to locate a resource in your own module or another specific module. Use
> ClassLoader.getResourceXXX when you want to search the class path. If you
> follow that then it makes it a lot easier to migrate existing code to
> modules.
>
> -Alan
Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Alan Bateman
On 16/07/2018 11:06, Stephen Colebourne wrote:
> In my experience (as I've written before), ClassLoader.getResources is
> perhaps the biggest pain point I've experienced in trying to move
> beyond Java 8. The method seems to have been very widely used, and IMO
> was considered to be preferred over Class.getResourceXxx. And it is
> very confusing to use once modules come into play.
Class.getResourceXXX is the better choice when locating ones own
resources, ClassLoader.getResourceXX is for searching for a resource in
other components, the ClassLoader provide some context on where to
search. There are of course libraries that were using ClassLoader to
locate their own resources and those cases may need some changes in the
event that the code is migrated to an explicit module and the resources
are encapsulated. If they aren't encapsulated then the existing code
using the ClassLoader methods will work of course.
>
> There is no simple replacement for the ability to search for resources
> across jar files.
The ClassLoader resources and getResources method work as before and
will locate resources in modular JARs when the resources are not
encapsulated.


> Module.getResourceXxx cannot be used as it only
> returns one resource from the unamed module, when there could be many
> (and the docs are no particularly clear on what they do). This is very
> inconvenient.
Module.getResourceXXX works with named modules too.

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Alan Bateman
In reply to this post by Remi Forax


On 16/07/2018 09:09, Remi Forax wrote:
> :
>
> Alan, we should patch jar to warn when there is a module-info.class that declare services with no corresponding META-INF/services.
>
I'll create an issue in JIRA for that. As things stands, the `jar` tool
does some sanity checks but it doesn't catch the case where the legacy
services configuration file doesn't match the module declaration. That
said, it probably needs the equivalent in one or more Maven plugins as
they are likely using the java.util.jar API rather than the `jar` tool.

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Bernard Amade
In reply to this post by Stephen Colebourne
>
> ServiceLoader is completely the wrong solution for config files. Its
> far too heavyweight.
>
after java 8
the case of internationalisation is the worst case of resource handling:
- creating diverse resource files (for different cultural contexts) means they might be spread along different jars
(you receive an app in your country - that uses a strange language-  ... then you create a specific deployment jar and add it to the app)
- since it is impossible to have the same "directory" (oups I mean "package") in different modules/jars it is required to create a special code to find the real directory
this is super-clumsy!
please find a  simpler approach ! (even if it entails exceptions to some principles)

thanks

Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Alan Bateman
On 16/07/2018 14:08, Bernard Amade wrote
> after java 8
> the case of internationalisation is the worst case of resource handling:
> - creating diverse resource files (for different cultural contexts) means they might be spread along different jars
> (you receive an app in your country - that uses a strange language-  ... then you create a specific deployment jar and add it to the app)
> - since it is impossible to have the same "directory" (oups I mean "package") in different modules/jars it is required to create a special code to find the real directory
> this is super-clumsy!
> please find a  simpler approach ! (even if it entails exceptions to some principles)
This sounds like a complaint about ResourceBundle. Best to start a new
thread if you have issues or suggestions for how it can be improved.
Also keep in mind that a huge effort went into getting ResourceBundle to
work with modules and there are several options for deploying
translations in different JAR files or modules (esp. if these
translation are .properties files rather than compiled resources). The
options are detailed in the ResourceBundle javadoc [1].

-Alan

[1]
https://download.java.net/java/early_access/jdk11/docs/api/java.base/java/util/ResourceBundle.html
Reply | Threaded
Open this post in threaded view
|

Re: ClassLoader.getResources vs Module.getResourceAsStream

Michał Zegan
In reply to this post by Sander Mak
Well, these are not stateless factories that you register as a service
and then discover to later use them to create some objects.
Those are factories that implement a specific algorithm like crypto
algorithm, or that are adapters for a crypto library like for JCA, in
the second case one factory class can be used to create many factories
for many algorithms depending on properties set on them.
Concrete algo instances are created when needed by connection objects.
That is why I cannot register them as some kind of services, i need
configuration to be able to define what algorithms are supported using
what factory class with what configuration for each algorithm, and this
would be ugly to do in code. And I would actually like for this
mechanism to also load some extensions, like secondary files in other
jars/modules that may register other algorithms that are not registered
by core, using builtin or extension factory classes.
I currently want to put that in META-INF that seems to be non-encapsulated.
W dniu 16.07.2018 o 08:52, Sander Mak pisze:

> In that case you'd expose the factories through the services mechanism (`provides com.acme.api.MyWidgetFactory with com.acme.factories.XmlBasedWidgetFactory` in the module descriptor). Or, if you must expose the XML itself to the outside world rather than the factories, you can create a service that offers the XML as String or InputStream. Each module then has an implementation reading its own encapsulated XML. The former approach is preferable IMO.
>
> Note that you can also use the services mechanism without module descriptors by placing text files in META-INF/services (https://docs.oracle.com/javase/tutorial/ext/basics/spi.html). However, if your code isn't a reusable library itself, you can also choose to modularize your own code and depend on non-modularized dependencies through their automatic module name (http://branchandbound.net/blog/java/2017/12/automatic-module-name/).
>
> Sander
>
> On 14 Jul 2018, at 17:58, Michał Zegan <[hidden email]<mailto:[hidden email]>> wrote:
>
> It is a completely new code. It is not modularized for now because some
> dependencies are not modularized, but I want it to be compatible to ease
> later modularization.
> My actual goal is to load xml files defining and configuring some
> factories, from all modules that contain them. Not sure how would you do
> that with services?
>
> W dniu 14.07.2018 o 17:53, Alan Bateman pisze:
> On 14/07/2018 16:38, Michał Zegan wrote:
> What is then a recommendation for searching for all resources with name
> x that i can access? Something like load all configuration including
> some kind of extensions. I cannot list resources easily.
>
> Services is the cleaner way to do this kind of thing, esp. if you have
> flexibility to change the code and move away from legacy ClassLoader
> getResources.
>
> -Alan
>
>