Proposal (revised): Allow illegal access to internal APIs by default in JDK 9

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Proposal (revised): Allow illegal access to internal APIs by default in JDK 9

mark.reinhold
(Thanks for all the feedback on the initial proposal [1].  Here's a
 revised version, which incorporates some of the suggestions received and
 includes a bit more advice.  An implementation is already available for
 testing in the Jigsaw EA builds [2].  Further comments welcome!)

Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention to the actual changes in
this release.  The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given at run time in JDK 8.

To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal-access operations to
internal APIs from code on the class path by default in JDK 9, and to
disallow them in a future release.  This will enable smoother application
migration in the near term, yet still enable and motivate the maintainers
of libraries and frameworks that use JDK-internal APIs to fix their code
to use proper exported APIs.

New command-line option: `--illegal-access`
-------------------------------------------

The recently-introduced `--permit-illegal-access` option [3] will be
replaced by a more-general option, `--illegal-access`.  This option takes
a single keyword parameter to specify a mode of operation, as follows:

  `--illegal-access=permit`

    This mode opens each package in each module in the run-time image to
    code in all unnamed modules, i.e., code on the class path, if that
    package existed in JDK 8.  This enables both static access, i.e., by
    compiled bytecode, and deep reflective access, via the platform's
    various reflection APIs.

    The first reflective-access operation to any such package causes a
    warning to be issued, but no warnings are issued after that point.
    This single warning describes how to enable further warnings.

    This mode will be the default for JDK 9.  It will be removed in a
    future release.

  `--illegal-access=warn`

    This mode is identical to `permit` except that a warning message is
    issued for each illegal reflective-access operation.  This is roughly
    equivalent to the current `--permit-illegal-access` option.

  `--illegal-access=debug`

    This mode is identical to `warn` except both a warning message and a
    stack trace are issued for each illegal reflective-access operation.
    This is roughly equivalent to combining `--permit-illegal-access`
    with `-Dsun.reflect.debugModuleAccessChecks`.

  `--illegal-access=deny`

    This mode disables all illegal-access operations except for those
    enabled by other command-line options, e.g., `--add-opens`.

    This mode will become the default in a future release.

When `deny` becomes the default mode then `permit` will likely remain
supported for at least one release, so that developers can continue to
migrate their code.  The `permit`, `warn`, and `debug` modes will, over
time, be removed, as will the `--illegal-access` option itself.  (For
launch-script compatibility the unsupported modes will most likely just
be ignored, after issuing a warning to that effect.)

How to prepare for the future
-----------------------------

The default mode, `--illegal-access=permit`, is intended to make you
aware when you have code on the class path that reflectively accesses
some JDK-internal API at least once.  To learn about all such accesses
you can use the `warn` or `debug` modes.  For each library or framework
on the class path that requires illegal access you have two options:

  - If the component's maintainers have already released a new,
    fixed version that no longer uses JDK-internal APIs then you
    can consider upgrading to that version.

  - If the component still needs to be fixed then we encourage you
    to contact its maintainers and ask them to replace their use
    of JDK-internal APIs with proper exported APIs [4].

If you must continue to use a component that requires illegal access then
you can eliminate the warning messages by using one or more `--add-opens`
options to open just those internal packages to which access is required.

To verify that your application is ready for the future, run it with
`--illegal-access=deny` along with any necessary `--add-opens` options.
Any remaining illegal-access errors will most likely be due to static
references from compiled code to JDK-internal APIs.  You can identify
those by running the `jdeps` tool with the `--jdk-internals` option.
(JDK 9 does not issue warnings for illegal static-access operations
because that would require deep JVM changes and degrade performance.)

Warning messages
----------------

The warning message issued when an illegal reflective-access operation is
detected has the following form:

    WARNING: Illegal reflective access by $PERPETRATOR to $VICTIM

where:

  - $PERPETRATOR is the fully-qualified name of the type containing
    the code that invoked the reflective operation in question plus
    the code source (i.e., JAR-file path), if available, and

  - $VICTIM is a string that describes the member being accessed,
    including the fully-qualified name of the enclosing type

In JDK 9's default mode, `--illegal-access=permit`, at most one of these
warning messages will be issued, accompanied by additional instructive
text.  Here is an example, from running Jython on the current Jigsaw EA
build [2]:

    $ java -jar jython-standalone-2.7.0.jar
    WARNING: An illegal reflective access operation has occurred
    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
    WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper
    WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
    WARNING: All illegal access operations will be denied in a future release
    Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
    [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
    Type "help", "copyright", "credits" or "license" for more information.
    >>> ^D
    $

If `--illegal-access=warn` is used then only warnings are displayed, with
no instructive text.  The run-time system makes a best-effort attempt to
suppress duplicate warnings for the same $PERPETRATOR and $VICTIM.  Here
is an example, again running Jython:

    $ java --illegal-access=warn -jar jython-standalone-2.7.0.jar
    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field sun.nio.ch.FileChannelImpl.fd
    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field java.io.FileDescriptor.fd
    WARNING: Illegal reflective access by org.python.core.PySystemState (file:/tmp/jython-standalone-2.7.0.jar) to method java.io.Console.encoding()
    Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
    [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
    Type "help", "copyright", "credits" or "license" for more information.
    >>> ^D
    $

Notes
-----

  - There is no `--illegal-access` mode that suppresses all warnings.
    This is intentional: It ensures that developers know that all
    illegal-access operations will be denied by default in a future
    release, at which time code that generates warnings today will fail.
    Warnings can be suppressed completely via one or more `--add-opens`
    options.

  - The first proposal [1] opened every package in every explicit module,
    rather than just the packages in modules in the run-time image, to
    every unnamed module.  Peter Levart pointed out [5] that this could
    tempt developers to use internal APIs that are new in JDK 9 (e.g.,
    `jdk.internal.misc.Unsafe`) and thus make the eventual transition
    from JDK 9 no less painful than that from JDK 8.  This proposal thus
    only opens internal packages that existed in JDK 8.

  - This proposal will require adjustments to JEP 260, "Encapsulate Most
    Internal APIs" [6].  APIs that are internal to the JDK will still be
    strongly encapsulated from the standpoint of code in modules, whether
    those modules are automatic or explicit, but they will not appear to
    be encapsulated at run time from the standpoint of code on the class
    path.

  - This change will not magically solve every JDK 9 adoption problem.
    The concrete types of the built-in class loaders are still different,
    `rt.jar` is still gone, the layout of a system image is still not the
    same, and the version string still has a new format.


[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html
[2] http://jdk.java.net/jigsaw/
[3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[4] This will usually but not always be possible, since there are still a
    few critical internal APIs without exported replacements [6].
[5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012708.html
[6] http://openjdk.java.net/jeps/260
Reply | Threaded
Open this post in threaded view
|

Re: Proposal (revised): Allow illegal access to internal APIs by default in JDK 9

David Blevins
I think this is the most pragmatic and reasoned middle ground one could possibly imagine.

I agree with the finely-tuned choices, specifically:

  - Not going completely silent.  Some asked for the ability to completely shut off the warnings.  This goes a little too far to one extreme.  There has to be some cost or discomfort or there won’t be the right movement.  IMHO, it would have been too far back the other way to go silent.  People will see these messages, they will post them to StackOverflow and get an education on how to go forward.  We need that.

  - Not choosing `--illegal-access=deny` by default for this version.  Some cited this as a setback.  There is an argument to be made that it would have been the bigger setback to go too strict too soon.  The danger would be when developers who don’t know this conversation and the details behind it learn “use this flag to make problems go away”.  At which point it becomes cargo cult and they’ll be afraid to stop using it because they never understood it completely in the first place.  Worse, we’re now buried under 3 years of lazy StackOverflow answers effectively telling people how to go backwards.

So although it feels less strict, one set of choices creates a conversation on how to move forward and one creates a conversation on how to go backward.

Of course time will tell, but I believe this proposal gets us what we want the soonest.


-David


> On Jun 5, 2017, at 11:45 AM, [hidden email] wrote:
>
> (Thanks for all the feedback on the initial proposal [1].  Here's a
> revised version, which incorporates some of the suggestions received and
> includes a bit more advice.  An implementation is already available for
> testing in the Jigsaw EA builds [2].  Further comments welcome!)
>
> Over time, as we've gotten closer and closer to the JDK 9 GA date, more
> and more developers have begun paying attention to the actual changes in
> this release.  The strong encapsulation of JDK-internal APIs has, in
> particular, triggered many worried expressions of concern that code that
> works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
> warning of this change was given at run time in JDK 8.
>
> To help the entire ecosystem migrate to the modular Java platform at a
> more relaxed pace I hereby propose to allow illegal-access operations to
> internal APIs from code on the class path by default in JDK 9, and to
> disallow them in a future release.  This will enable smoother application
> migration in the near term, yet still enable and motivate the maintainers
> of libraries and frameworks that use JDK-internal APIs to fix their code
> to use proper exported APIs.
>
> New command-line option: `--illegal-access`
> -------------------------------------------
>
> The recently-introduced `--permit-illegal-access` option [3] will be
> replaced by a more-general option, `--illegal-access`.  This option takes
> a single keyword parameter to specify a mode of operation, as follows:
>
>  `--illegal-access=permit`
>
>    This mode opens each package in each module in the run-time image to
>    code in all unnamed modules, i.e., code on the class path, if that
>    package existed in JDK 8.  This enables both static access, i.e., by
>    compiled bytecode, and deep reflective access, via the platform's
>    various reflection APIs.
>
>    The first reflective-access operation to any such package causes a
>    warning to be issued, but no warnings are issued after that point.
>    This single warning describes how to enable further warnings.
>
>    This mode will be the default for JDK 9.  It will be removed in a
>    future release.
>
>  `--illegal-access=warn`
>
>    This mode is identical to `permit` except that a warning message is
>    issued for each illegal reflective-access operation.  This is roughly
>    equivalent to the current `--permit-illegal-access` option.
>
>  `--illegal-access=debug`
>
>    This mode is identical to `warn` except both a warning message and a
>    stack trace are issued for each illegal reflective-access operation.
>    This is roughly equivalent to combining `--permit-illegal-access`
>    with `-Dsun.reflect.debugModuleAccessChecks`.
>
>  `--illegal-access=deny`
>
>    This mode disables all illegal-access operations except for those
>    enabled by other command-line options, e.g., `--add-opens`.
>
>    This mode will become the default in a future release.
>
> When `deny` becomes the default mode then `permit` will likely remain
> supported for at least one release, so that developers can continue to
> migrate their code.  The `permit`, `warn`, and `debug` modes will, over
> time, be removed, as will the `--illegal-access` option itself.  (For
> launch-script compatibility the unsupported modes will most likely just
> be ignored, after issuing a warning to that effect.)
>
> How to prepare for the future
> -----------------------------
>
> The default mode, `--illegal-access=permit`, is intended to make you
> aware when you have code on the class path that reflectively accesses
> some JDK-internal API at least once.  To learn about all such accesses
> you can use the `warn` or `debug` modes.  For each library or framework
> on the class path that requires illegal access you have two options:
>
>  - If the component's maintainers have already released a new,
>    fixed version that no longer uses JDK-internal APIs then you
>    can consider upgrading to that version.
>
>  - If the component still needs to be fixed then we encourage you
>    to contact its maintainers and ask them to replace their use
>    of JDK-internal APIs with proper exported APIs [4].
>
> If you must continue to use a component that requires illegal access then
> you can eliminate the warning messages by using one or more `--add-opens`
> options to open just those internal packages to which access is required.
>
> To verify that your application is ready for the future, run it with
> `--illegal-access=deny` along with any necessary `--add-opens` options.
> Any remaining illegal-access errors will most likely be due to static
> references from compiled code to JDK-internal APIs.  You can identify
> those by running the `jdeps` tool with the `--jdk-internals` option.
> (JDK 9 does not issue warnings for illegal static-access operations
> because that would require deep JVM changes and degrade performance.)
>
> Warning messages
> ----------------
>
> The warning message issued when an illegal reflective-access operation is
> detected has the following form:
>
>    WARNING: Illegal reflective access by $PERPETRATOR to $VICTIM
>
> where:
>
>  - $PERPETRATOR is the fully-qualified name of the type containing
>    the code that invoked the reflective operation in question plus
>    the code source (i.e., JAR-file path), if available, and
>
>  - $VICTIM is a string that describes the member being accessed,
>    including the fully-qualified name of the enclosing type
>
> In JDK 9's default mode, `--illegal-access=permit`, at most one of these
> warning messages will be issued, accompanied by additional instructive
> text.  Here is an example, from running Jython on the current Jigsaw EA
> build [2]:
>
>    $ java -jar jython-standalone-2.7.0.jar
>    WARNING: An illegal reflective access operation has occurred
>    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
>    WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper
>    WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
>    WARNING: All illegal access operations will be denied in a future release
>    Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
>    [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
>    Type "help", "copyright", "credits" or "license" for more information.
>>>> ^D
>    $
>
> If `--illegal-access=warn` is used then only warnings are displayed, with
> no instructive text.  The run-time system makes a best-effort attempt to
> suppress duplicate warnings for the same $PERPETRATOR and $VICTIM.  Here
> is an example, again running Jython:
>
>    $ java --illegal-access=warn -jar jython-standalone-2.7.0.jar
>    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
>    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field sun.nio.ch.FileChannelImpl.fd
>    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field java.io.FileDescriptor.fd
>    WARNING: Illegal reflective access by org.python.core.PySystemState (file:/tmp/jython-standalone-2.7.0.jar) to method java.io.Console.encoding()
>    Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
>    [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
>    Type "help", "copyright", "credits" or "license" for more information.
>>>> ^D
>    $
>
> Notes
> -----
>
>  - There is no `--illegal-access` mode that suppresses all warnings.
>    This is intentional: It ensures that developers know that all
>    illegal-access operations will be denied by default in a future
>    release, at which time code that generates warnings today will fail.
>    Warnings can be suppressed completely via one or more `--add-opens`
>    options.
>
>  - The first proposal [1] opened every package in every explicit module,
>    rather than just the packages in modules in the run-time image, to
>    every unnamed module.  Peter Levart pointed out [5] that this could
>    tempt developers to use internal APIs that are new in JDK 9 (e.g.,
>    `jdk.internal.misc.Unsafe`) and thus make the eventual transition
>    from JDK 9 no less painful than that from JDK 8.  This proposal thus
>    only opens internal packages that existed in JDK 8.
>
>  - This proposal will require adjustments to JEP 260, "Encapsulate Most
>    Internal APIs" [6].  APIs that are internal to the JDK will still be
>    strongly encapsulated from the standpoint of code in modules, whether
>    those modules are automatic or explicit, but they will not appear to
>    be encapsulated at run time from the standpoint of code on the class
>    path.
>
>  - This change will not magically solve every JDK 9 adoption problem.
>    The concrete types of the built-in class loaders are still different,
>    `rt.jar` is still gone, the layout of a system image is still not the
>    same, and the version string still has a new format.
>
>
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html
> [2] http://jdk.java.net/jigsaw/
> [3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
> [4] This will usually but not always be possible, since there are still a
>    few critical internal APIs without exported replacements [6].
> [5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012708.html
> [6] http://openjdk.java.net/jeps/260

Reply | Threaded
Open this post in threaded view
|

Re: Proposal (revised): Allow illegal access to internal APIs by default in JDK 9

Pepper Lebeck-Jobe
Hi, I'm a new poster to this list. I'm actively working on getting Gradle
to work on JDK 9, and I also think this proposal strikes a very good
balance between pragmatism (don't just break people today) and getting
developers motivated to clean up their code for the future.

Thank you,
Pepper Lebeck-Jobe

On Tue, Jun 6, 2017 at 8:53 AM David Blevins <[hidden email]> wrote:

> I think this is the most pragmatic and reasoned middle ground one could
> possibly imagine.
>
> I agree with the finely-tuned choices, specifically:
>
>   - Not going completely silent.  Some asked for the ability to completely
> shut off the warnings.  This goes a little too far to one extreme.  There
> has to be some cost or discomfort or there won’t be the right movement.
> IMHO, it would have been too far back the other way to go silent.  People
> will see these messages, they will post them to StackOverflow and get an
> education on how to go forward.  We need that.
>
>   - Not choosing `--illegal-access=deny` by default for this version.
> Some cited this as a setback.  There is an argument to be made that it
> would have been the bigger setback to go too strict too soon.  The danger
> would be when developers who don’t know this conversation and the details
> behind it learn “use this flag to make problems go away”.  At which point
> it becomes cargo cult and they’ll be afraid to stop using it because they
> never understood it completely in the first place.  Worse, we’re now buried
> under 3 years of lazy StackOverflow answers effectively telling people how
> to go backwards.
>
> So although it feels less strict, one set of choices creates a
> conversation on how to move forward and one creates a conversation on how
> to go backward.
>
> Of course time will tell, but I believe this proposal gets us what we want
> the soonest.
>
>
> -David
>
>
> > On Jun 5, 2017, at 11:45 AM, [hidden email] wrote:
> >
> > (Thanks for all the feedback on the initial proposal [1].  Here's a
> > revised version, which incorporates some of the suggestions received and
> > includes a bit more advice.  An implementation is already available for
> > testing in the Jigsaw EA builds [2].  Further comments welcome!)
> >
> > Over time, as we've gotten closer and closer to the JDK 9 GA date, more
> > and more developers have begun paying attention to the actual changes in
> > this release.  The strong encapsulation of JDK-internal APIs has, in
> > particular, triggered many worried expressions of concern that code that
> > works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
> > warning of this change was given at run time in JDK 8.
> >
> > To help the entire ecosystem migrate to the modular Java platform at a
> > more relaxed pace I hereby propose to allow illegal-access operations to
> > internal APIs from code on the class path by default in JDK 9, and to
> > disallow them in a future release.  This will enable smoother application
> > migration in the near term, yet still enable and motivate the maintainers
> > of libraries and frameworks that use JDK-internal APIs to fix their code
> > to use proper exported APIs.
> >
> > New command-line option: `--illegal-access`
> > -------------------------------------------
> >
> > The recently-introduced `--permit-illegal-access` option [3] will be
> > replaced by a more-general option, `--illegal-access`.  This option takes
> > a single keyword parameter to specify a mode of operation, as follows:
> >
> >  `--illegal-access=permit`
> >
> >    This mode opens each package in each module in the run-time image to
> >    code in all unnamed modules, i.e., code on the class path, if that
> >    package existed in JDK 8.  This enables both static access, i.e., by
> >    compiled bytecode, and deep reflective access, via the platform's
> >    various reflection APIs.
> >
> >    The first reflective-access operation to any such package causes a
> >    warning to be issued, but no warnings are issued after that point.
> >    This single warning describes how to enable further warnings.
> >
> >    This mode will be the default for JDK 9.  It will be removed in a
> >    future release.
> >
> >  `--illegal-access=warn`
> >
> >    This mode is identical to `permit` except that a warning message is
> >    issued for each illegal reflective-access operation.  This is roughly
> >    equivalent to the current `--permit-illegal-access` option.
> >
> >  `--illegal-access=debug`
> >
> >    This mode is identical to `warn` except both a warning message and a
> >    stack trace are issued for each illegal reflective-access operation.
> >    This is roughly equivalent to combining `--permit-illegal-access`
> >    with `-Dsun.reflect.debugModuleAccessChecks`.
> >
> >  `--illegal-access=deny`
> >
> >    This mode disables all illegal-access operations except for those
> >    enabled by other command-line options, e.g., `--add-opens`.
> >
> >    This mode will become the default in a future release.
> >
> > When `deny` becomes the default mode then `permit` will likely remain
> > supported for at least one release, so that developers can continue to
> > migrate their code.  The `permit`, `warn`, and `debug` modes will, over
> > time, be removed, as will the `--illegal-access` option itself.  (For
> > launch-script compatibility the unsupported modes will most likely just
> > be ignored, after issuing a warning to that effect.)
> >
> > How to prepare for the future
> > -----------------------------
> >
> > The default mode, `--illegal-access=permit`, is intended to make you
> > aware when you have code on the class path that reflectively accesses
> > some JDK-internal API at least once.  To learn about all such accesses
> > you can use the `warn` or `debug` modes.  For each library or framework
> > on the class path that requires illegal access you have two options:
> >
> >  - If the component's maintainers have already released a new,
> >    fixed version that no longer uses JDK-internal APIs then you
> >    can consider upgrading to that version.
> >
> >  - If the component still needs to be fixed then we encourage you
> >    to contact its maintainers and ask them to replace their use
> >    of JDK-internal APIs with proper exported APIs [4].
> >
> > If you must continue to use a component that requires illegal access then
> > you can eliminate the warning messages by using one or more `--add-opens`
> > options to open just those internal packages to which access is required.
> >
> > To verify that your application is ready for the future, run it with
> > `--illegal-access=deny` along with any necessary `--add-opens` options.
> > Any remaining illegal-access errors will most likely be due to static
> > references from compiled code to JDK-internal APIs.  You can identify
> > those by running the `jdeps` tool with the `--jdk-internals` option.
> > (JDK 9 does not issue warnings for illegal static-access operations
> > because that would require deep JVM changes and degrade performance.)
> >
> > Warning messages
> > ----------------
> >
> > The warning message issued when an illegal reflective-access operation is
> > detected has the following form:
> >
> >    WARNING: Illegal reflective access by $PERPETRATOR to $VICTIM
> >
> > where:
> >
> >  - $PERPETRATOR is the fully-qualified name of the type containing
> >    the code that invoked the reflective operation in question plus
> >    the code source (i.e., JAR-file path), if available, and
> >
> >  - $VICTIM is a string that describes the member being accessed,
> >    including the fully-qualified name of the enclosing type
> >
> > In JDK 9's default mode, `--illegal-access=permit`, at most one of these
> > warning messages will be issued, accompanied by additional instructive
> > text.  Here is an example, from running Jython on the current Jigsaw EA
> > build [2]:
> >
> >    $ java -jar jython-standalone-2.7.0.jar
> >    WARNING: An illegal reflective access operation has occurred
> >    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper
> (file:/tmp/jython-standalone-2.7.0.jar) to method
> sun.nio.ch.SelChImpl.getFD()
> >    WARNING: Please consider reporting this to the maintainers of
> jnr.posix.JavaLibCHelper
> >    WARNING: Use --illegal-access=warn to enable warnings of further
> illegal reflective access operations
> >    WARNING: All illegal access operations will be denied in a future
> release
> >    Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
> >    [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
> >    Type "help", "copyright", "credits" or "license" for more information.
> >>>> ^D
> >    $
> >
> > If `--illegal-access=warn` is used then only warnings are displayed, with
> > no instructive text.  The run-time system makes a best-effort attempt to
> > suppress duplicate warnings for the same $PERPETRATOR and $VICTIM.  Here
> > is an example, again running Jython:
> >
> >    $ java --illegal-access=warn -jar jython-standalone-2.7.0.jar
> >    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper
> (file:/tmp/jython-standalone-2.7.0.jar) to method
> sun.nio.ch.SelChImpl.getFD()
> >    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper
> (file:/tmp/jython-standalone-2.7.0.jar) to field
> sun.nio.ch.FileChannelImpl.fd
> >    WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper
> (file:/tmp/jython-standalone-2.7.0.jar) to field java.io.FileDescriptor.fd
> >    WARNING: Illegal reflective access by org.python.core.PySystemState
> (file:/tmp/jython-standalone-2.7.0.jar) to method java.io.Console.encoding()
> >    Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
> >    [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
> >    Type "help", "copyright", "credits" or "license" for more information.
> >>>> ^D
> >    $
> >
> > Notes
> > -----
> >
> >  - There is no `--illegal-access` mode that suppresses all warnings.
> >    This is intentional: It ensures that developers know that all
> >    illegal-access operations will be denied by default in a future
> >    release, at which time code that generates warnings today will fail.
> >    Warnings can be suppressed completely via one or more `--add-opens`
> >    options.
> >
> >  - The first proposal [1] opened every package in every explicit module,
> >    rather than just the packages in modules in the run-time image, to
> >    every unnamed module.  Peter Levart pointed out [5] that this could
> >    tempt developers to use internal APIs that are new in JDK 9 (e.g.,
> >    `jdk.internal.misc.Unsafe`) and thus make the eventual transition
> >    from JDK 9 no less painful than that from JDK 8.  This proposal thus
> >    only opens internal packages that existed in JDK 8.
> >
> >  - This proposal will require adjustments to JEP 260, "Encapsulate Most
> >    Internal APIs" [6].  APIs that are internal to the JDK will still be
> >    strongly encapsulated from the standpoint of code in modules, whether
> >    those modules are automatic or explicit, but they will not appear to
> >    be encapsulated at run time from the standpoint of code on the class
> >    path.
> >
> >  - This change will not magically solve every JDK 9 adoption problem.
> >    The concrete types of the built-in class loaders are still different,
> >    `rt.jar` is still gone, the layout of a system image is still not the
> >    same, and the version string still has a new format.
> >
> >
> > [1]
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html
> > [2] http://jdk.java.net/jigsaw/
> > [3]
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
> > [4] This will usually but not always be possible, since there are still a
> >    few critical internal APIs without exported replacements [6].
> > [5]
> http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012708.html
> > [6] http://openjdk.java.net/jeps/260
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Proposal (revised): Allow illegal access to internal APIs by default in JDK 9

Alan Bateman
In reply to this post by mark.reinhold
On 06/06/2017 04:34, Stephen Felts wrote:
> The new format is missing information that was useful in the earlier -Dsun.reflect.debugModuleAccessChecks=true.  It printed something like
> 'java.base does not "opens java.text" to unnamed module ...'  so it was really obvious what option was needed.
>
>
`-Dsun.reflect.debugModuleAccessChecks=true` can still be used to hunt
code that swallows IllegalAccessException or
InaccessibleObjectException, particularly useful when running with
`--illegal-access=deny`.

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: Proposal (revised): Allow illegal access to internal APIs by default in JDK 9

Peter Levart
In reply to this post by mark.reinhold
Hi,

I think the fine-tuning is right to the point now. Allows most of what
was allowed in JDK8, but no more than that.

Regards, Peter

On 06/05/2017 08:45 PM, [hidden email] wrote:

> (Thanks for all the feedback on the initial proposal [1].  Here's a
>   revised version, which incorporates some of the suggestions received and
>   includes a bit more advice.  An implementation is already available for
>   testing in the Jigsaw EA builds [2].  Further comments welcome!)
>
> Over time, as we've gotten closer and closer to the JDK 9 GA date, more
> and more developers have begun paying attention to the actual changes in
> this release.  The strong encapsulation of JDK-internal APIs has, in
> particular, triggered many worried expressions of concern that code that
> works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
> warning of this change was given at run time in JDK 8.
>
> To help the entire ecosystem migrate to the modular Java platform at a
> more relaxed pace I hereby propose to allow illegal-access operations to
> internal APIs from code on the class path by default in JDK 9, and to
> disallow them in a future release.  This will enable smoother application
> migration in the near term, yet still enable and motivate the maintainers
> of libraries and frameworks that use JDK-internal APIs to fix their code
> to use proper exported APIs.
>
> New command-line option: `--illegal-access`
> -------------------------------------------
>
> The recently-introduced `--permit-illegal-access` option [3] will be
> replaced by a more-general option, `--illegal-access`.  This option takes
> a single keyword parameter to specify a mode of operation, as follows:
>
>    `--illegal-access=permit`
>
>      This mode opens each package in each module in the run-time image to
>      code in all unnamed modules, i.e., code on the class path, if that
>      package existed in JDK 8.  This enables both static access, i.e., by
>      compiled bytecode, and deep reflective access, via the platform's
>      various reflection APIs.
>
>      The first reflective-access operation to any such package causes a
>      warning to be issued, but no warnings are issued after that point.
>      This single warning describes how to enable further warnings.
>
>      This mode will be the default for JDK 9.  It will be removed in a
>      future release.
>
>    `--illegal-access=warn`
>
>      This mode is identical to `permit` except that a warning message is
>      issued for each illegal reflective-access operation.  This is roughly
>      equivalent to the current `--permit-illegal-access` option.
>
>    `--illegal-access=debug`
>
>      This mode is identical to `warn` except both a warning message and a
>      stack trace are issued for each illegal reflective-access operation.
>      This is roughly equivalent to combining `--permit-illegal-access`
>      with `-Dsun.reflect.debugModuleAccessChecks`.
>
>    `--illegal-access=deny`
>
>      This mode disables all illegal-access operations except for those
>      enabled by other command-line options, e.g., `--add-opens`.
>
>      This mode will become the default in a future release.
>
> When `deny` becomes the default mode then `permit` will likely remain
> supported for at least one release, so that developers can continue to
> migrate their code.  The `permit`, `warn`, and `debug` modes will, over
> time, be removed, as will the `--illegal-access` option itself.  (For
> launch-script compatibility the unsupported modes will most likely just
> be ignored, after issuing a warning to that effect.)
>
> How to prepare for the future
> -----------------------------
>
> The default mode, `--illegal-access=permit`, is intended to make you
> aware when you have code on the class path that reflectively accesses
> some JDK-internal API at least once.  To learn about all such accesses
> you can use the `warn` or `debug` modes.  For each library or framework
> on the class path that requires illegal access you have two options:
>
>    - If the component's maintainers have already released a new,
>      fixed version that no longer uses JDK-internal APIs then you
>      can consider upgrading to that version.
>
>    - If the component still needs to be fixed then we encourage you
>      to contact its maintainers and ask them to replace their use
>      of JDK-internal APIs with proper exported APIs [4].
>
> If you must continue to use a component that requires illegal access then
> you can eliminate the warning messages by using one or more `--add-opens`
> options to open just those internal packages to which access is required.
>
> To verify that your application is ready for the future, run it with
> `--illegal-access=deny` along with any necessary `--add-opens` options.
> Any remaining illegal-access errors will most likely be due to static
> references from compiled code to JDK-internal APIs.  You can identify
> those by running the `jdeps` tool with the `--jdk-internals` option.
> (JDK 9 does not issue warnings for illegal static-access operations
> because that would require deep JVM changes and degrade performance.)
>
> Warning messages
> ----------------
>
> The warning message issued when an illegal reflective-access operation is
> detected has the following form:
>
>      WARNING: Illegal reflective access by $PERPETRATOR to $VICTIM
>
> where:
>
>    - $PERPETRATOR is the fully-qualified name of the type containing
>      the code that invoked the reflective operation in question plus
>      the code source (i.e., JAR-file path), if available, and
>
>    - $VICTIM is a string that describes the member being accessed,
>      including the fully-qualified name of the enclosing type
>
> In JDK 9's default mode, `--illegal-access=permit`, at most one of these
> warning messages will be issued, accompanied by additional instructive
> text.  Here is an example, from running Jython on the current Jigsaw EA
> build [2]:
>
>      $ java -jar jython-standalone-2.7.0.jar
>      WARNING: An illegal reflective access operation has occurred
>      WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
>      WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper
>      WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
>      WARNING: All illegal access operations will be denied in a future release
>      Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
>      [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
>      Type "help", "copyright", "credits" or "license" for more information.
>      >>> ^D
>      $
>
> If `--illegal-access=warn` is used then only warnings are displayed, with
> no instructive text.  The run-time system makes a best-effort attempt to
> suppress duplicate warnings for the same $PERPETRATOR and $VICTIM.  Here
> is an example, again running Jython:
>
>      $ java --illegal-access=warn -jar jython-standalone-2.7.0.jar
>      WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
>      WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field sun.nio.ch.FileChannelImpl.fd
>      WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field java.io.FileDescriptor.fd
>      WARNING: Illegal reflective access by org.python.core.PySystemState (file:/tmp/jython-standalone-2.7.0.jar) to method java.io.Console.encoding()
>      Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
>      [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
>      Type "help", "copyright", "credits" or "license" for more information.
>      >>> ^D
>      $
>
> Notes
> -----
>
>    - There is no `--illegal-access` mode that suppresses all warnings.
>      This is intentional: It ensures that developers know that all
>      illegal-access operations will be denied by default in a future
>      release, at which time code that generates warnings today will fail.
>      Warnings can be suppressed completely via one or more `--add-opens`
>      options.
>
>    - The first proposal [1] opened every package in every explicit module,
>      rather than just the packages in modules in the run-time image, to
>      every unnamed module.  Peter Levart pointed out [5] that this could
>      tempt developers to use internal APIs that are new in JDK 9 (e.g.,
>      `jdk.internal.misc.Unsafe`) and thus make the eventual transition
>      from JDK 9 no less painful than that from JDK 8.  This proposal thus
>      only opens internal packages that existed in JDK 8.
>
>    - This proposal will require adjustments to JEP 260, "Encapsulate Most
>      Internal APIs" [6].  APIs that are internal to the JDK will still be
>      strongly encapsulated from the standpoint of code in modules, whether
>      those modules are automatic or explicit, but they will not appear to
>      be encapsulated at run time from the standpoint of code on the class
>      path.
>
>    - This change will not magically solve every JDK 9 adoption problem.
>      The concrete types of the built-in class loaders are still different,
>      `rt.jar` is still gone, the layout of a system image is still not the
>      same, and the version string still has a new format.
>
>
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html
> [2] http://jdk.java.net/jigsaw/
> [3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
> [4] This will usually but not always be possible, since there are still a
>      few critical internal APIs without exported replacements [6].
> [5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012708.html
> [6] http://openjdk.java.net/jeps/260