Disallowing the dynamic loading of agents by default (revised)

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Disallowing the dynamic loading of agents by default (revised)

mark.reinhold
Thanks to everyone for the quick feedback on this topic, and especially
to Andrew for the constructive dialogue.

Here's a revised proposal:

  - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
    on by default in JDK 9 but off by default in JDK 10.

    This will allow launch scripts that use this option on JDK 10 to
    work on JDK 9 without change, and will allow early testing of the
    JDK 10 behavior on JDK 9.

  - Revise the `com.sun.tools.attach` API to forbid attachment to the
    current process or to an ancestor of the current process, and
    define a read-only system property that allows such attachment to
    be enabled via the command line.

    This will discourage the inadvertent use of libraries that, for
    better or for worse, intentionally violate strong encapsulation.

  - Enhance the `-jar` launcher option so that if the JAR file being
    launched contains a `Premain-Class` attribute then it's launched
    as both an application and as an agent for that application.

    This will allow `java -jar foo.jar` to be used in place of the
    more verbose `java -javaagent:foo.jar -jar foo.jar` [1].

Taken together, these changes are intended to enable the continued use
of legitimate dynamically-loaded agents without change on JDK 9 and with
a small change on JDK 10.  That later change will align the treatment of
such agents with the other means of breaking encapsulation (`--add-opens`,
etc.) in order to ensure integrity by default for all code.

This proposal does not attempt to lock down platform classes as distinct
from user classes.  Many agents have legitimate reasons to transform
platform classes, so an additional mechanism to protect those classes
does not appear to be worthwhile.

Comments?

- Mark


[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012000.html
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

David Lloyd
On 04/05/2017 11:15 AM, [hidden email] wrote:

> Thanks to everyone for the quick feedback on this topic, and especially
> to Andrew for the constructive dialogue.
>
> Here's a revised proposal:
>
>   - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
>     on by default in JDK 9 but off by default in JDK 10.
>
>     This will allow launch scripts that use this option on JDK 10 to
>     work on JDK 9 without change, and will allow early testing of the
>     JDK 10 behavior on JDK 9.
>
>   - Revise the `com.sun.tools.attach` API to forbid attachment to the
>     current process or to an ancestor of the current process, and
>     define a read-only system property that allows such attachment to
>     be enabled via the command line.

This is just plain weird from a security perspective, to say that
unrelated processes have more privilege to control the current process
than processes that are closely related.

Anyway this is yet another case where arbitrary artificial hurdles are
put in place for the purpose of human behavior modification.  Such
hurdles can always be bypassed, generally resulting in even uglier
situations that the one you're trying to avoid.  In this case I can just
fire a child process and then attach to it from the parent.  Or fire off
two sibling processes and have one attach to the other.  Nothing is
being saved here.

>     This will discourage the inadvertent use of libraries that, for
>     better or for worse, intentionally violate strong encapsulation.
>
>   - Enhance the `-jar` launcher option so that if the JAR file being
>     launched contains a `Premain-Class` attribute then it's launched
>     as both an application and as an agent for that application.
>
>     This will allow `java -jar foo.jar` to be used in place of the
>     more verbose `java -javaagent:foo.jar -jar foo.jar` [1].
>
> Taken together, these changes are intended to enable the continued use
> of legitimate dynamically-loaded agents without change on JDK 9 and with
> a small change on JDK 10.  That later change will align the treatment of
> such agents with the other means of breaking encapsulation (`--add-opens`,
> etc.) in order to ensure integrity by default for all code.
>
> This proposal does not attempt to lock down platform classes as distinct
> from user classes.  Many agents have legitimate reasons to transform
> platform classes, so an additional mechanism to protect those classes
> does not appear to be worthwhile.
>
> Comments?
>
> - Mark
>
>
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012000.html
>

--
- DML
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Michael Rasmussen
In reply to this post by mark.reinhold
On 5 April 2017 at 19:15,  <[hidden email]> wrote:
>   - Enhance the `-jar` launcher option so that if the JAR file being
>     launched contains a `Premain-Class` attribute then it's launched
>     as both an application and as an agent for that application.
>
>     This will allow `java -jar foo.jar` to be used in place of the
>     more verbose `java -javaagent:foo.jar -jar foo.jar` [1].

I would propose the presence of an addition option in the manifest to
enable/control this behavior.

In the case of standalone agents, running them as -jar are sometimes
used for a different purpose than running them as agents.
For instance, in the case of JRebel, java -jar jrebel.jar is used for
license activation/control, and should not activate the agent part.

Kind regards
Michael Rasmussen
JRebel, Product Manager
ZeroTurnaround
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

mark.reinhold
2017/4/5 10:00:13 -0700, [hidden email]:

> On 5 April 2017 at 19:15,  <[hidden email]> wrote:
>> ...
>>
>>  - Enhance the `-jar` launcher option so that if the JAR file being
>>    launched contains a `Premain-Class` attribute then it's launched
>>    as both an application and as an agent for that application.
>>
>>    This will allow `java -jar foo.jar` to be used in place of the
>>    more verbose `java -javaagent:foo.jar -jar foo.jar` [1].
>
> I would propose the presence of an addition option in the manifest to
> enable/control this behavior.
>
> In the case of standalone agents, running them as -jar are sometimes
> used for a different purpose than running them as agents.
> For instance, in the case of JRebel, java -jar jrebel.jar is used for
> license activation/control, and should not activate the agent part.

Interesting.  Perhaps we need a `Self-Premain-Class` attribute, or some
such.

Out of curiosity, would it do any harm in your `java -jar` case if the
agent is activated?

- Mark
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Alan Bateman
In reply to this post by David Lloyd
On 05/04/2017 17:55, David M. Lloyd wrote:

>
> This is just plain weird from a security perspective, to say that
> unrelated processes have more privilege to control the current process
> than processes that are closely related.
>
> Anyway this is yet another case where arbitrary artificial hurdles are
> put in place for the purpose of human behavior modification. Such
> hurdles can always be bypassed, generally resulting in even uglier
> situations that the one you're trying to avoid.  In this case I can
> just fire a child process and then attach to it from the parent.  Or
> fire off two sibling processes and have one attach to the other.  
> Nothing is being saved here.
This thread/proposal is concerned with libraries using APIs intended for
tools to do brain surgery in the current VM. Launching VMs and attaching
to those VMs isn't a concern, no issue with sibling VMs attaching to
each other either either.

-Alan.
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Remi Forax
In reply to this post by mark.reinhold
----- Mail original -----
> De: "mark reinhold" <[hidden email]>
> À: [hidden email]
> Envoyé: Mercredi 5 Avril 2017 18:15:20
> Objet: Disallowing the dynamic loading of agents by default (revised)

> Thanks to everyone for the quick feedback on this topic, and especially
> to Andrew for the constructive dialogue.
>
> Here's a revised proposal:
>
>  - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
>    on by default in JDK 9 but off by default in JDK 10.
>
>    This will allow launch scripts that use this option on JDK 10 to
>    work on JDK 9 without change, and will allow early testing of the
>    JDK 10 behavior on JDK 9.

yes


>
>  - Revise the `com.sun.tools.attach` API to forbid attachment to the
>    current process or to an ancestor of the current process, and
>    define a read-only system property that allows such attachment to
>    be enabled via the command line.
>
>    This will discourage the inadvertent use of libraries that, for
>    better or for worse, intentionally violate strong encapsulation.

don't get this one, as David said, if you span a new VM with an exec, you have more right ??


>
>  - Enhance the `-jar` launcher option so that if the JAR file being
>    launched contains a `Premain-Class` attribute then it's launched
>    as both an application and as an agent for that application.
>
>    This will allow `java -jar foo.jar` to be used in place of the
>    more verbose `java -javaagent:foo.jar -jar foo.jar` [1].


Can be very useful indeed.
(with another name that "Premain-Class" for backward compatibility).

>
> Taken together, these changes are intended to enable the continued use
> of legitimate dynamically-loaded agents without change on JDK 9 and with
> a small change on JDK 10.  That later change will align the treatment of
> such agents with the other means of breaking encapsulation (`--add-opens`,
> etc.) in order to ensure integrity by default for all code.
>
> This proposal does not attempt to lock down platform classes as distinct
> from user classes.  Many agents have legitimate reasons to transform
> platform classes, so an additional mechanism to protect those classes
> does not appear to be worthwhile.
>
> Comments?
>
> - Mark
>
>
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012000.html

Rémi
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Michael Rasmussen
In reply to this post by mark.reinhold
On 6 April 2017 at 02:34,  <[hidden email]> wrote:
> Interesting.  Perhaps we need a `Self-Premain-Class` attribute, or some
> such.
>
> Out of curiosity, would it do any harm in your `java -jar` case if the
> agent is activated?

As it is right now, yes, that would cause the execution to potentially fail.
Should a Self-Premain-Class (or like) not be implements, we would have
to check in the beginning of premain if we are running in java -jar
mode, and if such, make premain a no-op.

/Michael
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Andrew Dinn
In reply to this post by mark.reinhold
On 05/04/17 17:15, [hidden email] wrote:

> Thanks to everyone for the quick feedback on this topic, and especially
> to Andrew for the constructive dialogue.
>
> Here's a revised proposal:
>
>   - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
>     on by default in JDK 9 but off by default in JDK 10.
>
>     This will allow launch scripts that use this option on JDK 10 to
>     work on JDK 9 without change, and will allow early testing of the
>     JDK 10 behavior on JDK 9.

This at the least is very welcome. It will give Red Hat and others the
time needed to prepare their users for for this change.

>   - Revise the `com.sun.tools.attach` API to forbid attachment to the
>     current process or to an ancestor of the current process, and
>     define a read-only system property that allows such attachment to
>     be enabled via the command line.
>
>     This will discourage the inadvertent use of libraries that, for
>     better or for worse, intentionally violate strong encapsulation.

I guess I'm agnostic to this. I'm certainly not against it.

Is this change being proposed for JDK9 or JDK10? I'd prefer the latter
as that would give me more time to educate those using the Byteman
BMUnit package about the need to configure the requisite system
property. However, it's less of an issue than the
-XX:+EnableDynamicAgentLoading flag since, once configured, the system
property does no harm when switching from JDK9 back to JDK8.

>   - Enhance the `-jar` launcher option so that if the JAR file being
>     launched contains a `Premain-Class` attribute then it's launched
>     as both an application and as an agent for that application.
>
>     This will allow `java -jar foo.jar` to be used in place of the
>     more verbose `java -javaagent:foo.jar -jar foo.jar` [1].

I have no comments to offer on this as it is not a scenario I have had
to deal with.

> Taken together, these changes are intended to enable the continued use
> of legitimate dynamically-loaded agents without change on JDK 9 and with
> a small change on JDK 10.  That later change will align the treatment of
> such agents with the other means of breaking encapsulation (`--add-opens`,
> etc.) in order to ensure integrity by default for all code.

That's a compromise position I can live with. Much as I want to retain
the ability to load my agent dynamically I also acknowledge that this is
one side of a trade-off. In particular, this proposal incorporates the
need for the JVM /eventually/ to have -- as a default -- a guarantee
that some portion of the code base cannot be subject to change and hence
is available for optimization without the need for ever more complex
apparatus to ensure later de-optimization. That's something I understand
the significance of and regard as very important for the future of the
JVM and JVM-based languages (not just Java).

This probably means that at some point (maybe JDK10 time, maybe later,
depending upon when this might start to deliver performance
improvements) Red Hat will decide to advise most of their Java customers
to run without the option to load dynamic agents. That may lose our
support team some opportunities to diagnose certain types of problems
but may also enable those customers to gain from performance improvements.

> This proposal does not attempt to lock down platform classes as distinct
> from user classes.  Many agents have legitimate reasons to transform
> platform classes, so an additional mechanism to protect those classes
> does not appear to be worthwhile.

I'm happy with this although it might be worth reviewing it later.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

David Lloyd
In reply to this post by Alan Bateman
On 04/06/2017 02:56 AM, Alan Bateman wrote:

> On 05/04/2017 17:55, David M. Lloyd wrote:
>
>>
>> This is just plain weird from a security perspective, to say that
>> unrelated processes have more privilege to control the current process
>> than processes that are closely related.
>>
>> Anyway this is yet another case where arbitrary artificial hurdles are
>> put in place for the purpose of human behavior modification. Such
>> hurdles can always be bypassed, generally resulting in even uglier
>> situations that the one you're trying to avoid.  In this case I can
>> just fire a child process and then attach to it from the parent.  Or
>> fire off two sibling processes and have one attach to the other.
>> Nothing is being saved here.
> This thread/proposal is concerned with libraries using APIs intended for
> tools to do brain surgery in the current VM.

I know, I'm giving examples of how such a library could circumvent this
restriction.  Another example is to start a child process and a
grandchild process, and then have the child process exit.

Adding technical hurdles to send a social message is frankly pretty
lousy engineering.  It never fails to backfire.

--
- DML
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Andrew Dinn
On 06/04/17 13:56, David M. Lloyd wrote:

> On 04/06/2017 02:56 AM, Alan Bateman wrote:
>> On 05/04/2017 17:55, David M. Lloyd wrote:
>>
>>>
>>> This is just plain weird from a security perspective, to say that
>>> unrelated processes have more privilege to control the current process
>>> than processes that are closely related.
>>>
>>> Anyway this is yet another case where arbitrary artificial hurdles are
>>> put in place for the purpose of human behavior modification. Such
>>> hurdles can always be bypassed, generally resulting in even uglier
>>> situations that the one you're trying to avoid.  In this case I can
>>> just fire a child process and then attach to it from the parent.  Or
>>> fire off two sibling processes and have one attach to the other.
>>> Nothing is being saved here.
>> This thread/proposal is concerned with libraries using APIs intended for
>> tools to do brain surgery in the current VM.
>
> I know, I'm giving examples of how such a library could circumvent this
> restriction.  Another example is to start a child process and a
> grandchild process, and then have the child process exit.
>
> Adding technical hurdles to send a social message is frankly pretty
> lousy engineering.  It never fails to backfire.

This proposal comes with an even simpler way to 'circumvent this
restriction' i.e. set the requisite property on the command line.

I think the point is not really to stop rogue code from hoisting an
agent by such indirect means (Mark Reinhold agreed that if such rogue
code is in the runtime you have already handed over the keys to the
castle). It is to make normal code explicitly acknowledge what it is
doing (more precisely, to make those deploying the code be more explicit
about their intention to do so).

That's not such a terrible thing to transition to in the longer term,
assuming we are given time to manage the transition.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Alan Bateman
In reply to this post by David Lloyd
On 06/04/2017 13:56, David M. Lloyd wrote:

>
> I know, I'm giving examples of how such a library could circumvent
> this restriction.  Another example is to start a child process and a
> grandchild process, and then have the child process exit.
The examples in your first mail aren't a problem. Yes, the example in
this mail of an intermediate process terminating and being re-parented
to the init process will workaround any check that uses the process tree
but I don't think we should be overly distracted by wild-ass code doing
that. The purpose is simply to make it inconvenient for general purpose
libraries to get at these APIs for the purposes of doing brain surgery
in the current VM.

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Alasdair Nottingham
In reply to this post by mark.reinhold
Mark,

I much prefer this proposal and it covers my use case which is fantastic Some comments below:

> On Apr 5, 2017, at 12:15 PM, [hidden email] wrote:
>
> Thanks to everyone for the quick feedback on this topic, and especially
> to Andrew for the constructive dialogue.
>
> Here's a revised proposal:
>
>  - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
>    on by default in JDK 9 but off by default in JDK 10.
>
>    This will allow launch scripts that use this option on JDK 10 to
>    work on JDK 9 without change, and will allow early testing of the
>    JDK 10 behavior on JDK 9.

I think giving more time to react to the change is good, but I think this just
provides more notice that dynamic attach will go away, it doesn’t ultimately
provide a solution for the problems that are currently solved using dynamic
attach of agents.

>
>  - Revise the `com.sun.tools.attach` API to forbid attachment to the
>    current process or to an ancestor of the current process, and
>    define a read-only system property that allows such attachment to
>    be enabled via the command line.
>
>    This will discourage the inadvertent use of libraries that, for
>    better or for worse, intentionally violate strong encapsulation.

I think just preventing self-attach would be enough. I don’t think you need
to worry about the hierarchy. If you are going to the lengths of launching new
JVM’s to attach the agent I’m pretty sure that you have found out that you
cannot self attach, and are therefore already in the camp of knowing you are
doing something bad.

>
>  - Enhance the `-jar` launcher option so that if the JAR file being
>    launched contains a `Premain-Class` attribute then it's launched
>    as both an application and as an agent for that application.
>
>    This will allow `java -jar foo.jar` to be used in place of the
>    more verbose `java -javaagent:foo.jar -jar foo.jar` [1].
>

I like this idea, it solves my problem in a much simpler way than having to
self attach. I don’t mind what the header is called, so if Self-Premain-Class is
used I can cope with that. I’m assuming this will continue to work in Java SE 10
and only dynamic attach will be disabled in Java SE 10.

> Taken together, these changes are intended to enable the continued use
> of legitimate dynamically-loaded agents without change on JDK 9 and with
> a small change on JDK 10.  That later change will align the treatment of
> such agents with the other means of breaking encapsulation (`--add-opens`,
> etc.) in order to ensure integrity by default for all code.
>
> This proposal does not attempt to lock down platform classes as distinct
> from user classes.  Many agents have legitimate reasons to transform
> platform classes, so an additional mechanism to protect those classes
> does not appear to be worthwhile.
>
> Comments?
>
> - Mark
>
>
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012000.html

Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Andrew Dinn
On 06/04/17 18:48, Alasdair Nottingham wrote:
> I think giving more time to react to the change is good, but I think this just
> provides more notice that dynamic attach will go away, it doesn’t ultimately
> provide a solution for the problems that are currently solved using dynamic
> attach of agents.

I agree that it provides no such solution. However, I also agree with
John Rose's observation (which may have sailed by you unawares in the
wide ocean this thread has come to occupy) that in the long term there
probably is no such solution.

Eventually, we (yes, Red Hat hope to be involved in this work as much as
Oracle and, no doubt, the other OpenJDK contributors) -- we hope to make
the JVM smart enough to profit from certain guarantees of invariance in
the code base and Jigsaw is one move towards that goal. At some point
that is going inevitably going to mean choosing between the flexibility
offered by transformations effected by dynamically loaded agents or the
performance benefits offered by hard restrictions on such a flexible
runtime. Java -- specifically the JVM -- does an amazing job of coping
with dynamic changes to the code base (not just agents but also lazy
loading and other dynamic reconfigurations to accommodate dynamic code
usage) and that is a great part of its magic. However, Arthur C Clarke
got it right -- to those in the know the magic is actually just a very
advanced technology. Advancing it further is bound to require trade-offs
and compromises where you can't have one without losing the other
(whatever Frank Sinatra says).

> I think just preventing self-attach would be enough. I don’t think you need
> to worry about the hierarchy. If you are going to the lengths of launching new
> JVM’s to attach the agent I’m pretty sure that you have found out that you
> cannot self attach, and are therefore already in the camp of knowing you are
> doing something bad.

This is a fair point.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Martijn Verburg
In reply to this post by mark.reinhold
Hi all,

I've come back into this thread late, but for us (jClarity) we're OK with
the revised proposal, it will give us the time with slow moving enterprise
customers to move to an explicit Java agent approach.

Cheers,
Martijn

On 7 April 2017 at 13:53, Stephen Felts <[hidden email]> wrote:

> I have a problem with the second point.
>
> Oracle's application server has "FAST SWAP" functionality similar to
> JRebel built-in so that a developer can speed up the "edit -> build ->
> deploy -> test" cycle.
> It uses an internal agent that attaches to the server so customers don't
> need to see it, configure it, or even know the name of it.
> It only works on application classes in an exploded ear or war file.
> This proposal will make it so the JRebel agent will work but our own agent
> won't.
>
> Ironically, this feature only works in development mode and developers are
> the ones most likely to run without any scripts having the CLASSPATH set up
> and just run java weblogic.Server.
> So as I've said many times I don't want any JDK9 command line options.
>
> In trying to protect against the guilty, you are killing the innocent.
>
> It seems to me that it's late in Phase 2 of ramp-down of JDK 9 and this
> feature is not baked or well received.  IMO no changes should be done in
> this area for JDK 9.
>
>
> -----Original Message-----
> From: jigsaw-dev [mailto:[hidden email]] On Behalf
> Of [hidden email]
> Sent: Wednesday, April 5, 2017 12:15 PM
> To: [hidden email]
> Subject: Disallowing the dynamic loading of agents by default (revised)
>
> Thanks to everyone for the quick feedback on this topic, and especially to
> Andrew for the constructive dialogue.
>
> Here's a revised proposal:
>
>   - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
>     on by default in JDK 9 but off by default in JDK 10.
>
>     This will allow launch scripts that use this option on JDK 10 to
>     work on JDK 9 without change, and will allow early testing of the
>     JDK 10 behavior on JDK 9.
>
>   - Revise the `com.sun.tools.attach` API to forbid attachment to the
>     current process or to an ancestor of the current process, and
>     define a read-only system property that allows such attachment to
>     be enabled via the command line.
>
>     This will discourage the inadvertent use of libraries that, for
>     better or for worse, intentionally violate strong encapsulation.
>
>   - Enhance the `-jar` launcher option so that if the JAR file being
>     launched contains a `Premain-Class` attribute then it's launched
>     as both an application and as an agent for that application.
>
>     This will allow `java -jar foo.jar` to be used in place of the
>     more verbose `java -javaagent:foo.jar -jar foo.jar` [1].
>
> Taken together, these changes are intended to enable the continued use of
> legitimate dynamically-loaded agents without change on JDK 9 and with a
> small change on JDK 10.  That later change will align the treatment of such
> agents with the other means of breaking encapsulation (`--add-opens`,
> etc.) in order to ensure integrity by default for all code.
>
> This proposal does not attempt to lock down platform classes as distinct
> from user classes.  Many agents have legitimate reasons to transform
> platform classes, so an additional mechanism to protect those classes does
> not appear to be worthwhile.
>
> Comments?
>
> - Mark
>
>
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-
> April/012000.html
>
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

mark.reinhold
In reply to this post by Michael Rasmussen
2017/4/6 2:07:27 -0700, [hidden email]:

> On 6 April 2017 at 02:34,  <[hidden email]> wrote:
>> Interesting.  Perhaps we need a `Self-Premain-Class` attribute, or some
>> such.
>>
>> Out of curiosity, would it do any harm in your `java -jar` case if the
>> agent is activated?
>
> As it is right now, yes, that would cause the execution to potentially fail.
> Should a Self-Premain-Class (or like) not be implements, we would have
> to check in the beginning of premain if we are running in java -jar
> mode, and if such, make premain a no-op.

Okay, so how about this revised proposal:

  - Define a new JAR-file manifest attribute, `Launcher-Agent-Class`.
    If this attribute is present in the manifest of an executable JAR
    file then the `agentmain` method of the specified class will be
    invoked prior to invoking the `main` method of the application.
    The JAR file will, effectively, be loaded also as a dynamic agent.

    This will allow `java -jar foo.jar` to be used in place of the more
    verbose `java -javaagent:foo.jar -jar foo.jar`.

- Mark
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

mark.reinhold
In reply to this post by Andrew Dinn
2017/4/6 3:09:49 -0700, Andrew Dinn <[hidden email]>:

> On 05/04/17 17:15, [hidden email] wrote:
>> Thanks to everyone for the quick feedback on this topic, and especially
>> to Andrew for the constructive dialogue.
>>
>> Here's a revised proposal:
>>
>>  - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
>>    on by default in JDK 9 but off by default in JDK 10.
>>
>>    This will allow launch scripts that use this option on JDK 10 to
>>    work on JDK 9 without change, and will allow early testing of the
>>    JDK 10 behavior on JDK 9.
>
> This at the least is very welcome. It will give Red Hat and others the
> time needed to prepare their users for for this change.
>
>>  - Revise the `com.sun.tools.attach` API to forbid attachment to the
>>    current process or to an ancestor of the current process, and
>>    define a read-only system property that allows such attachment to
>>    be enabled via the command line.
>>
>>    This will discourage the inadvertent use of libraries that, for
>>    better or for worse, intentionally violate strong encapsulation.
>
> I guess I'm agnostic to this. I'm certainly not against it.
>
> Is this change being proposed for JDK9 or JDK10?

JDK 9.  With `-XX:+EnableDynamicAgentLoading` off by default in JDK 9,
this is one of the few ways we have left to discourage the use of
encapsulation-busting libraries.

>> ...
>>
>> Taken together, these changes are intended to enable the continued use
>> of legitimate dynamically-loaded agents without change on JDK 9 and with
>> a small change on JDK 10.  That later change will align the treatment of
>> such agents with the other means of breaking encapsulation (`--add-opens`,
>> etc.) in order to ensure integrity by default for all code.
>
> That's a compromise position I can live with. Much as I want to retain
> the ability to load my agent dynamically I also acknowledge that this is
> one side of a trade-off. In particular, this proposal incorporates the
> need for the JVM /eventually/ to have -- as a default -- a guarantee
> that some portion of the code base cannot be subject to change and hence
> is available for optimization without the need for ever more complex
> apparatus to ensure later de-optimization. That's something I understand
> the significance of and regard as very important for the future of the
> JVM and JVM-based languages (not just Java).

Indeed!  That's a good summary of the tradeoff.

> ...
>
>> This proposal does not attempt to lock down platform classes as distinct
>> from user classes.  Many agents have legitimate reasons to transform
>> platform classes, so an additional mechanism to protect those classes
>> does not appear to be worthwhile.
>
> I'm happy with this although it might be worth reviewing it later.

Yes -- it wouldn't surprise me if we come back to this as we start to
leverage integrity invariants for optimization.

- Mark
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

mark.reinhold
In reply to this post by Alasdair Nottingham
2017/4/6 10:48:12 -0700, [hidden email]:

> On Apr 5, 2017, at 12:15 PM, [hidden email] wrote:
>>
>> Thanks to everyone for the quick feedback on this topic, and especially
>> to Andrew for the constructive dialogue.
>>
>> Here's a revised proposal:
>>
>> - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
>>   on by default in JDK 9 but off by default in JDK 10.
>>
>>   This will allow launch scripts that use this option on JDK 10 to
>>   work on JDK 9 without change, and will allow early testing of the
>>   JDK 10 behavior on JDK 9.
>
> I think giving more time to react to the change is good, but I think this just
> provides more notice that dynamic attach will go away, it doesn’t ultimately
> provide a solution for the problems that are currently solved using dynamic
> attach of agents.

As Andrew Dinn writes nearby there really isn't a solution here, per se,
there's just a tradeoff between flexibility and performance.

>> - Revise the `com.sun.tools.attach` API to forbid attachment to the
>>   current process or to an ancestor of the current process, and
>>   define a read-only system property that allows such attachment to
>>   be enabled via the command line.
>>
>>   This will discourage the inadvertent use of libraries that, for
>>   better or for worse, intentionally violate strong encapsulation.
>
> I think just preventing self-attach would be enough. I don’t think you need
> to worry about the hierarchy. If you are going to the lengths of launching new
> JVM’s to attach the agent I’m pretty sure that you have found out that you
> cannot self attach, and are therefore already in the camp of knowing you are
> doing something bad.

Yes, I suspect you're right.  We'll just forbid self-attachment for now.

>> - Enhance the `-jar` launcher option so that if the JAR file being
>>   launched contains a `Premain-Class` attribute then it's launched
>>   as both an application and as an agent for that application.
>>
>>   This will allow `java -jar foo.jar` to be used in place of the
>>   more verbose `java -javaagent:foo.jar -jar foo.jar` [1].
>>
>
> I like this idea, it solves my problem in a much simpler way than having to
> self attach. I don’t mind what the header is called, so if Self-Premain-Class is
> used I can cope with that.

Please see my nearby reply to Michael Rasumussen -- the new attribute
will be named `Launcher-Agent-Class`.

>                            I’m assuming this will continue to work in Java SE 10
> and only dynamic attach will be disabled in Java SE 10.

`Launcher-Agent-Class` is a form of dynamic agent loading but it's
well-scoped and not so easy to use inadvertently, so I suspect it will
do no harm to leave it enabled even when dynamic loading via attachment
is disabled by default.

- Mark
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Michael Rasmussen
In reply to this post by mark.reinhold
On 10 April 2017 at 00:04,  <[hidden email]> wrote:

> Okay, so how about this revised proposal:
>
>   - Define a new JAR-file manifest attribute, `Launcher-Agent-Class`.
>     If this attribute is present in the manifest of an executable JAR
>     file then the `agentmain` method of the specified class will be
>     invoked prior to invoking the `main` method of the application.
>     The JAR file will, effectively, be loaded also as a dynamic agent.
>
>     This will allow `java -jar foo.jar` to be used in place of the more
>     verbose `java -javaagent:foo.jar -jar foo.jar`.
>
> - Mark

I think that is a better solution, also more backwards compatible, as
it won't cause agents to unexpectedly start from existing jar files
(as others also have pointed out in this thread).

/Michael
Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Gregg Wonderly
In reply to this post by mark.reinhold
I really believe that we can't really flip off working applications with a JDK upgrade.  It's developer and deployed cooperation no matter which way the switch are by default.  Starting with unknowing developers, deployed and users having to suffer through discovery of what will it take to make their software work again just seems like an exercise in anarchy rather than helping the community gain better opportunities out of a more flexible JRE/JDK platform.

This really is something that will negatively impact a much large body of ignorant users, who are not engaged here, than we can imagine.  

Gregg

Sent from my iPhone

> On Apr 7, 2017, at 8:20 AM, Martijn Verburg <[hidden email]> wrote:
>
> Hi all,
>
> I've come back into this thread late, but for us (jClarity) we're OK with
> the revised proposal, it will give us the time with slow moving enterprise
> customers to move to an explicit Java agent approach.
>
> Cheers,
> Martijn
>
>> On 7 April 2017 at 13:53, Stephen Felts <[hidden email]> wrote:
>>
>> I have a problem with the second point.
>>
>> Oracle's application server has "FAST SWAP" functionality similar to
>> JRebel built-in so that a developer can speed up the "edit -> build ->
>> deploy -> test" cycle.
>> It uses an internal agent that attaches to the server so customers don't
>> need to see it, configure it, or even know the name of it.
>> It only works on application classes in an exploded ear or war file.
>> This proposal will make it so the JRebel agent will work but our own agent
>> won't.
>>
>> Ironically, this feature only works in development mode and developers are
>> the ones most likely to run without any scripts having the CLASSPATH set up
>> and just run java weblogic.Server.
>> So as I've said many times I don't want any JDK9 command line options.
>>
>> In trying to protect against the guilty, you are killing the innocent.
>>
>> It seems to me that it's late in Phase 2 of ramp-down of JDK 9 and this
>> feature is not baked or well received.  IMO no changes should be done in
>> this area for JDK 9.
>>
>>
>> -----Original Message-----
>> From: jigsaw-dev [mailto:[hidden email]] On Behalf
>> Of [hidden email]
>> Sent: Wednesday, April 5, 2017 12:15 PM
>> To: [hidden email]
>> Subject: Disallowing the dynamic loading of agents by default (revised)
>>
>> Thanks to everyone for the quick feedback on this topic, and especially to
>> Andrew for the constructive dialogue.
>>
>> Here's a revised proposal:
>>
>>  - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
>>    on by default in JDK 9 but off by default in JDK 10.
>>
>>    This will allow launch scripts that use this option on JDK 10 to
>>    work on JDK 9 without change, and will allow early testing of the
>>    JDK 10 behavior on JDK 9.
>>
>>  - Revise the `com.sun.tools.attach` API to forbid attachment to the
>>    current process or to an ancestor of the current process, and
>>    define a read-only system property that allows such attachment to
>>    be enabled via the command line.
>>
>>    This will discourage the inadvertent use of libraries that, for
>>    better or for worse, intentionally violate strong encapsulation.
>>
>>  - Enhance the `-jar` launcher option so that if the JAR file being
>>    launched contains a `Premain-Class` attribute then it's launched
>>    as both an application and as an agent for that application.
>>
>>    This will allow `java -jar foo.jar` to be used in place of the
>>    more verbose `java -javaagent:foo.jar -jar foo.jar` [1].
>>
>> Taken together, these changes are intended to enable the continued use of
>> legitimate dynamically-loaded agents without change on JDK 9 and with a
>> small change on JDK 10.  That later change will align the treatment of such
>> agents with the other means of breaking encapsulation (`--add-opens`,
>> etc.) in order to ensure integrity by default for all code.
>>
>> This proposal does not attempt to lock down platform classes as distinct
>> from user classes.  Many agents have legitimate reasons to transform
>> platform classes, so an additional mechanism to protect those classes does
>> not appear to be worthwhile.
>>
>> Comments?
>>
>> - Mark
>>
>>
>> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-
>> April/012000.html
>>

Reply | Threaded
Open this post in threaded view
|

Re: Disallowing the dynamic loading of agents by default (revised)

Alan Bateman
In reply to this post by mark.reinhold
The Jigsaw EA builds [1] have been refreshed to include some of the
pieces in this proposal, specifically:

- If an executable JAR contains a java agent then it will be started by
`java -jar` when the main manifest has the `Launcher-Agent-Class`
attribute. The entry point that is invoked is the `agentmain` method. In
the HotSpot implementation then all Can-XXX attributes can be used. The
Boot-Class-Path attribute can be used too, say for cases where the
executable JAR brings a helper JAR with classes that need to be visible
to the boot loader (when instrumenting classes in modules defined to the
boot loader for example).

The `Launcher-Agent-Class` attribute will be ignored when on small
run-time image that doesn't include the `java.instrument` module.

One other point to mention is that the proposal (and implementation)
does not include any built-in support for providing options to the
agent. If configuration is needed then one simple approach is to include
a resource file in the JAR file with the options and have the agent read
that with Class.getResourceXXX.

- The attach API disallows attaching to the current process. For now,
the "jdk.attach.allowAttachSelf" system property can be used for cases
where the tool and target VM are the same process. The system property
is set on the command line with `-Djdk.attach.allowAttachSelf` or
`-Djdk.attach.allowAttachSelf=true`. It will be ignored on JDK 8 and older.

The builds don't have the Serguei Spitsyn's patch for
EnableDynamicAgentLoading yet. We're re-align that with the revised
proposal so that there are builds available to try out in the coming days.

-Alan

[1] https://jdk9.java.net/jigsaw/

12