Clarification needed for the amount of memory unmapped during imagefile closure.

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Clarification needed for the amount of memory unmapped during imagefile closure.

Jini George
Hello!

I needed a clarification regarding the amount of memory unmapped during
imagefile closure in src/java.base/share/native/libjimage/imageFile.cpp.

I noticed that when the "modules" file is opened in
ImageFileReader::open(), and the contents are mmap()-ed, the size to be
mmap()-ed is derived from map_size().

399     // Memory map image (minimally the index.)
400     _index_data = (u1*)osSupport::map_memory(_fd, _name, 0,
(size_t)map_size());

Which could be _file_size or _index_size, and for 64 bit processes, it
would be _file_size. (about 140 MB)

488     // Retrieve the size of the mapped image.
489     inline u8 map_size() const {
490         return (u8)(memory_map_image ? _file_size : _index_size);
491     }

But when the contents are unmapped in ImageFileReader::close(), the
amount of memory unmapped is only _index_size (which is considerably
lesser than _file_size).

427 // Close image file.
428 void ImageFileReader::close() {
429     // Deallocate the index.
430     if (_index_data) {
431         osSupport::unmap_memory((char*)_index_data, _index_size);
432         _index_data = NULL;
433     }

Wanted to check if this is an oversight, or if there is a reason behind
this and I am missing something. Shouldn't the amount of memory unmapped
be map_size() too ?

Any help would be appreciated !

Thanks in advance,
Jini.
Reply | Threaded
Open this post in threaded view
|

Re: Clarification needed for the amount of memory unmapped during imagefile closure.

Alan Bateman
On 05/12/2018 12:26, Jini George wrote:

> Hello!
>
> I needed a clarification regarding the amount of memory unmapped
> during imagefile closure in
> src/java.base/share/native/libjimage/imageFile.cpp.
>
> I noticed that when the "modules" file is opened in
> ImageFileReader::open(), and the contents are mmap()-ed, the size to
> be mmap()-ed is derived from map_size().
>
> 399     // Memory map image (minimally the index.)
> 400     _index_data = (u1*)osSupport::map_memory(_fd, _name, 0,
> (size_t)map_size());
>
> Which could be _file_size or _index_size, and for 64 bit processes, it
> would be _file_size. (about 140 MB)
>
> 488     // Retrieve the size of the mapped image.
> 489     inline u8 map_size() const {
> 490         return (u8)(memory_map_image ? _file_size : _index_size);
> 491     }
>
> But when the contents are unmapped in ImageFileReader::close(), the
> amount of memory unmapped is only _index_size (which is considerably
> lesser than _file_size).
>
> 427 // Close image file.
> 428 void ImageFileReader::close() {
> 429     // Deallocate the index.
> 430     if (_index_data) {
> 431         osSupport::unmap_memory((char*)_index_data, _index_size);
> 432         _index_data = NULL;
> 433     }
>
> Wanted to check if this is an oversight, or if there is a reason
> behind this and I am missing something. Shouldn't the amount of memory
> unmapped be map_size() too ?
It doesn't look right but needs closer examination. However, I'm curious
how you are running into it as it will be completely unmapped when the
VM terminates. Is this a tool or test that runs "in process"?

-Alan
Reply | Threaded
Open this post in threaded view
|

Re: Clarification needed for the amount of memory unmapped during imagefile closure.

Jim Laskey (Oracle)
An oversight, please file a bug.

— Jim


> On Dec 5, 2018, at 8:37 AM, Alan Bateman <[hidden email]> wrote:
>
> On 05/12/2018 12:26, Jini George wrote:
>> Hello!
>>
>> I needed a clarification regarding the amount of memory unmapped during imagefile closure in src/java.base/share/native/libjimage/imageFile.cpp.
>>
>> I noticed that when the "modules" file is opened in ImageFileReader::open(), and the contents are mmap()-ed, the size to be mmap()-ed is derived from map_size().
>>
>> 399     // Memory map image (minimally the index.)
>> 400     _index_data = (u1*)osSupport::map_memory(_fd, _name, 0, (size_t)map_size());
>>
>> Which could be _file_size or _index_size, and for 64 bit processes, it would be _file_size. (about 140 MB)
>>
>> 488     // Retrieve the size of the mapped image.
>> 489     inline u8 map_size() const {
>> 490         return (u8)(memory_map_image ? _file_size : _index_size);
>> 491     }
>>
>> But when the contents are unmapped in ImageFileReader::close(), the amount of memory unmapped is only _index_size (which is considerably lesser than _file_size).
>>
>> 427 // Close image file.
>> 428 void ImageFileReader::close() {
>> 429     // Deallocate the index.
>> 430     if (_index_data) {
>> 431         osSupport::unmap_memory((char*)_index_data, _index_size);
>> 432         _index_data = NULL;
>> 433     }
>>
>> Wanted to check if this is an oversight, or if there is a reason behind this and I am missing something. Shouldn't the amount of memory unmapped be map_size() too ?
> It doesn't look right but needs closer examination. However, I'm curious how you are running into it as it will be completely unmapped when the VM terminates. Is this a tool or test that runs "in process"?
>
> -Alan

Reply | Threaded
Open this post in threaded view
|

Re: Clarification needed for the amount of memory unmapped during imagefile closure.

Jini George
In reply to this post by Alan Bateman
Thank you very much for the response, Alan. I have been working on
https://bugs.openjdk.java.net/browse/JDK-8200613, which is wrt the
shared strings CDS regions not getting dumped onto the process corefiles
on Linux. To fix this, I am modifying the process coredump_filter file
to enable file-backed private mappings of the process to also get dumped
into the corefiles. This causes the mmap()-ed image file to also get
dumped into the corefile -- and we are trying to avoid this by calling
ImageFileReader::close() before the corefile gets dumped, so that we can
avoid the additional 140MB in the corefile.

My query was prompted by the fact that inspite of calling
ImageFileReader::close() before getting the corefile dumped, there was
not much of a difference in the corefile size.

Thanks,
Jini.


On 12/5/2018 6:07 PM, Alan Bateman wrote:

> On 05/12/2018 12:26, Jini George wrote:
>> Hello!
>>
>> I needed a clarification regarding the amount of memory unmapped
>> during imagefile closure in
>> src/java.base/share/native/libjimage/imageFile.cpp.
>>
>> I noticed that when the "modules" file is opened in
>> ImageFileReader::open(), and the contents are mmap()-ed, the size to
>> be mmap()-ed is derived from map_size().
>>
>> 399     // Memory map image (minimally the index.)
>> 400     _index_data = (u1*)osSupport::map_memory(_fd, _name, 0,
>> (size_t)map_size());
>>
>> Which could be _file_size or _index_size, and for 64 bit processes, it
>> would be _file_size. (about 140 MB)
>>
>> 488     // Retrieve the size of the mapped image.
>> 489     inline u8 map_size() const {
>> 490         return (u8)(memory_map_image ? _file_size : _index_size);
>> 491     }
>>
>> But when the contents are unmapped in ImageFileReader::close(), the
>> amount of memory unmapped is only _index_size (which is considerably
>> lesser than _file_size).
>>
>> 427 // Close image file.
>> 428 void ImageFileReader::close() {
>> 429     // Deallocate the index.
>> 430     if (_index_data) {
>> 431         osSupport::unmap_memory((char*)_index_data, _index_size);
>> 432         _index_data = NULL;
>> 433     }
>>
>> Wanted to check if this is an oversight, or if there is a reason
>> behind this and I am missing something. Shouldn't the amount of memory
>> unmapped be map_size() too ?
> It doesn't look right but needs closer examination. However, I'm curious
> how you are running into it as it will be completely unmapped when the
> VM terminates. Is this a tool or test that runs "in process"?
>
> -Alan
Reply | Threaded
Open this post in threaded view
|

Re: Clarification needed for the amount of memory unmapped during imagefile closure.

Jini George
In reply to this post by Jim Laskey (Oracle)
Thank you very much, Jim. Will do and submit a patch for review.

Thanks,
Jini.

On 12/5/2018 6:58 PM, Jim Laskey wrote:

> An oversight, please file a bug.
>
> — Jim
>
>
>> On Dec 5, 2018, at 8:37 AM, Alan Bateman <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>> On 05/12/2018 12:26, Jini George wrote:
>>> Hello!
>>>
>>> I needed a clarification regarding the amount of memory unmapped
>>> during imagefile closure in
>>> src/java.base/share/native/libjimage/imageFile.cpp.
>>>
>>> I noticed that when the "modules" file is opened in
>>> ImageFileReader::open(), and the contents are mmap()-ed, the size to
>>> be mmap()-ed is derived from map_size().
>>>
>>> 399     // Memory map image (minimally the index.)
>>> 400     _index_data = (u1*)osSupport::map_memory(_fd, _name, 0,
>>> (size_t)map_size());
>>>
>>> Which could be _file_size or _index_size, and for 64 bit processes,
>>> it would be _file_size. (about 140 MB)
>>>
>>> 488     // Retrieve the size of the mapped image.
>>> 489     inline u8 map_size() const {
>>> 490         return (u8)(memory_map_image ? _file_size : _index_size);
>>> 491     }
>>>
>>> But when the contents are unmapped in ImageFileReader::close(), the
>>> amount of memory unmapped is only _index_size (which is considerably
>>> lesser than _file_size).
>>>
>>> 427 // Close image file.
>>> 428 void ImageFileReader::close() {
>>> 429     // Deallocate the index.
>>> 430     if (_index_data) {
>>> 431         osSupport::unmap_memory((char*)_index_data, _index_size);
>>> 432         _index_data = NULL;
>>> 433     }
>>>
>>> Wanted to check if this is an oversight, or if there is a reason
>>> behind this and I am missing something. Shouldn't the amount of
>>> memory unmapped be map_size() too ?
>> It doesn't look right but needs closer examination. However, I'm
>> curious how you are running into it as it will be completely unmapped
>> when the VM terminates. Is this a tool or test that runs "in process"?
>>
>> -Alan
>