[MODEXPW-269] mod-data-export-worker: memory settings Created: 29/Sep/22  Updated: 12/Oct/22  Resolved: 12/Oct/22

Status: Closed
Project: mod-data-export-worker
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: TBD
Reporter: Viachaslau Khandramai (Inactive) Assignee: Viachaslau Khandramai (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: PNG File image-2022-10-11-09-03-11-771.png     PNG File image-2022-10-11-09-03-11-889.png     PNG File image-2022-10-11-09-03-12-080.png     PNG File image-2022-10-11-09-03-12-299.png    
Issue links:
Relates
relates to FOLIO-3603 mod-data-export-worker: memory settings Closed
Sprint: Firebird - Sprint 150
Story Points: 1
Development Team: Firebird
RCA Group: TBD

 Description   

mod-data-export-worker requires the following Java memory settings for stable operating: -XX:MetaspaceSize=384m -XX:MaxMetaspaceSize=512m -Xmx2048m - https://github.com/folio-org/mod-data-export-worker/blob/master/README.md#memory-configuration

Memory configuration should be updated on hosted reference env.



 Comments   
Comment by Mikita Siadykh [ 29/Sep/22 ]

hi Wayne Schneider David Crossley 
could you please review and add to requests list?

Comment by David Crossley [ 06/Oct/22 ]

These settings are to be configured in the LaunchDescriptor section of your ModuleDescriptor.

Refer to https://dev.folio.org/guides/module-descriptor/#launchdescriptor-properties

So configure the "JAVA_OPTIONS" to have the settings that you need:
https://dev.folio.org/guides/module-descriptor/#env-java_options

Also configure the "Memory":
https://dev.folio.org/guides/module-descriptor/#memory
(The value that you are suggesting does seem high for the basic system that is folio-snapshot, as explained there. But do use what you need for this system.)

The folio-ansible is currently configured to over-ride the "JAVA_OPTIONS" setting of your LaunchDescriptor (as shown here). So please advise us when you have published the updated ModuleDescriptor. Then we will remove the folio-ansible over-ride, and we will do a test build of folio-snapshot-test.

Comment by Viachaslau Khandramai (Inactive) [ 07/Oct/22 ]

Hello David Crossley , I've updated MD with -Xmx2048m and container memory for the following reasons: 3/2*2048m = 3072m = 3 072 000 000.

 

Comment by David Crossley [ 07/Oct/22 ]

However please see the note at this section:
https://dev.folio.org/guides/module-descriptor/#env-java_options
Viz:

setting -Xmx and -Xms disables the automatic heap sizing

No other FOLIO module uses "-Xmx" setting.

That is beyond my expertise. I note that the link to that article is now broken. If you find a new explanation then please tell us, so that we can adjust that doc.

This configuration is up to you.

Comment by Magda Zacharska [ 07/Oct/22 ]

Hi Viachaslau Khandramai - how this ticket can be verified?

Comment by David Crossley [ 10/Oct/22 ]

I did a build of "folio-snapshot-test" and discovered that because this module has other unusual configuration in folio-ansible (e.g. FOLIO-3581 Closed ) then folio-ansible does need to replicate/over-ride all of the settings from the docker "env" portion of your LaunchDescriptor.

So that is now done, but only for the "folio-snapshot-test".

Please verify via folio-snapshot-test-okapi.dev.folio.org

As usual you can obtain the module docker logs via: 
https://dev.folio.org/faqs/how-to-obtain-refenv-logs/

Please note that you need to verify as soon as possible, as we need to utilise "folio-snapshot-test" for other purposes.

After you have verified then i will apply this configuration to folio-ansible mainline.

Comment by David Crossley [ 11/Oct/22 ]

Ping – Viachaslau Khandramai and Mikita Siadykh and Magda Zacharska 

Please respond. We need to move on.

Comment by Viachaslau Khandramai (Inactive) [ 11/Oct/22 ]

Hi David Crossley ,

bulk-edit is verified on https://folio-snapshot-test.dev.folio.org/. The main flow works fine - i.e. there are no any issues in storage configurations:

data-export-worker is started with needed parameters (from the logs): exec java -XX:MaxRAMPercentage=66.0 -XX:MetaspaceSize=384m -XX:MaxMetaspaceSize=512m -Xmx2048m -XX:+ExitOnOutOfMemoryError -cp . -jar /usr/verticles/mod-data-export-worker-fat.jar

CC Magda Zacharska Mikita Siadykh 

Thanks,

Slava

Comment by David Crossley [ 11/Oct/22 ]

Thanks. This is now configured for folio-ansible proper.

The re-build of "folio-snapshot-2" later today will utilise these new settings.

Note that in the future if you need to modify the docker "env" settings (which should always be shown in your LaunchDescriptor for other operators) then you will need to raise a FOLIO DevOps ticket for these to be replicated in folio-ansible.

(Other modules do not require this, as their LaunchDescriptor settings are used as-is by folio-ansible.)

Comment by David Crossley [ 12/Oct/22 ]

Checked the docker logs on folio-snapshot-2. Okay.

Done.

 

Generated at Thu Feb 08 23:29:17 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.