[FOLIO-2242] Update group_vars files in folio-ansible to remove Java memory settings Created: 06/Sep/19  Updated: 03/Jun/20  Resolved: 16/Oct/19

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P2
Reporter: Wayne Schneider Assignee: David Crossley
Resolution: Done Votes: 0
Labels: devops, platform-backlog
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: Text File summary-201910110824.txt     Text File summary-201910132248.txt     Text File summary-201910140109.txt     Text File summary-201910140252.txt    
Issue links:
Blocks
blocks FOLIO-1729 Use container memory limits to manage... Closed
is blocked by FOLIO-2234 Add LaunchDescriptor settings to each... Closed
is blocked by FOLIO-2235 Add LaunchDescriptor settings to each... Closed
Relates
relates to FOLIO-2185 SPIKE: how to maintain resource deplo... Closed
relates to FOLIO-2310 Update group_vars/release and next-re... Closed
Sprint: CP: sprint 74
Story Points: 3
Development Team: Core: Platform

 Description   

Java memory settings are now controlled by the LaunchDescriptor – or will be, once the blockers for this issue are resolved.

(This ticket addresses removing Xmx for snapshot and testing. See FOLIO-2310 Closed for "release" and "next-release".)



 Comments   
Comment by Ian Hardy [ 19/Sep/19 ]

Question on this one, apologies if I'm reading this wrong. In this transition phase the container memory limit is set to 1.33 * the value of the max heap size in folio-ansible like this:

JAVA_OPTIONS, value: "-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -Xmx256m"

When the this is removed from folio ansible, the ld has:

"-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap"

Wouldn't we lose that 1/3 bit of headroom for the OS etc, and get OOM killed again if the main java process reaches the max heap size?

Comment by Wayne Schneider [ 19/Sep/19 ]

Ian Hardy – We will certainly need to test it, but as I understand it, the UseCGroupMemoryLimitForHeap option simply tells Java to see the container limit as the total available memory. Java then operates as usual on a physical system, regulating its heap size to remain within the total memory limit (presumably setting max heap to less than the total size). I did some cursory testing with mod-notes, and that did seem to be the case.

Comment by David Crossley [ 14/Oct/19 ]

I have done some testing over the last few days.

With a branch of folio-ansible, did the Jenkins fresh build of folio-snapshot-test, then did ssh to the instance and gathered statistics about the memory usage by each docker container.

  • A) Attachment summary-201910110824.txt – After one run of statistics gathering.
  • B) Attachment summary-201910132248.txt – After doing some data loading of MARC records, some inventory search/display, a number of runs of statistics gathering thoughout the day.

Then removed the -Xmx settings in folio-ansible group_vars for "snapshot" and "snapshot-core" only. (The mod-agreements and mod-invoice have different options, so not touching those.)
Did the Jenkins fresh build of folio-snapshot-test, then activity and statistics gathering.

  • C) Attachment summary-201910140109.txt – After four runs statistics gathering. After mod-agreements has finished loading.
  • D) Attachment summary-201910140252.txt – After doing some data loading of MARC records, some inventory search/display, some Checkouts, a number of runs of statistics gathering thoughout the day.

Some notes about the statistics summaries:

The first part is just showing entries from the individual stats files for a couple of modules.

The second part shows a crude chart of the change in memory usage. Two characters for each stats run. So "__" means no change. The positive numbers show an increase on a scale of 1 to 5, where +5 is 10% (or more) increase, and +1 is 0.5 percent increase. Negative numbers are a reduction in memory usage.

It seems that the memory usage is as expected after removing the -Xmx settings.

Comment by David Crossley [ 15/Oct/19 ]

The -Xmx settings are now removed from snapshot and testing.

Comment by David Crossley [ 16/Oct/19 ]

Closed this ticket as done. However, there is still FOLIO-2310 Closed for the group_vars of "release" and "next-release".

Generated at Thu Feb 08 23:19:15 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.