Requirements: 

Jira Legacy
server System Jira columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key MODINVOICE-101

, 

Jira Legacy
server System Jira columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key MODORDERS-311

, 

Jira Legacy
server System Jira columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key MODORGS-45

...

Deep analysis of the memory utilization was performed with using Eclipse Memory Analyzer on the basis of memory dumps prepared in advance during the operation of the service (after 1 hour of service operating).

Issues Found

JMX reporting based on Dropwizard metrics

Description

The heap memory size is constantly increasing under load, reaches the maximum value for the container and a service failure occurs. The service does not recover after a load disconnect.

Detailed investigation

Emulation of the load on local equipment showed the following results.

CPU/Heap utilization:

Heap dumps: mod-invoices.hprof, mod-invoices-storage.hprof.

Heap dumps analysis showed that problem relates to drop wizard metrics collecting.

Image AddedImage Added

After manual disabling of metrics collecting load test passes successfully with the following CPU/Heap utilization during 4 hours:

mod-invoice

Image Modified

mod-invoice-storage

Image Modified

Heap dumps analysis showed that issue was fixed.

Summary

Disable Dropwizard metrics on production environment.