Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

Requirements: 
Jira Legacy
server

...

System Jira
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId

...

01505d01-

...

b853-

...

3c2e-

...

90f1-

...

ee9b165564fc
keyMODINVOICE-101
Jira Legacy
server

...

System Jira
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId

...

01505d01-

...

b853-

...

3c2e-

...

90f1-

...

ee9b165564fc
keyMODORDERS-311
Jira Legacy
server

...

System Jira
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId

...

01505d01-

...

b853-

...

3c2e-

...

90f1-

...

ee9b165564fc
keyMODORGS-45

Memory issues detection approach

Throttled mode of service load is most convenient way to identify memory leaks or another memory utilization issues. The utility https://github.com/khandramai/gatling-folio-performance for emulation of load regime was implemented based on load and performance testing framework Gatling.

...

Deep analysis of the memory utilization was performed with using Eclipse Memory Analyzer on the basis of memory dumps prepared in advance during the operation of the service (after 1 hour of service operating).

...

Issues Found

JMX reporting based on Dropwizard metrics

Description

TBDThe heap memory size is constantly increasing under load, reaches the maximum value for the container and a service failure occurs. The service does not recover after a load disconnect.

Detailed investigation

Emulation of the load on local equipment showed the following results.

CPU/Heap utilization:

Heap dumps: mod-invoices.hprofmod-invoices-storage.hprof.

Heap dumps analysis showed that problem relates to drop wizard metrics collecting. After manual disabling of metrics collecting load test passes successfully with the following CPU/Heap utilization during 4 hours:

mod-invoice

Image Modified

mod-invoice-storage

Image Modified

Heap dumps analysis showed that issue was fixed.

Summary

Disable Dropwizard metrics on production environment.