Memory Leaks: HttpClients

Description

Overview:
When running a check-in-check-out longevity (24hr) test with considerable load (20 virtual users) mod-pubsub shows clear signs of leaking HTTP Clients in which the client objects were not closed or reused. The CPU utilization also increased from 0 to 350%

Steps to Reproduce:
Run the attached check-in-check-out test for a long time at high load.

Expected Results:
Memory trend stabilizes after an hour max

Actual Results:
Memory trend rose for the first 12 hours, leveled out in the second half only because the database was overloaded with requests.


CPU usage showing the rising trend continues for at least 18 hours. Okapi's CPU usage went down over time, probably being dragged down by degrading performance of mod-pubsub.

Additional Information:
mod-pubsub's heap dump may be provided upon request. Because the compressed file is still big, over 100MB, it can't be attached here nor on Confluence.
Heap dump:


Interested parties:

CSP Request Details

None

CSP Rejection Details

None

Potential Workaround

None

Attachments

3

Checklist

hide

TestRail: Results

Activity

Show:

Martin Tran March 11, 2021 at 1:53 PM

The HTTP Clients leaks issue is resolved.

Martin Tran March 11, 2021 at 1:52 PM

Hi , Yes, this bug can be closed. We verified in a longevity test that there are no more HTTP clients leaks.

Ann-Marie Breaux March 11, 2021 at 5:31 AM

Hi In your notes for Mark V's meeting, I saw this:
Mod-pubsub: verified that v2.0.1 does not have HTTP Clients leaks anymore.
Does that mean we can close this bug?

Ann-Marie Breaux March 1, 2021 at 1:54 PM

Hi Please review now that a new version of mod-pubsub has been released. We think this has been corrected.

Ann-Marie Breaux February 10, 2021 at 2:01 PM

This bug needs to be retested and reviewed after the RMB update to PubSub is completed.

Done

Details

Assignee

Reporter

Labels

Priority

Development Team

Folijet

Fix versions

Release

R1 2021

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created November 9, 2020 at 11:28 PM
Updated May 27, 2021 at 10:15 PM
Resolved March 11, 2021 at 1:53 PM
TestRail: Cases
TestRail: Runs