Data Import Update multi tenant (Poppy)
Overview
- This document contains the results of testing Data Import for MARC Bibliographic records with an update job in the Poppy release. Ticket: PERF-779
Summary
Test set run №2 (updated configuration for mod-permission)
- Three tests of DI with update jobs with 25k records were carried out on the pcp1 cluster.
- DI with a new configuration for mod-permission was successful. DI jobs are processed one by one in test run #3.
- Mod-permission version changed from 6.4.0 → 6.3.2 doesn't solve the issue with NullPointerException: "mappingParameters" is null for configuration parameter for mod-permission in Mem Hard Limit=1684, Mem Soft limit=1544, Xmx=1024m.
- Changed configuration parameter for mod-permission in Mem Hard Limit=1684→2384, Mem Soft limit=1544→2244, Xmx=1024m→1500m allowed to run tests successfully and avoided issue with "mappingParameters" is null.
- Memory consumption for mod-data-import was 36%, mod-inventory - 94%. No memory leaks are suspected.
- CPU utilization for mod-data-import spiked to 72%, mod-inventory - 171%. It was observed that CPU utilization for mod-inventory has a growing trend.
- DB CPU utilization was 97%.
- DB connections - 500 on average and 681 during spikes connected to mod-permission spikes on the CPU graph.
- Slow queries found - SELECT jsonb FROM fs07000002_mod_permissions.permissions
Test set run №1
- Three tests of DI with update jobs with 25k records were carried out on the pcp1 cluster.
- The first and second tests were successful. The third test of DI on three tenants running concurrently failed after a third data import started. Errors attached.
Slow queries found for SELECT jsonb FROM fs07000001_mod_permissions.permissions
- Issue with a spike from mod-permission that led to test #3 problems were investigated. The module stopped after a timeout. Additional analysis with a method that consumed most of the resources attached.
Recommendations and Jiras
- Created Jira ticket to Investigate "mappingParameters" is null error: PERF-801
Test Results
Test set run#2 (updated configuration for mod-permission)
Profile | Test # | Tenant | MARC File | DI Duration Poppy (hh:mm:ss) | Results |
---|---|---|---|---|---|
DI MARC Bib Update (PTF - Updates Success - 1) | 1 | fso7000001 | 25K.mrc | 00:17:18 | Completed |
2 | fso9000000 | 25K.mrc | 00:23:11 | Completed | |
fso7000002 | 25K.mrc | 00:24:16 | Completed | ||
3 | fso9000000 | 25K.mrc | 00:26:36 | Completed | |
fso7000002 | 25K.mrc | 00:38:01 | Completed | ||
fso7000001 | 25K.mrc | 00:49:24 | Completed |
Test set run#1
Profile | Test # | Tenant | MARC File | DI Duration Poppy (hh:mm:ss) | Results |
---|---|---|---|---|---|
DI MARC Bib Update (PTF - Updates Success - 1) | 1 | fso9000000 | 25K.mrc | 00:20:23 | Completed |
2 | fso9000000 | 25K.mrc | 00:24:48 | Completed | |
fso7000001 | 25K.mrc | 00:31:53 | Completed | ||
3 | fso9000000 | 25K.mrc | 00:27:10 | Completed | |
fso7000001 | 25K.mrc | 00:36:09 | with errors* | ||
fso7000002 | 25K.mrc | 00:30:29 | with errors* |
Comparison
The following table compares previous results of Poppy release with current results.
Profile | MARC File | Test run # | DI Duration | DI Delta, (hh:mm:ss) | |
Poppy (previous) | Poppy | ||||
DI MARC Bib Update (PTF - Updates Success - 1) | 25K.mrc | 1 | 00:14:50 | 00:17:18 | + 00:02:28 |
2 | not tested | 00:24:16 | - | ||
3 | not tested | 00:49:24 | - |
Errors for test set #1
mod-permission heap dump analysis