| Table of Contents | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
This document contains the results of testing Data Import for MARC Bibliographic records at Quesnelia release [non-ECS]. https://folio-org.atlassian.net/jira/people/712020:c7153665-e98d-4df6-a9f4-fe368ae2480f/boards/224?selectedIssue=PERF-836
Ticket:
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
Summary
All Data-imports jobs finished successfully.
Duration for DI increases correlates to the number of the records imported.
The average CPU utilization of modules for all Create and Update jobs did not exceed 150 %. Spikes at the beginning on mod-data-import module are expected because of large file uploading.
No memory leak is suspected for DI modules. During DI of 500K on both profiles, mod-pubsub has a sawtooth-like memory usage within the range of 40-60%.
Approximate DB CPU usage is close to 95% and this number goes for all jobs with files more than 10k records.
...
Test # | Data-import test | Duration Poppy | Duration Quesnelia | Difference, sec | Results | |
|---|---|---|---|---|---|---|
1. | 1k MARC BIB Create | PTF - Create 2 | 39 sec | 54 sec | ↓ 15 sec | Completed |
2. | 5k MARC BIB Create | PTF - Create 2 | 2 min 22 sec | 3 min 20 sec | ↓ 58 sec | Completed |
3. | 10k MARC BIB Create | PTF - Create 2 | 4 min 29 sec | 6 minutes | ↓ 1 min 31 sec | Completed |
4. | 25k MARC BIB Create | PTF - Create 2 | 10 min 38 sec | 13 min 41 sec | ↓ 3 min 3 sec | Completed |
5. | 50k MARC BIB Create | PTF - Create 2 | 20 min 26 sec | 21 min 59 sec | ↓ 1 min 33 sec | Completed |
6. | 100k MARC BIB Create | PTF - Create 2 | 2 hours 46 min Cancelled | 40 min 16 sec | Not applicable | Completed |
7. | 500k MARC BIB Create | PTF - Create 2 | Not tested | 3 hours 27 min | Not applicable | Completed |
8. | 1k MARC BIB Update | PTF - Updates Success - 6 | 34 sec (PTF - Updates Success - 1) | 1 min 59 sec | Not applicable | Completed |
9 | 2k MARC BIB Update | PTF - Updates Success - 6 | 1 min 09 sec (PTF - Updates Success - 1) | 2 min 43 sec | Not applicable | Completed |
10 | 5k MARC BIB Update | PTF - Updates Success - 6 | 2 min 31 sec (PTF - Updates Success - 1) | 7 min 10 sec | Not applicable | Completed |
11 | 10k MARC BIB Update | PTF - Updates Success - 6 | 5 min 13 sec (PTF - Updates Success - 1) | 10 min 27 sec | Not applicable | Completed |
12 | 25k MARC BIB Update | PTF - Updates Success - 6 | 12 min 27 sec (PTF - Updates Success - 1) | 23 min 16 sec | Not applicable | Completed |
13 | 50k MARC BIB Update | PTF - Updates Success - 6 | Not tested | 40 min 52 sec | Not applicable | Completed |
14 | 100k MARC BIB Update | PTF - Updates Success - 6 | Not tested | 1 hrs 2 min | Not applicable | Completed |
15 | 500k MARC BIB Update | PTF - Updates Success - 6 | Not tested | 5 hrs 31 min | Not applicable | Completed |
Test Runs
MARC BIB CREATE
...