Data Import > Create MARC authority records with CICO
- Iakov Zaborov
- Martin Tran
UIDATIMP-1033 - Getting issue details... STATUS
#records | began | end | Duration min |
|---|---|---|---|
| 5000 | 8:14 AM 2/3/2022 | 8:16 AM 2/3/2022 | 2 |
| 10000 | 8:35 AM 2/3/2022 | 8:39 AM 2/3/2022 | 4 |
| 25000 | 8:42 AM 2/3/2022 | 8:49 AM 2/3/2022 | 7 |
| 50000 | 10:45 AM 2/3/2022 | 10:58 AM 2/3/2022 | 13 |
CICO activity during data import 5K 10K and 25K
CICO activity during data import 50K two times (underlined in red)
summary CICO requst times
CI-CO 8 users
CI-CO 8 users + SRS MARC Authority Data Import
CI-CO 20 users
CI-CO 20 users + SRS MARC Authority Data Import
Comparisons
Check-in-check-out response times are affected by data import. The response time is about 2x to 3x for check in and check out.
| 8 users | 8 users + 50KCreate SRS MARC Authority | 20 users | 20 users + 50KCreate SRS MARC Authority | |
|---|---|---|---|---|
| CI average | 0.618s | 0.867s | 0.639s | 1.136s |
| CO average | 1.22s | 1.786s | 1.159s | 2.054s |
| CI max | 2.078s | 2.959s | 1.672s | 1.888s |
| CO max | 2.340s | 3.329s | 2.7s | 3.697s |
With background activities, MARC Authority imports are slightly slower.
Profile | Standalone Import Durations | Data Import Durations with CICO | |
|---|---|---|---|
| 1K | Default - Create SRS MARC Authority | 40 sec | |
5K | Default - Create SRS MARC Authority | 1 min 14 sec | 2 min |
| 10K | Default - Create SRS MARC Authority | 2 min 32 sec | 4 min |
| 25K | Default - Create SRS MARC Authority | 6 min | 7 min |
| 50K | Default - Create SRS MARC Authority | 12 min | 13 min |
It's clear that (MARC Authority) data import affects check-in check-out response times much more than check-in check-out affects data import durations.
max resource usage for modules comparison with Data Import MARC Authorities (Lotus Snapshot)
test | module | CPU % | CPU % with CICO in background |
|---|---|---|---|
| 50k | mod-source-record-manager | 685 | 955 (139%) |
| 50k | mod-data-import-cs | 404 | 410 (101%) |
| 50k | okapi | 385 | 630 (163%) |
| 50k | mod-inventory | 235 | 270 (114%) |
| 50k | mod source-record-storage | 231 | 440 (190%) |
| 50k | mod-inventory-storage | 117 | 380 (324%) |
| 50k | mod-data-import | 111 | 260 (222%) |
max resource usage:
module | RAM % | RAM % with CICO in background |
|---|---|---|
| mod-source-record-manager | 60 | 47 (78%) |
| mod-data-import-cs | 80 | 77 (96%) |
| okapi | 76 | 65 (85%) |
| mod-inventory | 87 | 56 (64%) |
| mod source-record-storage | 78-80 | 61 (78%) |
| mod-inventory-storage | 51 | 79 (154%) |
| mod-data-import | 22 | 22 (100%) |
Instance CPU during data import 50K two times
Memory during data import 50K two times after restart modules.
