Data Import > Create MARC authority records with CICO
- Iakov Zaborov
- Martin Tran
UIDATIMP-1033 - Getting issue details... STATUS
#records | began | end | Duration min |
|---|---|---|---|
| 5000 | 8:14 AM 2/3/2022 | 8:16 AM 2/3/2022 | 2 |
| 10000 | 8:35 AM 2/3/2022 | 8:39 AM 2/3/2022 | 4 |
| 25000 | 8:42 AM 2/3/2022 | 8:49 AM 2/3/2022 | 7 |
| 50000 | 10:45 AM 2/3/2022 | 10:58 AM 2/3/2022 | 13 |
CICO activity during data import 5K 10K and 25K
CICO activity during data import 50K two times (underlined in red)
summary CICO requst times
CI-CO 8 users
CI-CO 8 users + SRS MARC Authority Data Import
CI-CO 20 users
CI-CO 20 users + SRS MARC Authority Data Import
Comparisons
Check-in-check-out response times are affected by data import. The response time is about 2x to 3x for check in and check out.
| 8 users | 8 users + 50KCreate SRS MARC Authority | 20 users | 20 users + 50KCreate SRS MARC Authority | |
|---|---|---|---|---|
| CI average | 0.618s | 0.867s | 0.639s | 1.136s |
| CO average | 1.22s | 1.786s | 1.159s | 2.054s |
| CI max | 2.078s | 2.959s | 1.672s | 1.888s |
| CO max | 2.340s | 3.329s | 2.7s | 3.697s |
With background activities, MARC Authority imports are slightly slower.
Profile | Standalone Import Durations | Data Import Durations with CICO | |
|---|---|---|---|
| 1K | Default - Create SRS MARC Authority | 40 sec | |
5K | Default - Create SRS MARC Authority | 1 min 14 sec | 2 min |
| 10K | Default - Create SRS MARC Authority | 2 min 32 sec | 4 min |
| 25K | Default - Create SRS MARC Authority | 6 min | 7 min |
| 50K | Default - Create SRS MARC Authority | 12 min | 13 min |
It's clear that (MARC Authority) data import affects check-in check-out response times much more than check-in check-out affects data import durations.
max resource usage for modules comparison with Data Import MARC Authorities (Lotus Snapshot)
test | module | CPU % | CPU % with CICO in background |
|---|---|---|---|
| 50k | mod-source-record-manager | 685 | 955 (139%) |
| 50k | mod-data-import-cs | 404 | 410 (101%) |
| 50k | okapi | 385 | 630 (163%) |
| 50k | mod-inventory | 235 | 270 (114%) |
| 50k | mod source-record-storage | 231 | 440 (190%) |
| 50k | mod-inventory-storage | 117 | 380 (324%) |
| 50k | mod-data-import | 111 | 260 (222%) |
max resource usage:
module | RAM % | RAM % with CICO in background |
|---|---|---|
| mod-source-record-manager | 60 | 47 (78%) |
| mod-data-import-cs | 80 | 77 (96%) |
| okapi | 76 | 65 (85%) |
| mod-inventory | 87 | 56 (64%) |
| mod source-record-storage | 78-80 | 61 (78%) |
| mod-inventory-storage | 51 | 79 (154%) |
| mod-data-import | 22 | 22 (100%) |
Instance CPU during data import 50K two times
Memory during data import 50K two times after restart modules.
Instance CPU during data import 50K and CICO 20 users in background
Service CPU during data import 50K and CICO 20 users in background
Memory usage during data import 50K and CICO 20 users in background
