Data Import with Check-ins Check-outs (Orchid)

Overview

This document contains the results of testing Check-in/Check-out and Data Import for MARC Bibliographic records in the Orchid release to detect performance trends.

Ticket: PERF-472 - Getting issue details... STATUS

Summary

  • There is a significant performance degradation of data import for Orchid in comparison to Nolana results. Response time are about 2 times higher. It might be due to fixing differences in the database schemas. More details...
  • Data import response times are up to 22% higher with parallel Check-in/Check-out than pure Data import results. More details...
  • Check-in/Check-out response times are up to 168% higher with parallel Data import than pure Check-in/Check-out results. More details...
  • There is memory utilization increase observed which is caused by previous modules restarting (everyday cluster shut down process). More details...
  • Average CPU usage did not exceed 130 % for all the modules. Spikes can be observed in CPU usage of mod-data-import at the beginning of the Data Import jobs up to 170%. CPU usage is about 2 times lower than in the same test for Nolana. More details...
  • Average DB CPU usage during data import is about 95%. More details...

Test Runs 

Test #

Scenario

Load levelComment
1DI MARC Bib Create5K, 10K, 25K, 50K, 100K consequentially (with 5 min pause)
CICO 8 users
2DI MARC Bib Update5K, 10K, 25K, 50K, 100K consequentially (with 5 min pause)100k file was completed with errors (1 item discarded)
CICO 8 users

Test Results

Data import

Profile
MARC File

DI Duration
Check In, Check Out Response time (8 users)
CI Average secCO Average sec
DI MARC Bib Create (PTF - Create 2)5K.mrc00:05:01.7330.9611.442
10K.mrc00:09:06.7521.0581.624
25K.mrc00:24:28.1671.0561.621
50K.mrc00:43:03.7850.9361.519
100K.mrc01:35:50.7490.8681.468
DI MARC Bib Update (PTF - Updates Success - 1)5K.mrc00:04:52.4960.8551.339
10K.mrc00:09:22.7650.9161.398
25K.mrc00:24:02.2380.9221.425
50K.mrc00:47:13.8760.9041.456
100K.mrc01:40:25.5330.8381.415

Check-in/Check-out

ScenarioLoad levelRequestResponse time, sec

95 percaverage
Circulation Check-in/Check-out (without Data import)8 usersCheck-in0.4890.394
Check-out0.7930.724

Comparison

The following table compares test results of current release (Orchid) to the previous release numbers (Nolana) and to the baselines Orchid results (CICO without DI and DI without CICO).

File size


Data import duration

Deviation (compared DI Orchid without CICO and with CICO)


Check-in/Check-out 8 users response time  with Data Import (avg, sec)

Deviation

(compared CICO Orchid without DI and with DI)

without CICOwith CICO (8 users)Check-in Check-out 

Check-in 

Check-out 

Nolana*

Orchid**

Nolana*

Orchid 

Nolana*

Orchid  

Nolana*

Orchid 

5K MARC BIB Create

2 min 51 sec4 min 30 sec1 min 56 sec5 min 01 sec+11%

0.817

0.961

1.417

1.442+143%+96%

5K MARC BIB Update

2 min 27 sec4 min 2 sec2 min 51 sec4 min 52 sec+20%

0.747

0.855

1.094

1.339+117%+84%

10K MARC BIB Create

4 min 55 sec9 min 25 sec3 min 57 sec9 min 06 sec-3%

0.842

1.058

1.574

1.624+168%+124%
10K MARC BIB Update4 min 50 sec8 min 10 sec4 min 57 sec9 min 22 sec+14%

0.541

0.916

1.026

1.398+132%+93%
25K MARC BIB Create11 min 56 sec22 min 16 sec9 min 24 sec24 min 28 sec+9%

0.882

1.056

1.641

1.621+168%+123%
25K MARC BIB Update12 min 20 sec19 min 39 sec13 min 12 sec24 min 2 sec+22%

0.700

0.922

1.248

1.425+134%+96%
50K MARC BIB Create23 min 43 sec39 min 27 sec19 min 28 sec43 min 3 sec+9%

0.926

0.936

1.666

1.519+137%+109%
50K MARC BIB Update24 min 5 sec38 min 30 sec Completed with errors (1 item discarded)27 min 39 sec47 min 13 sec+22%

0.700

0.904

1.199

1.456+129%+101%
100K MARC BIB Create49 min 40 sec1 hour 38 min38 min 44 sec1 hour 35 min-3%

1.021

0.868

1.862

1.468+120%+102%
100K MARC BIB Update51 min 15 sec1 hour 33 min48 min 45 sec1 hour  40 min Completed with errors (1 item discarded)+7%

0.556

0.838

1.046

1.415+112%+95%

* Nolana DI and CICO results are taken from Data Import with Check-ins Check-outs Nolana.

** Orchid DI results are taken from Data Import test report (Orchid).