/
Data Import with Check-ins Check-outs (Poppy)

Data Import with Check-ins Check-outs (Poppy)

Dec 08, 2023

Overview

This document contains the results of testing Check-in/Check-out and Data Import for MARC Bibliographic records in the Poppy release.

Ticket: https://folio-org.atlassian.net/browse/PERF-697

Summary

There is significant performance improvement for data import in Poppy comparing with Orchid. Durations are closer to Nolana release results. CI/CO response times improved 10% in average with all DI jobs. There's only CI response time degraded in 100k job with 17%.

Comparing Orchid and Poppy releases DI durations in create jobs are up to 10% higher with 5k file and 5% for bigger files with parallel Check-in/Check-out than pure Data import results. DI durations are higher up to 30% in update jobs.

Response times of CI/CO in Poppy release are twice higher with Data Import job with 100k compared with pure CI/CO.

DI create jobs with CI/CO for files 10k, 25k, 50k, 100k perform better in Poppy release and DI update jobs perform better with 25k, 50k, 100k with CI/CO.

No memory leaks are observed.

Average CPU utilization increased for mod-inventory up to 20% comparing with Orchid. So it did not exceed 150% for all the modules. The highest consumption observed from mod-inventory. The rest of services were almost on the same level in the same test in Orchid  Data Import with Check-ins Check-outs Orchid and didn't exceed 60%.

DB needs more connections (in average +20 more) needed for the same tests as in Orchid for all create and update jobs.

Average DB CPU usage is the same as in Orchid - 95%.

Upd: During previous tests on pcp1 and ocp3 there were problems with DI jobs running big files (Create jobs 100k and higher, Update jobs with 25k and higher). The problem was solved after new deployment and updates of 13 modules in scope of ticket https://folio-org.atlassian.net/browse/RANCHER-1121 and https://folio-org.atlassian.net/browse/RANCHER-1114 and the large DI jobs are completing successfully now.

 

Test Runs 

Test #

Scenario

Load level

Comment

Test #

Scenario

Load level

Comment

1

DI MARC Bib Create

5K, 10K, 25K, 50K, 100K consequentially (with 5 min pause)

 

CICO

 8 users

 

2

DI MARC Bib Update

5K, 10K, 25K, 50K, 100K consequentially (with 5 min pause)

 

CICO

 8 users

 

Test Results

Data import

Total time for all Data Export jobs - 1 hour 16 minutes 47 seconds.

Profile


MARC File


DI Duration

Poppy (hh:mm:ss)

Check In, Check Out Response time (8 users)

Poppy

CI Average sec

CO Average sec

DI MARC Bib Create (PTF - Create 2)

5K.mrc

00:02:53

0.901

1.375

10K.mrc

00:04:32

0.902

1.47

25K.mrc

00:11:14

1

1.571

50K.mrc

00:21:55

0.981

1.46

100K.mrc

00:47:02

1.018

1.491

Data Export MARC Bib (Export for Data Import updates)



5K.mrc

00:02:09

0.495

0.836

10K.mrc

00:04:19

0.468

0.917

25K.mrc

00:10:30

0.497

0.935

50K.mrc

00:20:11

0.509

0.923

100K.mrc

00:39:38

 

 

DI MARC Bib Update (PTF - Updates Success - 1)

5K.mrc

00:03:19

0.755

1.169

10K.mrc

00:06:20

0.75

1.307

25K.mrc

00:14:04

0.822

1.403

50K.mrc

00:29:59

0.893

1.424

100K.mrc

01:03:03

0.908

1.51

Check-in/Check-out without DI

Scenario

Load level

Request

Response time, sec
Poppy

95 perc

average

Circulation Check-in/Check-out (without Data import)

8 users

Check-in

0.489

0.431

Check-out

0.969

0.828

Comparison

CICO with DI comparison

Profile

MARC File

DI Duration

Deviation, % (compared DI Poppy without CICO and with CICO)

DI Delta, (hh:mm:ss) Orchid/Poppy

Check In, Check Out Response time (8 users)

Check In, Check Out Response time (8 users)

Delta, %

without CI/CO

with CI/CO

Orchid

Poppy

Orchid/Poppy

Orchid/Poppy

Orchid*

Poppy

Orchid*

Poppy

CI Average sec

CO Average sec

CI Average sec

CO Average sec

CI

CO

DI MARC Bib Create (PTF - Create 2)

5K.mrc

00:04:30

00:02:39

00:05:01

00:02:53

+8.5% / 14 sec

 - 00:02:08

0.961

1.442

0.901

1.375

-6.24%

-4.65%

10K.mrc

00:09:25

00:05:00

00:09:06

00:04:32

-9.3% / 28 sec

- 00:04:35

1.058

1.624

0.902

1.47

-14.74%

-9.48%

25K.mrc

00:22:16

00:11:15

00:24:28

00:11:14

-0.2% / 1 sec

- 00:13:14

1.056

1.621

1

1.571

-5.30%

-3.08%

50K.mrc

00:39:27

00:22:16

00:43:03

00:21:55

-1.5% / 21 sec

- 00:21:09

0.936

1.519

0.981

1.46

4.81%

-3.88%

100K.mrc

01:38:00

00:49:58

01:35:50

00:47:02

-5.8% / 2 min 56 sec

- 00:48:49

0.868

1.468

1.018

1.491

17.28%

1.57%

DI MARC Bib Update (PTF - Updates Success - 1)

5K.mrc

00:04:02

00:02:28

00:04:52

00:03:19

+34% / 51 sec

- 00:01:33

0.855

1.339

0.755

1.169

-11.70%

-12.70%

10K.mrc

00:08:10

00:05:31

00:09:22

00:06:20

+15% / 49 sec

- 00:03:03

0.916

1.398

0.75

1.307

-18.12%

-6.51%

25K.mrc

00:19:39

00:14:50

00:24:02

00:14:04

-5.1% / 46 sec

- 00:09:58

0.922

1.425

0.822

1.403

-10.85%

-1.54%

50K.mrc

00:38:30

00:32:53

00:47:13

00:29:59

-8.8% / 2 min 54 sec

- 00:17:15

0.904

1.456

0.893

1.424

-1.22%

-2.20%

100K.mrc

01:33:00

01:14:39

01:40:25

01:03:03

-15.5% / 11 min 36 sec

- 00:37:23

0.838

1.415

0.908

1.51

8.35%

6.71%

The following table compares test results of current release (Orchid) to the previous release numbers (Orchid) and to the baselines Poppy results (CICO without DI and DI without CICO).

* Orchid DI and CICO results are taken from Data Import with Check-ins Check-outs Orchid.

*** Completed with errors

Detailed CICO response time comparison

Scenario

Load level

Request

Response time, sec
Orchid

Response time, sec
Poppy

95 perc

average

95 perc

average

Circulation Check-in/Check-out (without Data import)

8 users

Check-in

0.489

0.394

0.489

0.431

Check-out

0.793

0.724

0.969

0.828

Detailed CICO response time comparison for CICO with DI in Poppy

Request*

Response time (avg, sec)

Pure CICO

CICO + 100K MARC BIB Create

CICO + 100K MARC BIB Update

Check-Out Controller

0.828

1.491

1.51