PTF - Data Import Create/Update multi tenant (Quesnelia) [ECS]

PTF - Data Import Create/Update multi tenant (Quesnelia) [ECS]

Overview

  • This document contains the results of testing  Data Import for MARC Bibliographic records with an create and update jobs on the Quesnelia [ECS] release on qcon environment. 

PERF-859: [Quesnelia] [ECS] [Data Import] Creates + Updates of MARC BIB with MultiTenantsClosed 

Summary

  • Data Import tests finished successfully on qcon environment using the PTF - Create 2 profile and file with 10k, 25k and 50k records.

  • Data Import test for PTF - Updates Success - 2 profile with 25k records file on 3 tenants concurrently finished with errors for two records. 

  • Comparing with previous testing results Poppy and Quesnelia releases

    • Data Import processed DI MARC Bib Create jobs including test on 2 and 3 tenants concurrently without errors for Quesnelia releases.

    • Data Import processed DI MARC Bib Update job with 25k file for 3 tenants concurrently with two errors.

    • Data Import durations for Create job has performance improvement around 50% for Quesnelia releases.

    • Data Import durations for Update job stayed in the same time range in Average for Quesnelia releases.

  • When comparing the durations of the 'main' job on the first tenant and observing how the second and third jobs affect and slow down the main job, it becomes evident that processing times are twice as slow for three tenants compared to two tenants when handling a 10k file.

  • Additionally, for a DI MARC Bib Update job with 25k file , the duration increases by only 6%.

Test Runs and Results

This table contains durations for Data Import. 

Profile

Test #

Tenant

MARC File

DI Duration by Tenant Quesnelia
(hh:mm:ss)

Results

Profile

Test #

Tenant

MARC File

DI Duration by Tenant Quesnelia
(hh:mm:ss)

Results

DI MARC Bib Create (PTF - Create 2)





1


College
(cs00000int_0001)

10K.mrc

0:04:52

Completed

Professional 
(cs00000int_0002)

10K.mrc

0:05:24

Completed

2



College
(cs00000int_0001)

10K.mrc

0:06:35

Completed

Professional 
(cs00000int_0002)

10K.mrc

0:06:18

Completed

School
(cs00000int_0003)

10K.mrc

0:08:13

Completed

3

College
(cs00000int_0001)

25K.mrc

0:11:04

Completed

Professional 
(cs00000int_0002)

25K.mrc

0:16:16

Completed

4

College
(cs00000int_0001)

25K.mrc

0:11:47

Completed

Professional 
(cs00000int_0002

25K.mrc

0:19:12

Completed

School
(cs00000int_0003)

25K.mrc

0:23:41

Completed

5

College
(cs00000int_0001)

50K.mrc

0:36:02

Completed

Professional 
(cs00000int_0002)

50K.mrc

0:42:50

Completed

School
(cs00000int_0003)

50K.mrc

0:47:13

Completed

DI MARC Bib Update (PTF - Updates Success - 2)












6

College
(cs00000int_0001)

10K.mrc

0:11:53

Completed

Professional 
(cs00000int_0002)

10K.mrc

0:14:49

Completed

7

College
(cs00000int_0001)

10K.mrc

0:20:39

Completed

Professional 
(cs00000int_0002)

10K.mrc

0:20:07

Completed

School
(cs00000int_0003)

10K.mrc

0:21:05

Completed

8

College
(cs00000int_0001)

25K.mrc

0:38:41

Completed

Professional 
(cs00000int_0002)

25K.mrc

0:39:09

Completed

9

College
(cs00000int_0001)

25K.mrc

0:41:32

Completed
with errors*

Professional 
(cs00000int_0002)

25K.mrc

0:42:34

Completed

School
(cs00000int_0003)

25K.mrc

0:16:27

Completed

10



College
(cs00000int_0001)

50K.mrc

1:43:17

Completed

Professional 
(cs00000int_0002)

50K.mrc

1:50:34

Completed

School
(cs00000int_0003)

50K.mrc

1:59:29

Completed



This table contains durations for Data Import by each tests. 

Profile

Test #

Tenants

MARC File

DI Duration by Test Quesnelia
(hh:mm:ss)

Profile

Test #

Tenants

MARC File

DI Duration by Test Quesnelia
(hh:mm:ss)

DI MARC Bib Create (PTF - Create 2)

1

College (cs00000int_0001)
Professional (cs00000int_0002)

10K.mrc

0:09:17

2

College (cs00000int_0001)
Professional (cs00000int_0002)
School (cs00000int_0003)

10K.mrc

0:12:01

3

College (cs00000int_0001)
Professional (cs00000int_0002)

25K.mrc

0:20:09

4

College (cs00000int_0001)
Professional (cs00000int_0002)
School (cs00000int_0003)

25K.mrc

0:30:54

5

College (cs00000int_0001)
Professional (cs00000int_0002)
School (cs00000int_0003)

50K.mrc

1:00:44

DI MARC Bib Update (PTF - Updates Success - 2)

6

College (cs00000int_0001)
Professional (cs00000int_0002)

10K.mrc

00:20:07

7

College (cs00000int_0001)
Professional (cs00000int_0002)
School (cs00000int_0003)

10K.mrc

00:27:56

8

College (cs00000int_0001)
Professional (cs00000int_0002)

25K.mrc

00:47:08

9

College (cs00000int_0001)
Professional (cs00000int_0002)
School (cs00000int_0003)

25K.mrc

00:26:46

10

College (cs00000int_0001)
Professional (cs00000int_0002)
School (cs00000int_0003)

50K.mrc

02:22:13

Comparison

This table compares the DI durations of the 'main' job on the first tenant and how the second and third jobs affect and slow down the main job during Data Import.

Profile

MARC File

DI Duration "main" job
for 1 tenants

DI Duration "main" job
for 2 tenants

DI Duration "main" job
for 3 tenants

Profile

MARC File

DI Duration "main" job
for 1 tenants

DI Duration "main" job
for 2 tenants

DI Duration "main" job
for 3 tenants

DI MARC Bib Create (PTF - Create 2)

10K.mrc

0:05:35

*test run in other day

0:04:52

0:06:35

25K.mrc

0:15:27
*test run in other day

0:11:04

0:11:47

DI MARC Bib Update (PTF - Updates Success - 2)

10K.mrc

0:07:55

0:11:53

0:20:39

25K.mrc

0:22:39

0:38:41

0:41:32

This table contains сompare durations for second and third jobs compare to the main job for Data Import. 

Profile

Test #

Tenant

MARC File

DI Duration for second and third jobs
сompare to main job
(hh:mm:ss)

Profile

Test #

Tenant

MARC File

DI Duration for second and third jobs
сompare to main job
(hh:mm:ss)

DI MARC Bib Create (PTF - Create 2)





1


College
(cs00000int_0001)

10K.mrc

0:04:52

Professional 
(cs00000int_0002)

10K.mrc

0:05:24+11%