Versions Compared

Old Version 1

changes.mady.by.user Mykhailo Petryshyn

Saved on

compared with

New Version 2

changes.mady.by.user Mykhailo Petryshyn

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Overview

This document contains the results of testing Data Export tests on 61 tenants (excluding the central tenant):

  • (MARC BIB) on Mobius environment with Data Export tests for 500, 1k, 2k. Three CSV files were prepared to run Data Export with Default instances export profile.

Ticket:

Jira Legacy
serverSystem Jira
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyPERF-777

Summary

Recommendations & Jiras

Test Results

DE_instances T1(500-1K-2k). This table contains durations for Data-export for jobs with “Default instances export profile” on 61 tenants with 3 sets of data(500 records; 1000 records; 2000 records). 

Transaction name

DE 500, sec

DE 1K, sec

DE 2K, sec

DE_Exporting MARC Bib records workflow Transaction cs00000001_0001

12

32

127

DE_Exporting MARC Bib records workflow Transaction cs00000001_0002

16

51

207

DE_Exporting MARC Bib records workflow Transaction cs00000001_0003

23

73

330

DE_Exporting MARC Bib records workflow Transaction cs00000001_0004

22

100

494

DE_Exporting MARC Bib records workflow Transaction cs00000001_0005

27

122

606

DE_Exporting MARC Bib records workflow Transaction cs00000001_0006

29

131

704

DE_Exporting MARC Bib records workflow Transaction cs00000001_0007

28

154

782

DE_Exporting MARC Bib records workflow Transaction cs00000001_0008

34

170

875

DE_Exporting MARC Bib records workflow Transaction cs00000001_0009

39

191

936

DE_Exporting MARC Bib records workflow Transaction cs00000001_0010

38

216

965

DE_Exporting MARC Bib records workflow Transaction cs00000001_0011

40

228

1022

DE_Exporting MARC Bib records workflow Transaction cs00000001_0012

45

261

1054

DE_Exporting MARC Bib records workflow Transaction cs00000001_0013

45

277

1082

DE_Exporting MARC Bib records workflow Transaction cs00000001_0014

45

293

1108

DE_Exporting MARC Bib records workflow Transaction cs00000001_0015

52

308

1163

DE_Exporting MARC Bib records workflow Transaction cs00000001_0016

51

331

1161

DE_Exporting MARC Bib records workflow Transaction cs00000001_0017

56

372

1183

DE_Exporting MARC Bib records workflow Transaction cs00000001_0018

56

374

1209

DE_Exporting MARC Bib records workflow Transaction cs00000001_0019

63

399

1232

DE_Exporting MARC Bib records workflow Transaction cs00000001_0020

63

419

1254

DE_Exporting MARC Bib records workflow Transaction cs00000001_0021

68

432

1275

DE_Exporting MARC Bib records workflow Transaction cs00000001_0022

69

457

1271

DE_Exporting MARC Bib records workflow Transaction cs00000001_0023

72

479

1290

DE_Exporting MARC Bib records workflow Transaction cs00000001_0024

74

475

1306

DE_Exporting MARC Bib records workflow Transaction cs00000001_0025

74

489

1300

DE_Exporting MARC Bib records workflow Transaction cs00000001_0026

75

506

1319

DE_Exporting MARC Bib records workflow Transaction cs00000001_0027

85

503

1336

DE_Exporting MARC Bib records workflow Transaction cs00000001_0028

80

502

1331

DE_Exporting MARC Bib records workflow Transaction cs00000001_0029

91

535

1347

DE_Exporting MARC Bib records workflow Transaction cs00000001_0030

87

532

1344

DE_Exporting MARC Bib records workflow Transaction cs00000001_0031

87

530

1357

DE_Exporting MARC Bib records workflow Transaction cs00000001_0032

93

539

1349

DE_Exporting MARC Bib records workflow Transaction cs00000001_0033

97

535

1363

DE_Exporting MARC Bib records workflow Transaction cs00000001_0034

97

547

1357

DE_Exporting MARC Bib records workflow Transaction cs00000001_0035

97

560

1373

DE_Exporting MARC Bib records workflow Transaction cs00000001_0036

107

554

1349

DE_Exporting MARC Bib records workflow Transaction cs00000001_0037

102

557

1359

DE_Exporting MARC Bib records workflow Transaction cs00000001_0038

103

563

1370

DE_Exporting MARC Bib records workflow Transaction cs00000001_0039

109

571

1363

DE_Exporting MARC Bib records workflow Transaction cs00000001_0040

104

548

1364

DE_Exporting MARC Bib records workflow Transaction cs00000001_0041

109

552

1365

DE_Exporting MARC Bib records workflow Transaction cs00000001_0042

115

553

1362

DE_Exporting MARC Bib records workflow Transaction cs00000001_0043

119

497

1363

DE_Exporting MARC Bib records workflow Transaction cs00000001_0044

126

555

1363

DE_Exporting MARC Bib records workflow Transaction cs00000001_0045

121

550

1356

DE_Exporting MARC Bib records workflow Transaction cs00000001_0046

126

553

1352

DE_Exporting MARC Bib records workflow Transaction cs00000001_0047

120

544

1352

DE_Exporting MARC Bib records workflow Transaction cs00000001_0048

131

551

1347

DE_Exporting MARC Bib records workflow Transaction cs00000001_0049

126

546

1340

DE_Exporting MARC Bib records workflow Transaction cs00000001_0050

130

538

1336

DE_Exporting MARC Bib records workflow Transaction cs00000001_0051

127

533

1341

DE_Exporting MARC Bib records workflow Transaction cs00000001_0052

128

533

1337

DE_Exporting MARC Bib records workflow Transaction cs00000001_0053

127

529

1334

DE_Exporting MARC Bib records workflow Transaction cs00000001_0054

123

527

1320

DE_Exporting MARC Bib records workflow Transaction cs00000001_0055

122

516

1320

DE_Exporting MARC Bib records workflow Transaction cs00000001_0056

119

516

1316

DE_Exporting MARC Bib records workflow Transaction cs00000001_0057

114

508

1315

DE_Exporting MARC Bib records workflow Transaction cs00000001_0058

110

503

1306

DE_Exporting MARC Bib records workflow Transaction cs00000001_0059

108

501

1302

DE_Exporting MARC Bib records workflow Transaction cs00000001_0060

101

493

1298

DE_Exporting MARC Bib records workflow Transaction cs00000001_0061

98

487

1289

DE_Exporting MARC Bib records workflow Transaction cs00000001_0062

16

31

36

...

Service CPU Utilization

DE_instances T1(500-1K-2k). Service CPU usage during the test was quite low, only the mod-data-export-b okapi-b mod-permissions-b services had a slight increase up to 13%.

...

2024-02-05 19:52 Local time Top 10 CPU-consuming services
mod-consortia-b 19.4813457932
mod-data-export-b 13.0495252091
mod-permissions-b 11.8486188099
mod-pubsub-b 11.3424816303
okapi-b 11.0208864888
mod-inventory-b 8.95271167462
mod-users-b 8.38095871043
mod-configuration-b 8.00652350057
mod-quick-marc-b 6.94299935374


Memory Utilization

DE_instances T1(500-1K-2k). During testing, the memory usage does not increase

...

DB CPU Utilization

DE_instances T1(500-1K-2k). Except for the database spikes that started at 18:15 and repeated every 30 minutes. CPU usage was about 2.5%, during all Data Export of 500, 1000, and 2000 instances, CPU usage increased up to 4%;

...

DB Connections

DE_instances T1(500-1K-2k). Maximal number of DB connections was 1190 for 500 DE; 1270 for 1000DE and 1340 for 2000 DE.

...

DB Load

DE_instances T1(500-1K-2k).

...

SQL queries

DE_instances T1(500-1K-2k).

...

Errors / Additional information

Number of instances on each of the 62 tenant

Expand
titleNumber of instances on each of the 62 tenant

 cs00000001_0001

65359

 cs00000001_0002

22370

 cs00000001_0003

136291

 cs00000001_0004

77785

 cs00000001_0005

110093

 cs00000001_0006

293580

 cs00000001_0007

279493

 cs00000001_0008

160024

 cs00000001_0009

110918

 cs00000001_0010

83254

 cs00000001_0011

94171

 cs00000001_0012

227489

 cs00000001_0013

19131

 cs00000001_0014

20746

 cs00000001_0015

26899

 cs00000001_0016

153333

 cs00000001_0017

16492

 cs00000001_0018

22263

 cs00000001_0019

5312

 cs00000001_0020

44259

 cs00000001_0021

83988

 cs00000001_0022

86900

 cs00000001_0023

86900

 cs00000001_0024

385123

 cs00000001_0025

84215

 cs00000001_0026

153912

 cs00000001_0027

48176

 cs00000001_0028

64556

 cs00000001_0029

5140

 cs00000001_0030

97589

 cs00000001_0031

71895

 cs00000001_0032

80074

 cs00000001_0033

45886

 cs00000001_0034

18240

 cs00000001_0035

135704

 cs00000001_0036

114170

 cs00000001_0037

24226

 cs00000001_0038

225563

 cs00000001_0039

128523

 cs00000001_0040

111992

 cs00000001_0041

61091

 cs00000001_0042

634675

 cs00000001_0043

2831

 cs00000001_0044

70871

 cs00000001_0045

78320

 cs00000001_0046

109677

 cs00000001_0047

48531

 cs00000001_0048

53346

 cs00000001_0049

6952

 cs00000001_0050

190736

 cs00000001_0051

351325

 cs00000001_0052

137940

 cs00000001_0053

61834

 cs00000001_0054

16297

 cs00000001_0055

176009

 cs00000001_0056

160976

 cs00000001_0057

22596

 cs00000001_0058

123819

 cs00000001_0059

37715

 cs00000001_0060

38395

 cs00000001_0061

87117

 cs00000001_0062

1

Methodology/Approach

Infrastructure

PTF -environment mcpt

  • 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1

  • 2 database  instances, writer/reader

    NameMemory GIBvCPUsmax_connections

    db.r6g.xlarge

    32 GiB4 vCPUs2731

  • MSK tenant

    • 4 m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true

    • log.retention.minutes=480

    • default.replication.factor=3

Table contains modules, memory and CPU parameters