/
Retrieving instances and holdings - NLA report

Retrieving instances and holdings - NLA report

Jun 15, 2023

Overview

Tests were conducted in scope of PERF-597 task to assess performance of the Collection Service functionality (RIH scenario in JMeter script) and get the baseline numbers for future testing.

Summary

  • There is no difference in performance of functionality between normal and high load tests.

  • There is response time degradation in comparison with tests for all the workflows running at the same time. Details can be found at the Test results part.

  • During load test CPU utilization for nginx-okapi, mod-inventory-storage, pub-okapi, okapi-b and mod-source-record-storage modules increased to max 5%. Rest of the modules CPU utilization didn't change.

  • Service memory utilization was stable, and no memory leaks were suspected during tests.

Test Runs 

Test #

Test type

# configuration

Test duration

Comment

Test #

Test type

# configuration

Test duration

Comment

1

Normal load

RIH scenario with load of 3 requests/second

1 hour

Tests were conducted 2 times each, no differences in results were observed

2

High load

RIH scenario with load of 20 requests/second

1 hour

Test Results

 Test results from 1st test run (1st and 2nd  test run results are similar):

 

 

Request name

Response time normal load

Response time high load

Individual workflow

All the workflows*

All the workflows + DI + FY*

Individual workflow

All the workflows*

All the workflows + FY*

AVG (sec)

95 PCTL (sec)

AVG (sec)

95 PCTL (sec)

AVG (sec)

95 PCTL (sec)

AVG (sec)

95 PCTL (sec)

AVG (sec)

95 PCTL (sec)

AVG (sec)

95 PCTL (sec)

RIH_1GET source-storage/source-records

0.019

0.024

0.020

0.038 (+58%)

0.031

0.066 (+175%)

0.019

0.024

0.023

0.043 (+79%)

0.030

0.047 (+95%)

RIH_2GET holdings-storage/holdings

0.015

0.017

0.015

0.033 (+94%)

19.799**

30.018**(+30 sec)

0.014

0.017

0.019

0.041 (+141%)

0.027

0.049 (+188%)

*previous test results can be found here: Master Script normal load test - NLA report and Master Script high load test - NLA report (without Data Import)

**error rate for this request during the test was 35%

Instance CPU Utilization

normal load

Instance CPU Utilization did not exceed 7,5%.

high load

Instance CPU Utilization did not exceed 8,5%, except 1 spike at the beginning of the test.

Service CPU Utilization

normal load

During load test CPU utilization for nginx-okapi, mod-inventory-storage, pub-okapi and okapi-b modules increased to 1-1,5%. Rest of the modules CPU didn't change.

high load

 

During load test CPU utilization for nginx-okapi, mod-inventory-storage, pub-okapi and okapi-b modules increased to 5%, CPU utilization for mod-source-record-storage increased to 2%. Rest of the modules CPU didn't change.

Service Memory Utilization

normal load

This graph represents memory usage of test runs and shows that no memory leak is suspected for all of the modules.

high load

This graph represents memory usage of test runs and shows that no memory leak is suspected for all of the modules.


DB CPU Utilization

normal load

DB CPU usage was at the same level as without load - about 7,5%.

high load

DB CPU usage increased from 7,2% to 8,5%.

DB Connections

normal load

Connections count didn't change during the test.

high load

Connections count didn't change during the test.

Appendix

Infrastructure

PTF -environment ncp3 

  • 9 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1

  • 2 database  instances, one reader, and one writer

  • MSK ptf-kakfa-3

    • 4 m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true

    • log.retention.minutes=480

    • default.replication.factor=3

  • Kafka topics partitioning: - 2 partitions for DI topics

 

Modules memory and CPU parameters

Modules

Version

Task Definition

Running Tasks 

CPU

Memory

MemoryReservation

MaxMetaspaceSize

Xmx

Modules

Version

Task Definition

Running Tasks 

CPU

Memory

MemoryReservation

MaxMetaspaceSize

Xmx

mod-inventory-storage

26.0.0

1

2

1024

2208

1952

384

1440

mod-inventory

20.0.4

1

2

1024

2880

2592

512

1814

mod-tags

2.0.1

1

2

128

1024

896

128

768

mod-gobi

2.6.0

1

2

128

1024

896

128

700

mod-remote-storage

2.0.2

1

2

1024

4920

4472

512

3960

mod-invoice-storage

5.6.0

1

2

128

1024

896

128

700

edge-sip2

3.0.0

1

2

128

1024

896

128

768

mod-users-bl

7.5.0

1

2

512

1440

1152

128

922

edge-rtac

2.6.0

1

2

128

1024

896

128

768

mod-feesfines

18.2.1

1

2

128

1024

896

128

768

mod-rtac

3.5.0

1

2

128

1024

896

128

768

mod-erm-usage-harvester

4.3.0

1

2

128

1024

896

128

768

mod-search

2.0.1

1

2

400

2592

2480

1024

1440

mod-service-interaction

2.2.2

1

2

256

2048

1844

512

1290

edge-ncip

1.8.1

1

2

128

1024

896

128

768

mod-authtoken

2.13.0

1

2

512

1440

1152

128

922

mod-permissions

6.3.1

2

2

512

1684

1544

512

1024

mod-circulation-storage

16.0.0

1

2

1024

1536

1440

512

896

mod-ncip

1.13.1

1

2

128

1024

896

128

768

mod-pubsub

2.9.1

1

2

1024

1536

1440

512

922

edge-orders

2.8.1

1

2

1024

1536

1440

512

922

mod-circulation

23.5.4

1

2