/
EBSCONet Orders Renewal (Nolana)

EBSCONet Orders Renewal (Nolana)

Feb 14, 2023

Overview

This is initial report for EBSCONET orders renewal workflow testing https://folio-org.atlassian.net/browse/PERF-285. The purpose of this document is to highlight KPIs of EBSCONET workflow, find possible bottlenecks/ issues, and define the baseline performance for this workflow since it hasn't been tested before. EBSCONet is a client application that calls edge-orders to renew orders. The tests simulate EBSCONet calling edge-orders by issuing calls to retrieve a Purchase Order Line (POL) and updating it with relevant information for renewal.

Summary

During testing we've find several issues: 

  • NullPointerException

  • Internal server error

No memory leaks were found. 

High level results:

Test #

VUsers

Duration

response time (get) ms.

response time (put) ms

Error rate

avg

75%

95%

avg

75%

95%

1

2

7 min 10 s

249 

257

287

178

184

197

2.6% (52 calls)

2

2

7 min

247

254

288

178

184

197

3.3% (67 calls)

3

10

4 min 30 s

298

319

404

236

253

339

1.14% (57 calls)

4

10

4 min 10 s

308

332

417

242

263

465

1.56% (78 calls)

 

Recommendations & Jiras

Test Runs 

Test #

VUsers

Data set

Load generator size 

Load generator Memory(GiB) 

  1. normal conditions

2 users

  • Number of order lines being renewed 1000

  • Number of orders being renewed 804

  • 4 of these orders have 50 POLs each - 800 orders with 1 POL each

 

t3.medium

3

2. normal conditions

2 users

  • Number of order lines being renewed 1000

  • Number of orders being renewed 804

  • 4 of these orders have 50 POLs each - 800 orders with 1 POL each

 

t3.medium

3

3. extreme conditions

10 users

  • Number of order lines being renewed 5000

  • Number of orders being renewed 4804

  • 4 of these orders have 50 POLs each - 4800 orders with 1 POL each

t3.medium

3

4 extreme conditions

10 users

  • Number of order lines being renewed 5000

  • Number of orders being renewed 4804

  • 4 of these orders have 50 POLs each - 4800 orders with 1 POL each

t3.medium

3

Results

 

Test #

VUsers

Duration

response time (get) ms.

response time (put) ms

Error rate

avg

75%

95%

avg

75%

95%

1

2

7 min 10 s

249 

257

287

178

184

197

2.6% (52 calls)

2

2

7 min

247

254

288

178

184

197

3.3% (67 calls)

3

10

4 min 30 s

298

319

404

236

253

339

1.14% (57 calls)

4

10

4 min 10 s

308

332

417

242

263

465

1.56% (78 calls)

 

Memory Utilization

 

 

Nolana Avg%

 

Nolana Avg%

mod-organizations

25%

mod-organizations-storage

24%

mod-finance

29%

mod-finance-storage

28%

nginx-edge

2%

mod-ebsconet

37%

nginx-okapi

3%

mod-orders

44%

mod-orders-storage

34%

okapi

36%

mod-notes

43%

mod-configuration 

25%

edge-orders

20%

CPU Utilization 

Most used module - mod-orders-storage. During tests with 10 users it reaches 320% of CPU usage

 

 

RDS CPU Utilization 

Appendix

Infrastructure

PTF -environment ncp3-pvt

  • 9 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 

  • 2 instances of db.r6.xlarge database instances, one reader, and one writer

  • MSK ptf-kakfa-3 

    • 4 m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true

    • log.retention.minutes=480

    • default.replication.factor=3

 

Modules memory and CPU parameter

Modules

Version

Task Definition

Running Tasks 

CPU

Memory

MemoryReservation

MaxMetaspaceSize

Xmx

Modules

Version

Task Definition

Running Tasks 

CPU

Memory

MemoryReservation

MaxMetaspaceSize

Xmx

mod-organizations

1.6.0

2

2

128

1024

896

128

700

mod-organizations-storage

4.4.0

2

2

128

1024

896

128

700

mod-finance

4.6.2

2

2

128

1024

896

128

700

mod-finance-storage

8.3.1

2

2

128

1024

896

128

700

nginx-edge

nginx-edge:2022.03.02

1

2

128

1024

896

N/A

N/A

mod-ebsconet

1.4.0

2

2

128

1024

896

256

700

nginx-okapi

nginx-okapi:2022.03.02

1

2

128

1024

896

N/A

N/A

mod-orders

12.5.4

2

2

1024

2048

1440

256

896

mod-orders-storage

13.4.0

2

2

128

1024

896

128

700

okapi

4.14.7

1

3

1024

1684

1440

922

922

mod-mod-notes

4.0.0

2

2

128

1024

896

128

322

mod-configuration

5.9.0