Check-in-check-out Test Report (Quesnelia)

Owned by Leonid Kolesnykov
Last updated: Apr 24, 2024 by Mykhailo Petryshyn
8 min read

Overview

The test's target is to investigate the behaviour of the Check-In/Check-Out activities during fixed load tests.

PERF-835 - Getting issue details... STATUS

Summary

Check-In/Check-Out tests on Quesnelia carried out with 8, 20, 30, 75 virtual users during 45 minutes and with 30 virtual users during longevity test (24 hours) to detect any trends in resource utilization

45 minute tests

  • Response times in tests with 8, 20, 30 virtual users didn't differ a lot but the lowest response times were with 20 virtual users. Average in 20 vUsers test Check-In - 404 ms, average Check-Out - 729 ms
  • The highest response times results were with 75 virtual users. Average Check-In - 580 ms, Check-Out - 1039 ms

Longevity test

  • Average Check-In - 454 ms, Check-Out - 1060 ms (the results from the test with absent dcb-system-user)
  • No memory leaks during longevity test

Comparison with Poppy Results:

  • CI/CO response times improved by 10%-15% in 8, 20 vUsers and 5% in 30 vUsers tests.
  • For 75 vUsers test Check-in and Check-out response times improved 7% in Quesnelia release.

Resources

  • CPU utilization
    • 45 minute and longevity tests used CPU in correspondence with number of vUsers but there are some modules that spiked during 75 vUsers test -  mod-users-b - 132%, mod-authtoken-b - 98%, nginx-okapi - 84%.
  • Memory consumption
    • 45 minute and longevity tests didn't reveal some problems with memory usage by modules. So no memory leaks detected.
  • RDS CPU utilization average
    • 8 vUsers - 13%, 20 vUsers - 22%, 30 vUsers - 30%, 75 vUsers - 63% During longevity test CPU grew from 30% to 45%. So it has growing trend during longevity test that can be explained by absent dcb-system-user in mod-dcb module.
  • CPU (User) usage by broker
    • Common CPU utilization by broker during all tests was 15% with equal distribution between brokers

Common notes

  • Longevity test revealed CPU Utilization growing trend for mod-dcb-b module.

Recommendations

  • Control mod-dcb-b module related system user creation. For some reason dcb-system-user was not created after environment creation, additional redeploy of module, enabling resolved the issue. It may lead to a huge performance degradation. 
  • Add more CPU to mod-users to exclude module fluctuations
 Summary for tests with absent dcb-system-user

45 minute tests

  • Response times in tests with 8, 20, 30 virtual users didn't differ a lot but the lowest response times were with 8 virtual users. Average Check-In - 470 ms, average Check-Out - 838 ms
  • The highest response times results were with 75 virtual users. Average Check-In - 720 ms, Check-Out - 1301 ms
  • Additional test was carried out with changed configuration for mod-dcb-b (raised from 128 to 256). It helped to decrease CPU utilization for mod-dcb-b twice from 407% to 202% and nginx-okapi from 102% to 70% which helped to decrease response times more than 20% if to compare results with 128 CPU and 256 CPU. For this test average Check-In - 587 ms, Check-Out - 1055 ms. which is better than Poppy release - 5%.

Longevity test

  • Average Check-In - 454 ms, Check-Out - 1060 ms
  • No memory leaks during longevity test

Comparison with Poppy Results:

  • CI/CO response times increased by 4-7% for 20 and 30 vUsers tests.
  • For 75 vUsers test, the increase was 15%.
  • For 75 vUsers test CO response time was 1116 ms in Poppy and 1301 ms in Quesnelia

CPU utilization

  • 45 minute and longevity tests used CPU in correspondence with number of vUsers but there are some modules that spiked during 75 vUsers test -  mod-dcb-b - 407% with 128 CPU, mod-authtoken-b - 120%, nginx-okapi - 102%.
  • Additional test was carried out with changed configuration for mod-dcb-b (raised from 128 to 256). It helped to decrease CPU utilization for mod-dcb-b twice from 407% to 202% and nginx-okapi from 102% to 70% which helped to decrease response times more than 20% if to compare results with 128 CPU and 256 CPU.

Memory consumption

  • 45 minute and longevity tests didn't reveal some problems with memory usage by modules. So no memory leaks detected.

RDS CPU utilization

  • 8 vUsers - 13%, 20 vUsers - 22%, 30 vUsers - 30%, 75 vUsers - 50% During longevity test CPU grew from 30% to 45%. So it has growing trend during longevity test.

CPU (User) usage by broker

  • Common CPU utilization by broker during all tests was 15% with equal distribution between brokers

Common notes

  • Longevity test revealed CPU Utilization growing trend for mod-dcb-b module.

Recommendations

  • Investigate mod-dcb-b module behaviour. After 75 vUsers test modules require some time to stabilize CPU consumption.

Test Runs 

The following table contains  tests configuration information

Test #

vUsersRamp-up, sec

Duration, sec 

1

8802700
2202002700
3303002700
4757502700
53030086400

Results

Errors:

  • Error messages: POST_circulation/check-out-by-barcode (Submit_barcode_checkout)_POST_422. 422/Unprocessable Entity. Happen expectedly if instance was checked out already. Error rate - 0.06% which is acceptable.

Response time

The table contains results of Check-in, Check-out tests in Quesnelia release. 

45 minute tests


8 vUsers  20 vUsers  30 vUsers  75 vUsers  
Requests

Response Times (ms)

 Response Times (ms) Response Times (ms) Response Times (ms)
Label

#Samples

Average

95th pct

#Samples

Average

95th pct

#Samples

Average

95th pct

#Samples

Average

95th pct

Check-Out Controller1857741.82904.14614729.268596637767.83935146371039.211323
Check-In Controller1394408.24843378404.244755063427.2252711136580.9752
POST_circulation/check-out-by-barcode (Submit_barcode_checkout)1858256.343374615247.442976639252.3531014642343.5457
POST_circulation/check-in-by-barcode (Submit_barcode_checkin)1395172.66225.23382170.312055080174.75224.9511159242.61323
GET_circulation/loans (Submit_barcode_checkout)1856136.69185.154613133.161616636138.4617614628186.43259

Longevity test (with absent dcb-system-user)

 30 vUsers Longevity 


RequestsSamples, Response Times 

Label#Samples95th pctAverage
Check-Out Controller13559112471065
Check-In Controller101990525454
POST_circulation/check-out-by-barcode (Submit_barcode_checkout)135593410340
POST_circulation/check-in-by-barcode (Submit_barcode_checkin)101981228193

Comparisons

This table has comparison between average values of response times of Quesnelia and Poppy releases


8 vUsers   20 vUsers   30 vUsers   75 vUsers   
RequestsResponse Times   Response Times   Response Times   Response Times   

PoppyQuesnelia

PoppyQuesnelia

PoppyQuesnelia

PoppyQuesnelia

LabelAverageDelta,msDifference,%AverageDelta,msDifference,%AverageDelta,msDifference,%AverageDelta,msDifference,%
Check-Out Controller832741-91-10.94%805729-76-9.44%807767-40-4.96%11161039-77-6.90%
Check-In Controller479408-71-14.82%450404-46-10.22%449427-22-4.90%622580-42-6.75%

Comparison of Longevity test (absent dcb-system-user in Quesnelia)

30 vUsers Longevity  
Response Times 

PoppyQuesnelia

AverageAverageDelta,msDifference,%
10251065403.90%
471454-17-3.61%


API requests where response times >= 100 ms

API

30 vUsers  Poppy

Average, ms

30 vUsers  Quesnelia

Average, ms

POST checkout-by-barcode

281

252
POST checkin-by-barcode 

173

175
GET circulation/loans

135

138
 Results for tests with absent dcb-system-user

Response time

The table contains results of Check-in, Check-out tests in Quesnelia release. 

45 minute tests


8 vUsers  20 vUsers  30 vUsers  75 vUsers  

Requests

Samples, Response Times

Samples, Response Times

Samples, Response Times Samples, Response Times

Label

#Samples

95th pct

Average

#Samples

95th pct

Average

#Samples

95th pct

Average

#Samples

95th pct

Average

Check-Out Controller17851007.7837.944576993842.6564961020870.21437918761301.92
Check-In Controller1437554470.23285550.7467.715045562480.62106071047720.2
POST_circulation/check-out-by-barcode (Submit_barcode_checkout)1785378290.324577334282.216499342288.0714383665426.66
POST_circulation/check-in-by-barcode (Submit_barcode_checkin)1439266207.483293237195.475056235200.1210629464295.4

Longevity test

 30 vUsers Longevity 


RequestsSamples, Response Times 

Label#Samples95th pctAverage
Check-Out Controller13559112471065
Check-In Controller101990525454
POST_circulation/check-out-by-barcode (Submit_barcode_checkout)135593410340
POST_circulation/check-in-by-barcode (Submit_barcode_checkin)101981228193

Comparisons

This table has comparison between average values of response times of Quesnelia and Poppy releases


8 vUsers   20 vUsers   30 vUsers   75 vUsers   
RequestsResponse Times 

Response Times 

Response Times 

Response Times 


PoppyQuesnelia

PoppyQuesnelia

PoppyQuesnelia

PoppyQuesnelia

LabelAverageDelta,msDifference,%AverageDelta,msDifference,%AverageDelta,msDifference,%AverageDelta,msDifference,%
Check-Out Controller832837.945.940.71%805842.6537.654.68%807870.263.27.83%11161301.92185.9216.66%
Check-In Controller479470.2-8.8-1.84%450467.7117.713.94%449480.6231.627.04%622720.298.215.79%

Comparison table for tests after mod-dcb-b CPU 128 as default raised to 256 (Task definition: revision: qcp1-mod-dcb:3). Additional test without mod-dcb-b (# of tasks = 0) 


75 vUsers mod-dcb-b CPU 12875 vUsers mod-dcb-b CPU 25675 vUsers mod-dcb-b - Tasks = 0
RequestsResponse Times 

Response Times 

Response Times 


PoppyQuesnelia

PoppyQuesnelia

PoppyQuesnelia

LabelAverageAverageDelta,msDifference,%AverageAverageDelta,msDifference,%AverageAverageDelta,msDifference,%
Check-Out Controller11161301.92185.9216.66%11161055-61-5.47%1116111710.09%
Check-In Controller622720.298.215.79%622587-35-5.63%62262971.13%


Comparison of Longevity test

30 vUsers Longevity  
Response Times 

PoppyQuesnelia

AverageAverageDelta,msDifference,%
10251065403.90%
471454-17-3.61%


API requests where response times >= 100 ms

API

30 vUsers  Poppy

Average, ms

30 vUsers  Quesnelia

Average, ms

POST checkout-by-barcode

281

288
POST checkin-by-barcode 

173

200
GET circulation/loans

135

158

Resources Utilization

CPU Utilization

45 minute and longevity tests used CPU in correspondence with number of vUsers but there are some modules that spiked during 75 vUsers test -  mod-users-b - 132%, mod-authtoken-b - 98%, nginx-okapi - 84%.
45 minute tests

 CPU utilization by modules
Service NameCPU 8 vUsersCPU 20 vUsersCPU 30 vUsersCPU 75 vUsers
mod-users-b21.0838.9560.75132.25
mod-inventory-b14.7516.8418.2318.04
okapi-b14.5433.3748.8682.97
mod-configuration-b11.7424.436.5879.42
mod-authtoken-b9.8923.4331.3798.15
mod-pubsub-b7.889.0811.2515.37
mod-dcb-b7.7611.2510.3316.89
mod-feesfines-b7.6614.7121.5447.69
mod-quick-marc-b6.685.966.136.41
mod-data-import-b4.074.224.174.67
pub-okapi3.949.3313.3232.52
mod-inventory-storage-b3.647.5911.9314.2
mod-oa-b2.692.412.463.58
nginx-okapi2.285.037.9984.13
mod-di-converter-storage-b2.232.372.422.51
mod-patron-b2.222.382.382.95
mod-source-record-manager-b1.921.891.911.98
mod-source-record-storage-b1.861.881.911.95
mod-circulation-storage-b1.492.563.527.36
mod-search-b1.481.173.641.25
mod-circulation-b1.333.014.029.26
edge-patron-b0.9711.061.11
mod-patron-blocks-b0.841.411.983.47
mod-users-bl-b0.440.450.450.5
mod-entities-links-b0.390.320.350.37
pub-edge0.080.090.090.09

Longevity test (with absent dcb-system-user)

Memory Consumption

45 minute and longevity tests didn't reveal some problems with memory usage by modules. So no memory leaks detected. Modules that consumed max memory - mod-search - 94%, mod-oa - 78%, mod-inventory - 72%, mod-dcb - 71%

 Memory usage by modules
Service NameMemory 75 vUsers
mod-search-b94.2
mod-oa-b78.38
mod-inventory-b72.3
mod-dcb-b70.98
mod-users-b63.74
okapi-b48.33
mod-pubsub-b47.6
mod-source-record-storage-b45.07
mod-data-import-b40.66
mod-di-converter-storage-b39.51
mod-source-record-manager-b35.98
mod-inventory-storage-b35.84
mod-feesfines-b34.69
mod-quick-marc-b31.16
mod-configuration-b30.76
mod-patron-blocks-b30.69
mod-users-bl-b30.24
mod-circulation-b29.17
mod-entities-links-b26.99
mod-patron-b24
mod-authtoken-b21.89
mod-circulation-storage-b21.31
edge-patron-b18.19
nginx-okapi5.3
pub-okapi4.63
pub-edge4.46

45 minute tests

Longevity test

RDS CPU Utilization

RDS CPU utilized:

8 vUsers - 13%, 20 vUsers - 22%, 30 vUsers - 30%, 75 vUsers - 50% During longevity test CPU grew from 30% to 45%. So it has growing trend during longevity test.

45 minute tests

Longevity test

RDS Database Connections

For both 45 minute and longevity tests RDS used max 615-630 connections. Without test it was 550 connections.

45 minute tests

Longevity test

CPU (User) usage by broker

Common CPU utilization by broker during all tests was 15% with equal distribution between brokers

45 minute tests

Longevity test

Database load


75 vUsers

 Resources Utilization

CPU Utilization

45 minute and longevity tests used CPU in correspondence with number of vUsers but there are some modules that spiked during 75 vUsers test -  mod-dcb-b - 407% with 128 CPU, mod-authtoken-b - 120%, nginx-okapi - 102%.
Additional test was carried out with changed configuration for mod-dcb-b (raised from 128 to 256). It helped to decrease CPU utilization for mod-dcb-b twice from 407% to 202% and nginx-okapi from 102% to 70% which helped to decrease response times more than 20% if to compare results with 128 CPU and 256 CPU.

45 minute tests

 CPU utilization by modules
Service NameCPU 8 vUsersCPU 20 vUsersCPU 30 vUsersCPU 75 vUsers, 128CPU 30 vUsers LongevityCPU 75 vUsers, 256
mod-users-b29.3165.1189.61206.6396.57199.92
mod-dcb-b21.5346.2569.61407.79155.82201.69
okapi-b16.5238.8555.6690.656.7793
mod-inventory-b14.2818.1717.9715.1415.5815.77
mod-authtoken-b12.0820.8132.34120.6230.98115.95
mod-configuration-b10.7321.9833.9571.3330.0270.5
mod-pubsub-b9.1711.6713.6118.0912.3926.5
mod-feesfines-b8.2617.123.8854.7928.8754.37
mod-quick-marc-b7.27.637.578.197.937.88
pub-okapi3.919.4513.9933.4412.4727.68
mod-inventory-storage-b3.447.6511.9913.2410.6712.16
nginx-okapi2.566.359.35102.627.6969.58
mod-patron-b2.312.522.522.32.282.44
mod-oa-b2.282.172.672.753.173.52
mod-di-converter-storage-b2.22.192.292.392.121.97
mod-source-record-manager-b1.931.91.871.881.811.95
mod-data-import-b1.71.021.020.961.571.59
mod-source-record-storage-b1.681.721.731.771.761.83
mod-circulation-storage-b1.462.593.647.433.537.06
mod-circulation-b1.463.544.5810.255.6211.21
mod-search-b1.212.662.343.282.943.94
edge-patron-b0.920.9611.060.950.97
mod-patron-blocks-b0.751.381.82.742.382.97
mod-users-bl-b0.510.540.530.50.580.66
mod-entities-links-b0.350.360.440.420.40.39
pub-edge0.140.060.10.080.070.11

45 minute test with 75 vUsers (revision with 256 CPU for mod-dcb-b)

Longevity test

Memory Consumption

45 minute and longevity tests didn't reveal some problems with memory usage by modules. So no memory leaks detected.

 Memory usage by modules
Service NameMemory 75 vUsersMemory 30 vUsers Longevity
mod-oa-b8179.24
mod-dcb-b68.6369.16
mod-users-b63.6562.98
mod-search-b55.3440.83
mod-pubsub-b54.7961.98
okapi-b48.9460.38
mod-inventory-b47.4950.79
mod-source-record-manager-b45.0645.76
mod-quick-marc-b43.5129.92
mod-di-converter-storage-b35.0229.58
mod-feesfines-b34.1834.12
mod-source-record-storage-b33.7631.37
mod-inventory-storage-b33.1240.06
mod-entities-links-b32.325.08
mod-configuration-b32.0730.21
mod-data-import-b31.3139.73
mod-patron-blocks-b31.1330.92
mod-users-bl-b28.9525.82
mod-circulation-b27.4126.71
mod-patron-b26.6225.17
mod-circulation-storage-b26.4328.83
mod-authtoken-b21.8422.1
edge-patron-b18.318.14
nginx-okapi4.854.8
pub-okapi4.694.69
pub-edge4.584.63

45 minute tests

Longevity test

RDS CPU Utilization

RDS CPU utilized:

8 vUsers - 13%, 20 vUsers - 22%, 30 vUsers - 30%, 75 vUsers - 50% During longevity test CPU grew from 30% to 45%. So it has growing trend during longevity test.

45 minute tests

Longevity test

RDS Database Connections

For both 45 minute and longevity tests RDS used max 615 connections. Without test it was 550 connections.

45 minute tests

Longevity test

CPU (User) usage by broker

Common CPU utilization by broker during all tests was 15% with equal distribution between brokers

45 minute tests

Longevity test

Database load

Appendix

Infrastructure

PTF -environment qcp1
  • 11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 
  • 1 instance of db.r6.xlarge database instance: Writer instance
  • MSK tenant [ kafka configurations]
    • 4 kafka.m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true
    • log.retention.minutes=480
    • default.replication.factor=3

DB table records size:

  • fs09000000
    • instances -            24493326
    • items -                  25662862
    • holding records - 24459683

Modules

 Check-In / Check-Out related modules
ModuleTask Def. RevisionModule VersionTask CountMem Hard LimitMem Soft limitCPU unitsXmxMetaspaceSizeMaxMetaspaceSize
qcp1-pvt








Mon Mar 25 12:41:11 UTC 2024








mod-data-import4mod-data-import:3.1.0-SNAPSHOT.1741204818442561292384512
mod-search2mod-search:3.2.0-SNAPSHOT.199225922480204814405121024
mod-configuration2mod-configuration:5.11.0-SNAPSHOT.3482102489612876888128
mod-entities-links3mod-entities-links:3.0.0-SNAPSHOT.782259224802096144001024
pub-edge2pub-edge:2023.06.142102489612876800
edge-patron2edge-patron:5.1.0-SNAPSHOT.1532102489625676888128
mod-users-bl2mod-users-bl:7.6.1-SNAPSHOT.28221440115251292288128
mod-inventory-storage2mod-inventory-storage:27.1.0-SNAPSHOT.71924096369020483076384512
mod-rtac2mod-rtac:3.6.0-SNAPSHOT.1662102489612876888128
mod-circulation-storage2mod-circulation-storage:17.2.0-SNAPSHOT.38422880259215361814384512
mod-circulation-item2mod-circulation-item:1.0.0-SNAPSHOT.162102489612876888128
mod-pubsub2mod-pubsub:2.13.0-SNAPSHOT.1962153614401024922384512
mod-di-converter-storage2mod-di-converter-storage:2.2.0-SNAPSHOT.1082102489612876888128
mod-patron-blocks2mod-patron-blocks:1.10.0-SNAPSHOT.15221024896102476888128
mod-source-record-manager2mod-source-record-manager:3.8.0-SNAPSHOT.30625600500020483500384512
mod-quick-marc2mod-quick-marc:5.1.0-SNAPSHOT.1341228821761281664384512
nginx-okapi2nginx-okapi:2023.06.14210248961024000
okapi-b2okapi:5.1.23168414401024922384512
mod-feesfines2mod-feesfines:19.1.0-SNAPSHOT.1892102489612876888128
mod-dcb2mod-dcb:1.1.0-SNAPSHOT.182102489612876888128
mod-oai-pmh2mod-oai-pmh:3.13.0-SNAPSHOT.39324096369020483076384512
pub-okapi2pub-okapi:2023.06.142102489612876800
 All modules
ModuleTask Def. RevisionModule VersionTask CountMem Hard LimitMem Soft limitCPU unitsXmxMetaspaceSizeMaxMetaspaceSize
qcp1-pvt








Mon Mar 25 12:41:11 UTC 2024








mod-remote-storage2mod-remote-storage:3.2.0-SNAPSHOT.16224920447210243960512512
mod-ncip2mod-ncip:1.14.5-SNAPSHOT.2242102489612876888128
mod-finance-storage2mod-finance-storage:8.6.0-SNAPSHOT.15921024896102470088128
mod-agreements2mod-agreements:6.1.0-SNAPSHOT.195215921488128000
mod-ebsconet2mod-ebsconet:2.2.0-SNAPSHOT.74212481024128700128256
mod-organizations2mod-organizations:1.9.0-SNAPSHOT.842102489612876888128
edge-sip22edge-sip2:3.2.0-SNAPSHOT.2092102489612876888128
mod-serials-management2mod-serials-management:1.0.0-SNAPSHOT.252248023121281792384512
mod-settings2mod-settings:1.0.3-SNAPSHOT.632102489620076888128
mod-data-import4mod-data-import:3.1.0-SNAPSHOT.1741204818442561292384512
edge-dematic2edge-dematic:2.1.2-SNAPSHOT.1301102489612876888128
mod-search2mod-search:3.2.0-SNAPSHOT.199225922480204814405121024
mod-inn-reach2mod-inn-reach:3.2.0-SNAPSHOT.86236003240102428805121024
mod-tags2mod-tags:2.1.1-SNAPSHOT.1322102489612876888128
edge-courses2edge-courses:1.3.2-SNAPSHOT.11382102489612876888128
mod-authtoken2mod-authtoken:2.15.0-SNAPSHOT.29021440115251292288128
mod-inventory-update2mod-inventory-update:3.2.2-SNAPSHOT.822102489612876888128
mod-notify2mod-notify:3.1.1-SNAPSHOT.2622102489612876888128
mod-configuration2mod-configuration:5.11.0-SNAPSHOT.3482102489612876888128
mod-orders-storage2mod-orders-storage:13.7.0-SNAPSHOT.2152102489651270088128
edge-caiasoft2edge-caiasoft:2.1.1-SNAPSHOT.822102489612876888128
mod-login-saml2mod-login-saml:2.8.0-SNAPSHOT.2802102489612876888128
mod-erm-usage-harvester2mod-erm-usage-harvester:4.5.0-SNAPSHOT.1232102489612876888128
mod-licenses2mod-licenses:5.1.0-SNAPSHOT.1472248023121281792384512
mod-gobi2mod-gobi:2.8.0-SNAPSHOT.2442102489612876888128
mod-password-validator2mod-password-validator:3.1.1-SNAPSHOT.137214401298128768384512
mod-bulk-operations2mod-bulk-operations:2.0.0-SNAPSHOT.8123072260010241536384512
mod-fqm-manager2mod-fqm-manager:1.1.0-SNAPSHOT.10832102489612876888128
edge-dcb2edge-dcb:1.1.0-SNAPSHOT.162102489612876888128
mod-graphql2mod-graphql:1.12.1.4012102489612876888128
mod-finance2mod-finance:4.9.0-SNAPSHOT.1602102489612876888128
mod-erm-usage2mod-erm-usage:4.7.0-SNAPSHOT.992102489612876888128
mod-batch-print3mod-batch-print:1.2.0-SNAPSHOT.102102489612876888128
mod-copycat2mod-copycat:1.6.0-SNAPSHOT.1972102489612876888128
mod-lists2mod-lists:1.1.0-SNAPSHOT.12662102489612876888128
mod-entities-links3mod-entities-links:3.0.0-SNAPSHOT.782259224802096144001024
mod-permissions4mod-permissions:6.5.0-SNAPSHOT.3742168415445121024384512
pub-edge2pub-edge:2023.06.142102489612876800
mod-orders2mod-orders:12.8.0-SNAPSHOT.33222048144010241024384512
edge-patron2edge-patron:5.1.0-SNAPSHOT.1532102489625676888128
edge-ncip2edge-ncip:1.9.3-SNAPSHOT.1032102489612876888128
edge-inn-reach2edge-inn-reach:3.1.1-SNAPSHOT.452102489612876888128
mod-users-bl2mod-users-bl:7.6.1-SNAPSHOT.28221440115251292288128
mod-oa2mod-oa:2.1.0-SNAPSHOT.622102489612876888128
mod-inventory-storage2mod-inventory-storage:27.1.0-SNAPSHOT.71924096369020483076384512
mod-invoice3mod-invoice:5.8.0-SNAPSHOT.18721440115251292288128
mod-user-import2mod-user-import:3.9.0-SNAPSHOT.1922102489612876888128
mod-sender3mod-sender:1.11.1-SNAPSHOT.942102489612876888128
edge-oai-pmh2edge-oai-pmh:2.8.0-SNAPSHOT.15521512136010241440384512
mod-data-export-worker2mod-data-export-worker:3.2.0-SNAPSHOT.19523072280010242048384512
mod-rtac2mod-rtac:3.6.0-SNAPSHOT.1662102489612876888128
mod-circulation-storage2mod-circulation-storage:17.2.0-SNAPSHOT.38422880259215361814384512
mod-calendar2mod-calendar:2.6.0-SNAPSHOT.2182102489612876888128
mod-source-record-storage2mod-source-record-storage:5.8.0-SNAPSHOT.35325600500020483500384512
mod-event-config2mod-event-config:2.6.1-SNAPSHOT.1002102489612876888128
mod-courses2mod-courses:1.4.9-SNAPSHOT.1432102489612876888128
mod-circulation-item2mod-circulation-item:1.0.0-SNAPSHOT.162102489612876888128
mod-inventory2mod-inventory:20.2.0-SNAPSHOT.50122880259210241814384512
mod-email2mod-email:1.16.1-SNAPSHOT.1222102489612876888128
mod-pubsub2mod-pubsub:2.13.0-SNAPSHOT.1962153614401024922384512
mod-circulation2mod-circulation:24.2.0-SNAPSHOT.64922880259215361814384512
mod-di-converter-storage2mod-di-converter-storage:2.2.0-SNAPSHOT.1082102489612876888128
edge-rtac2edge-rtac:2.7.0-SNAPSHOT.2122102489612876888128
edge-orders2edge-orders:2.10.0-SNAPSHOT.1232102489612876888128
mod-users2mod-users:19.3.0-SNAPSHOT.6792102489612876888128
mod-template-engine2mod-template-engine:1.19.2-SNAPSHOT.1112102489612876888128
mod-patron-blocks2mod-patron-blocks:1.10.0-SNAPSHOT.15221024896102476888128
mod-audit2mod-audit:2.9.0-SNAPSHOT.2072102489612876888128
edge-fqm2edge-fqm:1.1.0-SNAPSHOT.842102489612876888128
mod-source-record-manager2mod-source-record-manager:3.8.0-SNAPSHOT.30625600500020483500384512
nginx-edge2nginx-edge:2023.06.1421024896128000
mod-quick-marc2mod-quick-marc:5.1.0-SNAPSHOT.1341228821761281664384512
nginx-okapi2nginx-okapi:2023.06.14210248961024000
okapi-b2okapi:5.1.23168414401024922384512
mod-feesfines2mod-feesfines:19.1.0-SNAPSHOT.1892102489612876888128
mod-invoice-storage2mod-invoice-storage:5.8.0-SNAPSHOT.12321872153610241024384512
mod-dcb2mod-dcb:1.1.0-SNAPSHOT.182102489612876888128
mod-service-interaction2mod-service-interaction:3.1.0-SNAPSHOT.1172204818442561290384512
mod-patron2mod-patron:6.1.0-SNAPSHOT.1472102489612876888128
mod-oai-pmh2mod-oai-pmh:3.13.0-SNAPSHOT.39324096369020483076384512
edge-connexion2edge-connexion:1.2.0-SNAPSHOT.722102489612876888128
mod-kb-ebsco-java2mod-kb-ebsco-java:4.1.0-SNAPSHOT.1622102489612876888128
mod-notes2mod-notes:5.2.0-SNAPSHOT.30621024896128952384512
mod-data-export-spring2mod-data-export-spring:3.2.0-SNAPSHOT.1491204818442561536384512
mod-organizations-storage2mod-organizations-storage:4.7.0-SNAPSHOT.1252102489612876888128
mod-login2mod-login:7.11.0-SNAPSHOT.3142144012981024768384512
pub-okapi2pub-okapi:2023.06.142102489612876800
mod-eusage-reports2mod-eusage-reports:2.1.0-SNAPSHOT.652102489612876888128


Methodology/Approach

  • Run data preparation script before each CI/CO test
  • Update .jmx file script for Quesnelia release
  • Create  artefact and upload to carrier-io
  • Use Jenkins job to change parameters and run tests
  • Test CI/CO with 8, 20, 30, 75 concurrent users for 45 minutes each. 
  • Test CI/CO with 30 users for 24 hours to detect any trends in memory.