Test status: PASSED

Overview

• Regression testing of Check-In/Check-Out (CI/CO) fixed load tests on eureka based environment in Ramsons non-ECS release.

• Test was carried out on central tenant.

• The purposes of CI/CO testing:

  • To test how different load from CI/CO flow affect the system

  • To define response times of Check-In and Check-Out

  • To define the response times for requests that take longer than 100 milliseconds

  • To find any trends for resource utilization and recommend improvements

  • To check how system behaves over extended period during longevity test (SSO Session Max limit in keycloak is 10 hours)

  • Compare results (current and previous)

  • The current ticket: 

    PERF-1065 - [Ramsons] [non-ECS] [Eureka] CI/CO

Summary

• Common results:

  • The Ramsons release shows good performance up to 30 virtual users (vUsers) without notable performance degradation. The significant reduction in RTs across all tests observed compared to Quesnelia. The slight RT increase after 5 hours in the CO flow in longevity test should be investigated.

• Tests #1, #2, #3, #4

•  

  • Average response times (RT) in tests with 8, 20, 30 vUsers were the same in average (delta - 80 ms.). Average RT in 20 vUsers test CI - 480 ms, CO - 862 ms.

  • Average RT in test with 75 vUsers response times grew +40% compared to 20 vUsers. Average RT in 75 vUsers test CI - 933 ms, CO - 1472 ms.

• Test #5

•  

  • Average RT in test #5 with 30 vUsers. CI - 442 ms, CO - 971 ms.. There's no expected degradation during 10 hours test in Average. But we can observe RT growing after 5 hours of test + 200 ms in CO flow.

  • No visible memory leaks found. Accumulating affect in longevity test sh

• Comparison with Quesnelia results:

  • CI RT decreased 17% for 8 vUsers, 25% - 20 vUsers, 33% - 30 vUsers, 79% - 75 vUsers.

  • CO RT decreased 36% for 8 vUsers, 36% - 20 vUsers, 42% - 30 vUsers, 88% - 75 vUsers

  • Longevity test: CI RT decreased 66%, CO RT decreased 78%

Resources

• CPU utilization

  • CPU utilization depends on number of concurrent virtual users and increasing accordingly. Top used modules (75 vUsers): mod-remote-storage - 102%, mod-feesfines - 55%, mod-roles-keycloak - 51%, mod-inventory-storage - 48%, mod-users - 45%

• Memory consumption

  • Top module memory usage:  folio-keycloak - 83%,  mod-agreements - 73% in 75 vUsers test. Folio-keycloak  used 90% in longevity. 

• RDS CPU utilization average

  • RDS CPU utilized in average: 8 vUsers - 15%, 20 vUsers - 23%, 30 vUsers - 30%, 75 vUsers - 66% During longevity test CPU was growing from 32% to 42%.

• CPU (User) usage by broker

  • 8 vUsers - 15%, 20 vUsers - 16%, 30 vUsers - 17%, 75 vUsers - 22%. During Longevity test - 17%

Recommendations & Jiras

• To avoid Internal server errors during testing on eureka environment new values were applied for mod-users-keycloak and mod-users-keycloak - Sidecar 1. 

Test Runs 

The following table contains  tests configuration information

Results

Response time

The table contains results of Check-in, Check-out tests in Ramsons release (with modified mod-users-keycloak revision). 

Test #1, #2, #3, #4

Open

Graph (Response times) Test #1

Open

Open

Graph (Response times) Test #2

Open

Open

Graph (Response times) Test #3

Open

Open

Graph (Response times) Test #4

Open

Open

Open

This table has comparison between average values of response times of Ramsons and Quesnelia releases (default mod-users-keycloak values)

This table has comparison between average values of response times of Ramsons and Quesnelia releases (modified mod-users-keycloak values)

Test #5

Graph (Response times) Test #5

Open