Test status: PASSED

Overview

• Regression testing of Check-In/Check-Out (CI/CO) fixed load tests on okapi based environment in Ramsons non-ECS release.

• The purposes of CI/CO testing:

  • To define response times of transaction controllers for Check-In and Check-Out

  • To define the response times for requests that take longer than 100 milliseconds

  • To find any trends for resource utilization and recommend improvements

  • To check how system behaves over extended period during longevity test

  • Compare results (current and previous

Summary

• Common results:

  • CI/CO tests showed stable response times across low loads (8, 20 vUsers), a 50% increase under high load (75 vUsers), moderate degradation during longevity test, no memory leaks and up to 14% (80 ms) degradation in CO and up to 20% (100 ms) in CI 20 vUsers flow compared to Quesnelia release.

• Tests #1, #2, #3, #4

Resources

• CPU utilization

  • Tests #1, #2, #3, #4 and longevity tests used CPU in correspondence with number of vUsers but there are some modules that spiked during 75 vUsers test -  mod-users-b - 132%, mod-authtoken-b - 98%, nginx-okapi - 84%.

• Memory consumption

  • Tests #1, #2, #3, #4 and longevity tests didn't reveal some problems with memory usage by modules. So no memory leaks detected.

• RDS CPU utilization average

  • 8 vUsers - 13%, 20 vUsers - 22%, 30 vUsers - 30%, 75 vUsers - 63% During longevity test CPU grew from 30% to 45%. So it has growing trend during longevity test. The same CPU utilization as it was in quesnelia.

• CPU (User) usage by broker

  • As MSK cluster is linked to all PTF clusters so the time range which can reflect only CI/CO during longevity test (test #5) - from midnight till 7 a.m. Max consumption rate here - 10%. Also we may observe impact of other CI/CO tests - the max consumption rate - 40% for all clusters (tests #1, #2, #3, #4).

Recommendations & Jiras

• The previous results report:

  • Check-in-check-out Test Report (Quesnelia)

• The current ticket: 

  PERF-970 - [Ramsons] [non-ECS] CI/CO In Review

• mod-serials-management-b affect DB connection growth 200 connection in average. Disabling this module do not affect response times or error rate but significantly decrease DB connection number.

• Revisions for modules should have CPU=0 as a default value so this should be changed on module deployment level.

Test Runs 

The following table contains  tests configuration information

Results

Errors:

• Error messages: POST_circulation/check-out-by-barcode (Submit_barcode_checkout)_POST_422. 422/Unprocessable Entity. Happen expectedly if instance was checked out already. Error rate - 0.002% which is acceptable.

Response time

The table contains results of Check-in, Check-out tests in Ramsons release. 

Test #1, #2, #3, #4

Graph (Response times) Test #1

Open

Open

Graph (Response times) Test #2

Open

Open

Graph (Response times) Test #3

Open

Open

Graph (Response times) Test #4

Open

Open

Test #5

Graph (Response times) Test #5

Open

Open

Comparisons

This table has comparison between average values of response times of Ramsons and Quesnelia releases

Comparison of longevity test

API requests where response times >= 100 milliseconds

Resources Utilization

CPU Utilization

During 45 minute tests CPU utilized mostly during high load (75 vUsers) by okapi - 84%, mod-authtoken spiked every 3 minutes from 5 to 30%, mod-inventory-storage - 23%, mod-inventory - 17%, mod-pubsub - 17%, nginx-okapi - 10%, mod-circulation - 10%, mod-circulation-storage - 3%

During longevity CPU utilized mostly by okapi - 37%, mod-authtoken spiked every 3 minutes from 5 to 20%, mod-inventory - 12%, mod-pubsub - 11%, mod-circulation - 5%, mod-circulation-storage - 3%