Overview

The test's target is to investigate the behaviour of the Check-In/Check-Out activities during fixed load tests.

https://folio-org.atlassian.net/browse/PERF-835

Summary

Check-In/Check-Out tests on Quesnelia carried out with 8, 20, 30, 75 virtual users during 45 minutes and with 30 virtual users during longevity test (24 hours) to detect any trends in resource utilization

45 minute tests

• Response times in tests with 8, 20, 30 virtual users didn't differ a lot but the lowest response times were with 20 virtual users. Average in 20 vUsers test Check-In - 404 ms, average Check-Out - 729 ms

• The highest response times results were with 75 virtual users. Average Check-In - 580 ms, Check-Out - 1039 ms

Longevity test

• Average Check-In - 454 ms, Check-Out - 1060 ms (the results from the test with absent dcb-system-user)

• No memory leaks during longevity test

Comparison with Poppy Results:

• CI/CO response times improved by 10%-15% in 8, 20 vUsers and 5% in 30 vUsers tests.

• For 75 vUsers test Check-in and Check-out response times improved 7% in Quesnelia release.

Resources

• CPU utilization

  • 45 minute and longevity tests used CPU in correspondence with number of vUsers but there are some modules that spiked during 75 vUsers test -  mod-users-b - 132%, mod-authtoken-b - 98%, nginx-okapi - 84%.

• Memory consumption

  • 45 minute and longevity tests didn't reveal some problems with memory usage by modules. So no memory leaks detected.

• RDS CPU utilization average

  • 8 vUsers - 13%, 20 vUsers - 22%, 30 vUsers - 30%, 75 vUsers - 63% During longevity test CPU grew from 30% to 45%. So it has growing trend during longevity test that can be explained by absent dcb-system-user in mod-dcb module.

• CPU (User) usage by broker

  • Common CPU utilization by broker during all tests was 15% with equal distribution between brokers

Common notes

• Longevity test revealed CPU Utilization growing trend for mod-dcb-b module.

Recommendations

• Control mod-dcb-b module related system user creation. For some reason dcb-system-user was not created after environment creation, additional redeploy of module, enabling resolved the issue. It may lead to a huge performance degradation. 

• Add more CPU to mod-users to exclude module fluctuations

Test Runs 

The following table contains  tests configuration information

Results

Errors:

• Error messages: POST_circulation/check-out-by-barcode (Submit_barcode_checkout)_POST_422. 422/Unprocessable Entity. Happen expectedly if instance was checked out already. Error rate - 0.06% which is acceptable.

Response time

The table contains results of Check-in, Check-out tests in Quesnelia release. 

45 minute tests

Longevity test (with absent dcb-system-user)

Comparisons

This table has comparison between average values of response times of Quesnelia and Poppy releases

Comparison of Longevity test (absent dcb-system-user in Quesnelia)

API requests where response times >= 100 ms