Overview

The test's target is to investigate the behaviour of the Check-In/Check-Out activities during fixed load tests.

PERF-702 - Getting issue details... STATUS

Summary

Check-In/Check-Out tests on Poppy carried out with 8, 20, 30, 75 users during 45 minutes and with 30 users during 14 hours - longevity test .

45 minute tests

• Within Poppy release 45 min tests best response times results were with 20 users. Average CI - 0.45 sec, CO - 0.805 sec. During the highest load (75 users) response times degraded ~ 30% in average.
• Comparing Poppy and Orchid tests with 20 users during 45 minutes there's no significant difference but for tests with higher vUsers amount we see improvements for CI and CO which should be investigated.

Longevity test

• Within Poppy release Longevity test with 30 vUsers degraded compared to 45 min test with: CI - 0.471 sec(+4.90%), CO - 1.025 sec(+27.01%).  But CI/CO is stable still during a long fixed load.
• Comparing Longevity results of Orchid vs Poppy - CI response times changed from 1.061 sec to 0.471 sec and CO - from 3.233 sec to 1.025 sec.
• Database utilization has growing trend for longevity test.

Common notes

• Main API call slightly degraded (POST CO: +5.07%)
• Memory consumption spikes at the beginning of testing usually go after cluster start and stabilize. But here memory leak for mod-pubsub module was suspected.
• Spikes observed during all tests during 5-10 seconds for all requests with every minute frequency. Response times are up to 6 seconds.

Recommendations

• Investigate mod-pubsub module behaviour  PERF-721 - Getting issue details... STATUS
• Investigate response times differences between releases. Run additional test on Orchid release with 30 vUsers for 45 minutes and compare with Poppy results to find the source of changes.

Test Runs 

The following table contains information about a test model and related Grafana snapshots:

Results

The following table contains information about test results and their comparison with the baseline (poppy 8 VU).

• Comparing Longevity test results against 45 min with 30 users within Poppy release we see almost the same results for Check-In 0.471 sec(+4.90%) in average and 1.025 sec(+27.01%) for Check-Out.
• Within Poppy the response times during test with 75 users grew up to 30% compared to 8 users test.

Errors:

• Error messages: POST_circulation/check-out-by-barcode (Submit_barcode_checkout)_POST_422. Reason for that is a doubled usage of checked-out item barcodes. 

Response time