Check-in-check-out Test Report (Orchid)

Owned by Artem Vronik
Last updated: Apr 11, 2023 by Martin Tran
8 min read

Overview

The test's target is to investigate the behaviour of the Check-In/Check-Out activities during fixed load tests.

PERF-470 - Getting issue details... STATUS


Summary

  • The essential difference (even without absolute comparing): ~20% of degradation for average response time between current test results and:
  • 8 Container instances for ncp5 instead 9 for ncp3 could have a negative impact but not critical
  • No memory leaks were found during the series of tests
  • Based on the longevity test results, the response time for 30 virtual users during the long period of time was degraded by ~ 75% in comparison with the fixed load test for 8 virtual users of the same configuration. Moreover, 'mod-users' CPU Utilization was more than 100% during all test execution. After ~16 hours of the longevity test run, significant degradation of CPU Utilization took place by services involved in Check-in/Check-out operations.
  • Unexpected behaviour of 'mod-search' CPU utilization during fixed load retests with 20 virtual users
  • ERROR LOG Messages have not been found, but there were a lot of WARN LOG Messages (see Additional) connected to the queries usage without an index 

Recommendations

  • Discuss & analyze the situation with WARN LOG Messages (see Additional):
    • Potentially create indexes based on WARN LOG messages
    • Clarify action items for 'The configuration' WARN LOG Messages
  • Discuss & analyze the worst API calls and potential root causes with a development team
  • Decrease the number of virtual users for longevity testing Orchid release candidate (down to 20 or even 8)
  • Discuss & analyze the involvement of 'mod-search' in Check-in/Check-out operations, and pay attention to 'mod-search' CPU utilization during future tests


Test Runs 

The following table contains information about a test model and related Grafana snapshots:

Test #

Test Conditions

vUsersRamp-up, sec

Duration, sec 

Grafana

1.

circulation_checkInCheckOut_orchid8802'780http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&from=1679388600000&to=1679391720000&var-percentile=95&var-test_type=fix_load&var-test=circulation_checkInCheckOut_orchid&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All&var-Request=All
2.circulation_checkInCheckOut_orchid202002'900http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&from=1679392500000&to=1679395560000&var-percentile=95&var-test_type=fix_load&var-test=circulation_checkInCheckOut_orchid&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All&var-Request=All
3.circulation_checkInCheckOut_orchid303003'000http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&from=1679396220000&to=1679399400000&var-percentile=95&var-test_type=fix_load&var-test=circulation_checkInCheckOut_orchid&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All&var-Request=All
4.circulation_checkInCheckOut_orchid757503'450http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&from=1679400360000&to=1679404020000&var-percentile=95&var-test_type=fix_load&var-test=circulation_checkInCheckOut_orchid&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All&var-Request=All
5.circulation_checkInCheckOut_orchid3030086'700http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&from=1679070600000&to=1679157300000&var-percentile=95&var-test_type=fix_load&var-test=circulation_checkInCheckOut_orchid&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All&var-Request=All

Based on previous results, the decision of additional execution was made:

Test #

Test Conditions

vUsersRamp-up, sec

Duration, sec 

Grafana

1*

circulation_checkInCheckOut_orchid8802'780http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&var-percentile=95&var-test_type=fix_load&var-test=circulation_checkInCheckOut_orchid&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All&var-Request=All&from=1679932800000&to=1679935800000
2*circulation_checkInCheckOut_orchid8802'780http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&var-percentile=95&var-test_type=fix_load&var-test=circulation_checkInCheckOut_orchid&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All&var-Request=All&from=1679940840000&