Check-in-check-out Test Report (Orchid)
Overview
The test's target is to investigate the behaviour of the Check-In/Check-Out activities during fixed load tests.
- PERF-470Getting issue details... STATUS
Summary
- The essential difference (even without absolute comparing): ~20% of degradation for average response time between current test results and:
- 8 Container instances for ncp5 instead 9 for ncp3 could have a negative impact but not critical
- No memory leaks were found during the series of tests
- Based on the longevity test results, the response time for 30 virtual users during the long period of time was degraded by ~ 75% in comparison with the fixed load test for 8 virtual users of the same configuration. Moreover, 'mod-users' CPU Utilization was more than 100% during all test execution. After ~16 hours of the longevity test run, significant degradation of CPU Utilization took place by services involved in Check-in/Check-out operations.
- Unexpected behaviour of 'mod-search' CPU utilization during fixed load retests with 20 virtual users
- ERROR LOG Messages have not been found, but there were a lot of WARN LOG Messages (see Additional) connected to the queries usage without an index
Recommendations
- Discuss & analyze the situation with WARN LOG Messages (see Additional):
- Potentially create indexes based on WARN LOG messages
- Clarify action items for 'The configuration' WARN LOG Messages
- Discuss & analyze the worst API calls and potential root causes with a development team
- Decrease the number of virtual users for longevity testing Orchid release candidate (down to 20 or even 8)
- Discuss & analyze the involvement of 'mod-search' in Check-in/Check-out operations, and pay attention to 'mod-search' CPU utilization during future tests
Test Runs
The following table contains information about a test model and related Grafana snapshots:
Based on previous results, the decision of additional execution was made: