| Table of Contents |
|---|
...
- Longevity test was running 10 hours and began failing because of token expiration. Possible reason here is SSO Session Max limitation 10 hours (fs09000000/realm-settings/sessions).
- Average response times in 10 hours test for Check-In - 1003 ms, Check-Out - 2870 ms
- First 5 hours 30 minutes average response times Check-In - 770 ms, Check-Out - 1773 ms, the next 4 hours 30 minutes - Check-In - 1295 ms, Check-Out - 4280 ms
- Longevity during 24 hours (after session expiration issue was resolved by increasing SSO session Max parameter from 10 to 24 hours)
- Average response times for all 24 test Check-In - 1311 ms, Check-Out - 4541 ms
- The same degradation observed as in 10 hours test. Response time increased significantly in 6 hours after test start - Check-In from 1045 ms to 1526 ms, Check-out from 2523 ms to 5262 ms. See Graphs for response times.
Comparison with previous results in Quesnelia release (qcp1 environment) results:
- 8, 20, 30 vUsers tests were stable. And 75 vUsers test revealed saturation point during ramp-up period with 33 vUsers. After this point response times increased significantly with the same throughput.
- There's a significant performance degradation compared to previous results from tests in quesnelia release on qcp1 environment (non-eureka):
- 8 vUsers - Check-out - 96%, Check-in - 48%
- 20 vUsers - Check-out - 61%, Check-in - 33%
- 30 vUsers - Check-out - 89%, Check-in - 55%
- 75 vUsers - Check-out - 1111%, Check-in - 695% (during whole test)
- 75 vUsers - Check-out - 48 %, Check-in - 7 % (before saturation point was found)
- 30 vUsers (Longevity) - Check-out - 326%, Check-in - 188% (24 hours test)
...
- RDS CPU utilization average
- For both 45 minute and longevity tests RDS used max 867 connections. Without test it was 812 connections.
- CPU (User) usage by broker
- Common CPU utilization by broker during longevity was 18, 18, 16, 14% accordingly to broker distribution. Resource utilization decreased significantly after 6 hours 20 minutes (17:40 UTC) to 14, 12, 12, 11%
Notes:
- The reason of degradation, which occurs after 6 hours of testing, has not yet been found.
...
| 8 vUsers | 20 vUsers | 30 vUsers | 75 vUsers | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Requests | Response Times (ms) | Response Times (ms) | Response Times (ms) | Response Times (ms) | ||||||||
| Label | #Samples | Average | 95th pct | #Samples | Average | 95th pct | #Samples | Average | 95th pct | #Samples | Average | 95th pct |
| Check-out Controller | 1745 | 1459.64 | 1694 | 4140 | 1356.82 | 1569.95 | 6047 | 1646.96 | 2725.6 | 6567 | 12589.1 | 19031 |
| Check-In Controller | 1288 | 603.95 | 706 | 3297 | 625.78 | 665 | 4560 | 748.3 | 1340.05 | 4957 | 4610.3 | 8229.2 |
| POST_circulation/check-out-by-barcode (Submit_barcode_checkout) | 1746 | 661.9 | 755.95 | 4142 | 613.14 | 659 | 6047 | 729.59 | 1242.8 | 6594 | 3868.57 | 7169.25 |
| POST_circulation/check-in-by-barcode (Submit_barcode_checkin) | 1289 | 240.43 | 287.5 | 3305 | 277.17 | 293 | 4572 | 286.61 | 363 | 4969 | 1642.04 | 3832.5 |
| GET_circulation/loans (Submit_barcode_checkout) | 1745 | 285.6 | 340 | 4141 | 275.47 | 307 | 6047 | 347.89 | 640.6 | 6571 | 2528.16 | 5109 |
| GET_circulation/loans (Submit_patron_barcode) | 1746 | 36.23 | 262 | 6599 | 1548.85 | 3773 | ||||||
| GET_circulation/requests (Submit_barcode_checkin) | 4960 | 1171.58 | 3366.5 | |||||||||
| GET_circulation/requests_status_Open (Submit_patron_barcode) | 6596 | 1120.86 | 3117.2 | |||||||||
| GET_circulation/requests (Submit_barcode_checkout) | 6565 | 997.17 | 3095.7 | |||||||||
| GET_users (Submit_patron_barcode) | 1747 | 121.36 | 129 | 6050 | 136.53 | 368.45 | 6603 | 872.72 | 2508.6 | |||
| GET_inventory/items (Submit_barcode_checkin) | 1289 | 107.55 | 130.5 | 3305 | 111.12 | 136 | 4576 | 165.7 | 469.15 | 4970 | 848.62 | 2773 |
| GET_users (Get_check_in_page) | 1290 | 120.27 | 140 | 4577 | 165.61 | 506.3 | 4977 | 836.34 | 2577.1 | |||
| GET_inventory/items (Submit_barcode_checkout) | 1746 | 107.88 | 130 | 4143 | 110.18 | 136 | 6048 | 160.44 | 468.55 | 6594 | 757.24 | 2602 |
| GET_circulation/requests_status_openAwaitingPickup (Submit_patron_barcode) | 6594 | 718.98 | 2636 | |||||||||
- empty cells - response times less than 100 ms
Longevity test
| Expand | ||
|---|---|---|
| ||
Longevity 24 hours
|
...
During the test RDS grew from 32% to 40% in average. Spikes observed after 10 hours
RDS Database Connections
For both 45 minute and longevity tests RDS used max 867 connections. Without test it was 812 connections.
45 minute tests
Longevity test
CPU (User) usage by broker
Common CPU utilization by broker during all tests was 18, 15, 15, 13 % accordingly to broker distribution in 45 min tests
Common CPU utilization by broker during longevity was 18, 18, 16, 14% accordingly to broker distribution. Resource utilization decreased significantly after 6 hours 20 minutes (17:40 UTC) to 14, 12, 12, 11%
45 minute tests
Longevity test
Database load
45 minute tests
Longevity
Target Group 5XX Count
...
- Steps to avoid possible session or token expiration:
- modify tenant/realm-settings/tokens in keycloaks UI to 24 hours (1 day)
- modify tenant/realm-settings/sessions in keycloaks UI to 24 hours (1 day) Default - 10 hours
- Run data preparation script before each CI/CO test
- Update .jmx file script for Quesnelia release
- Create artefact and upload to carrier-io
- Use Jenkins job to change parameters and run tests
- Test CI/CO with 8, 20, 30, 75 concurrent users for 45 minutes each.
- Test CI/CO with 30 users for 24 hours to detect any trends in memory.
...