FameFlower Test Results
| Table of Contents |
|---|
...
In this workflow we are checking the performance of the loading items on instance record (in holdings accordion) workflow running in the Fameflower release. We will test it with 1, 5, 8, and 20 virtual users for 30 minutes. A longevity test will also be executed to see if there were memory issues.
...
Overall load items workflow time in seconds
Average (seconds) 50th %tile (seconds) 75th %tile (seconds) 95th %tile (seconds) 1 user 1.015 0.96 14.67 s 1.322 5 users 1.236 1.156 1.464 1.704 8 users 1.512 1.403 1.741 2.02 20 users 1.649 1.535 1.896 2.252 Slow APIs taking more than 100 ms to return
API 1 user (75th %tile) 5 users (75th %tile) 8 users (75th %tile) 20 Users (75th %tile) GET source-storage/formattedRecords/{id} 5.32 s 19.92 s 34.83 s 1.46 min GET inventory/instances keyword all "aba"
355.60 ms 753.86 ms 909.02 ms 2.87 s POST circulation/check-out-by-barcode 679.72 ms 1.04 s 1.11 s 1.45 s GET inventory/items 164.45 ms 251.71 ms 286.87 ms 405.85 ms GET inventory/instances/{id} 119.93 ms 115.98 ms 123.88 ms 131.89 ms GET circulation/loans 221.95 ms 254.08 ms 251.45 ms 256.18 ms GET locations 221.09 ms 142.89 ms 152.26 ms 205.71 ms GET circulation/requests 81.77 ms 212.38 ms 257.80 ms 443.79 ms - Excess logging of missing indexes - 64K lines in 45 minutes run. Logging level could be reduced to WARNING or INFO, but at the cost of having less data to work with should there be a need to troubleshoot.
- JVM profiling shows JSON de/serialization operations one of the slowest operations.
...
Test | Virtual Users | Duration | OKAPI log level | Profiled | Ramp up (total time in seconds) |
1. | 1 | 30 min | INFO | No | 1 |
2. | 5 | 30 min | INFO | No | 50 |
3. | 8 | 30 min | INFO | No | 80 |
4. | 20 | 30 min | INFO | No | 200 |
5. | 1 | 30 min | INFO | Yes | 1 |
6. | 5 | 30 min | INFO | Yes | 50 |
7. | 8 | 30 min | INFO | Yes | 80 |
8. | 20 | 30 min | INFO | Yes | 200 |
...
- Can drill down these methods for more information at : http://ec2-3-93-19-104.compute-1.amazonaws.com/grafana/d/b2iP-9hiz/java-profiling-folio?orgId=1&from=1589940195166&to=1589943457693&var-service_tag=mod-circulation&var-aggregation=10s
Database
Database does not show much CPU usage for 1, 5, 8 and 20 users runs. 100% CPU usage for 5, 8, 20 users.
The following WARNING statements of missing indexes were generated during a test run and logged by mod-source-record-storage:
...
| 1 user | 5 users | 8 users | 20 users | |||||
|---|---|---|---|---|---|---|---|---|
| Average % | Range % | Average % | Range % | Average % | Range % | Average % | Range % | |
| Okapi | 2.12 | 0.29-8.94 | 2.87 | 1.56-13.65 | 2.78 | 0.31-16.82 | 2.85 | 0.32-11.65 |
| mod-inventory | 0.84 | 0.31-14.82 | 0.68 | 0.30-4.01 | 0.67 | 0.26-7.79 | 0.65 | 0.27-3.72 |
| mod-inventory-storage | 1.59 | 0.33-8.17 | 1.87 | 0.56-9.25 | 1.81 | 0.24-9.15 | 1.78 | 0.25-23.28 |
| mod-circulation | 0.67 | 0.24-4.12 | 0.82 | 0.48-4.44 | 0.96 | 0.26-4.04 | 0.82 | 0.28-4.12 |
| mod-circulation-storage | 0.71 | 0.25-6.60 | 0.74 | 0.31-2.79 | 0.86 | 0.29-3.27 | 0.71 | 0.27-2.50 |
| mod-source-record-storage | 0.34 | 0.23-1.43 | 0.50 | 0.26-1.86 | 0.53 | 0.31-2.41 | 0.43 | 0.29-1.92 |
Memory
Memory was stable throughout the runs, only a spike here or there, but in a 30 minutes run they were consistent.
| 1 user | 5 users | 8 users | 20 users | |
|---|---|---|---|---|
| Average | Average | Average | Average | |
| Okapi | 40% | 42% | 40% | 40% |
| mod-inventory | 40% | 43% | 44% | 43% |
| mod-inventory-storage | 40% | 40% | 41% | 41% |
| mod-circulation | 72% | 75% | 72% | 74% |
| mod-circulation-storage | 32% | 32% | 32% | 32% |
| mod-source-record-storage | 53% | 53% | 53% | 53% |
Appendix
See Attached FameFlower Performance Test Runs.xlsx for details