FameFlower Test Results
...
In this workflow we are checking the performance of the loading items on instance record (in holdings accordion) workflow running in the Fameflower release. We will test it with 1, 5, 8, and 20 virtual users for 30 minutes. A longevity test will also be executed to see if there were memory issues.
...
High Level Summary
Overall check in, check out load items time in seconds
Average (seconds) 50th %tile (seconds) 75th %tile (seconds) 95th %tile (seconds) Check-in Check-out Check-in Check-out Check-in Check-out Check-in Check-out 1 user 1.015 1.234 0.96 1.277 1.071 1.409 1.322 1.653 5 users 1.236 1.488 1.156 1.393 1.464 1.869 1.704 2.219 8 users 1.512 1.751 1.403 1.852 1.741 2.031 2.02 2.274 20 users 1.649 1.898 1.535 1.996 1.896 2.211 2.252 2.539 Slow APIs taking more than 100 ms to return
API 1 user (75th %tile) 5 users (75th %tile) 8 users (75th %tile) 20 Users (75th %tile) GET /source-storage/formattedRecords/{id} 5.24 s 905 ms 906 ms 988 ms GET_/inventory/instances keyword all "aba"
POST_/circulation/check-out-by-barcode 283 ms 346 ms 449 ms 479 ms Get GET inventory/items 217 ms 232 ms 237 ms 281 ms GET _/ inventory/instances/{id} GET _/ circulation/loans - Excess logging of missing indexes - 64K lines in 45 minutes run. Logging level could be reduced to WARNING or INFO, but at the cost of having less data to work with should there be a need to troubleshoot.
- JVM profiling shows JSON de/serialization operations one of the slowest operations.
Test Runs
30-min Runs for export instance UUIDs workflow:
Test | Virtual Users | Duration | OKAPI log level | Profiled | Ramp up (total time in seconds) |
1. | 1 | 30 min | INFO | No | 1 |
2. | 5 | 30 min | INFO | No | 50 |
3. | 8 | 30 min | INFO | No | 80 |
4. | 20 | 30 min | INFO | No | 200 |
5. | 1 | 30 min | INFO | Yes | 1 |
6. | 5 | 30 min | INFO | Yes | 50 |
7. | 8 | 30 min | INFO | Yes | 80 |
8. | 20 | 30 min | INFO | Yes | 200 |
...
- Can drill down these methods for more information at : http://ec2-3-93-19-104.compute-1.amazonaws.com/grafana/d/b2iP-9hiz/java-profiling-folio?orgId=1&from=1589940195166&to=1589943457693&var-service_tag=mod-circulation&var-aggregation=10s
Database
Database does not show much CPU usage for 1, 5, 8 and 20 users runs. At maximum only 25% CPU usage for the high case of 20 users.
The following WARNING statements of missing indexes were generated during a test run and logged by mod-circulation-storage:
CPU Utilization
| 1 user | 5 users | 8 users | 20 users | |||||
|---|---|---|---|---|---|---|---|---|
| Average % | Range % | Average % | Range % | Average % | Range % | Average % | Range % | |
| Okapi | 2.12 | 0.29-8.94 | ||||||
| mod-inventory | ||||||||
| mod-inventory-storage | ||||||||
| mod-circulation | ||||||||
| mod-circulation-storage | ||||||||
Memory
Memory was stable throughout the runs, only a spike here or there, but in a 30 minutes run they were consistent.
| 1 user | 5 users | 8 users | 20 users | |
|---|---|---|---|---|
| Average | Average | Average | Average | |
| Okapi | 40% | 42% | 40% | 40% |
| mod-inventory | 40% | 43% | 44% | 43% |
| mod-inventory-storage | 40% | 40% | 41% | 41% |
| mod-circulation | 72% | 75% | 72% | 74% |
| mod-circulation-storage | 32% | 32% | 32% | 32% |
Appendix
See Attached FameFlower Performance Test Runs.xlsx for details