| Table of Contents | ||
|---|---|---|
|
...
- 79 back-end modules deployed in 153 ECS tasks (changed. In Lotus, it was75 back-end modules deployed in 159 ECS tasks)
- 3 okapi ECS tasks
- 10 m6i.2xlarge EC2 instances (changed. In Lotus it was 9 m5.xlarge)
- 2 db.r6g.xlarge AWS RDS instance
- INFO logging level
- mod-inventory (running tasks -2)
- 1024 CPU units, 1684 MiB
- mod-inventory-storage (running tasks -2)
- 1024 CPU units, 1684MB mem
- mod-circulation (running tasks -2)
- 1024 CPU units, 896MB mem
- mod-circulation-storage (running tasks -2)
- 1024 CPU units, 1024MB mem
- mod-pubsub (running tasks -2)
- 1024 CPU units, 1296 MiB
- mod-patron-blocks (running tasks -2)
- 1024 CPU units, 896MB mem
- mod-feesfines (running tasks -2)
- 128CPU units, 896MB mem
- mod-authtoken (running tasks -2)
- 128 CPU units, 896MB mem
- okapi (running tasks -3)
- 1024 CPU units, 1360 MB mem
High-Level Summary
- Morning Glory performs much better than Lotus response times for check-ins are mid 400ms, checkout 600ms - 800ms, with little variations between 1 and 50 users
- Worst-performing APIs are only POST /checkin-by-barcode and POST /checkout-by-barcode. They have improved since Lotus, now averaging in the mid 200ms even in the 50 users scenario.
- GET /circulation/loans take 107ms on average in the 20 users and 50 users cases.
...
Test | Virtual Users | Duration | Load generator size (recommended) | Load generator Memory(GiB) (recommended) |
1. | 1 user | 30 mins | t3.medium | 3 |
2. | 5 users | 30 mins | t3.medium | 3 |
3. | 8 users | 30 mins | t3.medium | 3 |
4. | 20 users | 30 mins | t3.medium | 4 |
| 5. | 50 users | 30 mins | t3.large | 6 |
| 9. | 20 users longevity | 16 hours | t3.xlarge | 14 |
Results
Response Times (Average of all tests listed above, in seconds)
...
Note: LT = Lotus build, MG = Kiwi Morning Glory build
| Average | 50th percentile | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Check-in MG | Check-in LT | Delta | Check-out MG | Check-out LT | Delta | Check-in MG | Check-in LT | Delta | Check-out MG | Check-out LT | Delta | |
| 1 user | 0.516 | 0.73 | 29.32% | 0.795 | 1.188 | 33.08% | 0.49 | 0.682 | 28.15% | 0.758 | 1.084 | 30.07% |
| 5 users | 0.418 | 0.62 | 32.58% | 0.646 | 0.976 | 33.81% | 0.408 | 0.582 | 29.90% | 0.624 | 0.915 | 31.80% |
| 8 users | 0.411 | 0.615 | 33.17% | 0.627 | 0.971 | 35.43% | 0.4 | 0.581 | 31.15% | 0.606 | 0.904 | 32.96% |
| 20 users | 0.401 | 0.661 | 39.33% | 0.615 | 1.052 | 41.54% | 0.393 | 0.62 | 36.61% | 0.6 | 0.967 | 37.95% |
| 50 users | 0.371 | - | - | 0.601 | - | - | 0.373 | - | - | 0.59 | - | - |
...
The timeline highlighted below encompasses all 5 test runs (1, 5, 8, 20, and 50 users).
Modules CPUs and Memory Utilization
...
| 20-users tests | Avg | Max |
|---|---|---|
| mod-users | 18% | 18% |
| mod-pubsub | 5% | 5% |
| okapi | 11% | 11% |
| mod-circulation | 4% | 4% |
| mod-circulation-storage | 5% | 5% |
| mod-inventory | 6% | 6% |
| mod-inventory-storage | 7% | 7% |
| mod-patron-blocks | 1% | 1% |
| mod-feesfines | 14% | 15% |
| mod-authtoken | 24% | 66% |
| 50-users tests | Avg | Max |
|---|---|---|
| mod-users | 38% | 39% |
| mod-pubsub | 7% | 7% |
| okapi | 25% | 26% |
| mod-circulation | 6% | 7% |
| mod-circulation-storage | 9% | 10% |
| mod-inventory | 7% | 7% |
| mod-inventory-storage | 12% | 14% |
| mod-patron-blocks | 2% | 2% |
| mod-feesfines | 31% | 33% |
| mod-authtoken | 47% | 116% |
Services' memory seems to be stable during the test runs.
...