...
- Kiwi performs much better than Juniper GA, response times for check-ins are <900ms, checkout <1500ms, with not much variation between 1 and 20 users
- Database performance is better and uses much less CPU compared to Juniper GA
- Worst-performing APIs are still POST /checkin-by-barcode and POST /checkout-by-barcode. The response time are still about 500ms. GET /circulation/loans also takes more than 200ms. GET /inventory/item (by barcode) takes less than 100ms now.
- Longevity test shows response times worsen over time, probably due to the growing DB CPU utilization.
Potentially could address this situationJira Legacy server System JiraJIRA serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key CIRCSTORE-304
...
In general there is no regression in performance. The response times between Kiwi and Juniper are very close to each other for 1-8 users load unless they were in the 95th percentile group or the 20 users load where Kiwi clearly out-perform Juniper. In the tables below, the Delta columns express the differences between Juniper and Kiwi releases in percentage. Any percentage +/-5% is not statistically is within the margin of error. It is also noteworthy that Kiwi seems to invoke the GET /automated-patron-blocks 3 times instead of once
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
Note: JP = Juniper build, KW = Kiwi build
| Average | 50th percentile | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Check-in JP | Check-in KW | Delta | Check-out JP | Check-out KW | Delta | Check-in JP | Check-in KW | Delta | Check-out JP | Check-out KW | Delta | |
| 1 user | 0.944 | 0.838 | 11.23% | 1.579 | 1.582 | -0.19% | 0.835 | 0.767 | 8.14% | 1.411 | 1.464 | -3.76% |
| 5 users | 0.811 | 0.73 | 9.99% | 1.359 | 1.376 | -1.25% | 0.750 | 0.676 | 9.87% | 1.23 | 1.272 | -3.41% |
| 8 users | 0.889 | 0.758 | 14.74% | 1.425 | 1.392 | 2.32% | 0.785 | 0.674 | 14.14% | 1.262 | 1.228 | 2.69% |
| 20 users | 1.386 | 0.899 | 35.14% | 2.21 | 1.506 | 31.86% | 1.172 | 0.731 | 37.63% | 1.887 | 1.303 | 30.95% |
...
Average Response Time in milliseconds. Note: JP = Juniper build, KW = Juniper build
API | 1 user JP (75th %tile) | 1 user KW (75th %tile) | 5 users JP (75th %tile) | 5 users KW (75th %tile) | 8 users JP (75th %tile) | 8 users KW (75th %tile) | 20 users JP (75th %tile) | 20 users KW (75th %tile) |
|---|---|---|---|---|---|---|---|---|
| POST checkout-by-barcode | 526 | 550 | 476 | 436 | 479 | 428 | 861 | 461 |
| POST checkin-by-barcode | 457 | 444 | 402 | 368 | 489 | 438 | 852 | 504 |
| GET circulation/loans | 246 | 305 | 245 | 262 | 272 | 267 | 516 | 303 |
| GET inventory/items | 171 | 124 | 166 | 96 | 170 | 92 | 250 | 99 |
...
Due to the left() function (more details in
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
...
In the 8-users tests, mod-inventory spiked to almost 400% while averaged around 175%. This is an anomaly that can be disregarded. We have seen mod-inventory spiked while being idle or during a test run of any workflow before, mostly due to DI events_cache topic processing. In other tests this is not a problem, so it does not seem to be a bad trend here.
...
Miscellaneous
- Raw test data:
View file name CICO-kiwi.xlsx height 250