SIP2-288 - Book-sorter performance issues
Submitted | Jul 14, 2024 |
|---|---|
Approved |
|
Status | in progress |
Impact | MEDIUM |
Arch Ticket | |
Prod ticket |
Problem Statement
LOC’s book sorter machine has an issue with data lag/sync between FOLIO and the machine.
Context
Book sorter machine integrates with Folio via SIP2
The process involves 2 calls to Folio
1. The item barcode is scanned, and a call is made to identify the item.
2. Using the returned response, the item’s location is established, and the second call is made to check in the item to the appropriate tenant.
The book sorter machine will then deliver the item to the appropriate bin for delivery to be shelved.
Check-in rate corresponds to a worker manually placing each book on the book sorter machine (every few seconds at peak)
Issue
The book sorter will time out during the operation, fail to sort the book, and deliver the book to the error bin instead.
The book sorter machine has a hard-coded timeout of 1 second for the entire process
FOLIO will respond to each call within approximately 2 seconds, with an overall response time of 4 seconds or more.
SIP2 Request Flows
Each executed command performs a login request with the provided pair of credentials.
Credentials are provided during the login command (
username | password | servicePointId).Each new pair of credentials could affect the authorization cache in
folio-module-sidecar(usessessionIdfor authorization cache-key)
1. Login Request Diagram
2. Item Information Request Diagram
3. Item Check-In Request Diagram
Related Information
Token cache was disabled in the following stories:
Okapi request caching for authorized requests:
Eureka Sidecar caching for authorized requests:
Solution
1. Minimize the number of login requests per session
The total number of requests per the item sorting series of requests is 10:
2 for the Login command
5 for the Item Information command
3 for the Item Checkin command
Although each login operation is relatively quick, performing it 10 times within a single session significantly impacts performance. To optimize this, an access token should be issued once at the beginning of the session and refreshed as needed using the HTTP POST /authn/refresh Token API request. If the refresh operation fails for any reason, a new access token should be obtained and stored in the cache as a fallback.
Optimization Results
edge-sip2 container being used on Volaris Rancher environment with increased memory limits (with default ~120Mi container is unstable):
resources:
limits:
memory: 600Mi
requests:
memory: 500MiBefore optimization (Single Login - 5 min)
Label | # Samples | Average | Min | Max | Std. Dev. | Error % | Throughput | Received | Sent | Avg. Bytes |
|---|---|---|---|---|---|---|---|---|---|---|
Init connection | 1 | 197 | 0 | 197 | 0 | 0.00% | 5.07614 | 0.15 | 0 | 31 |
Login | 1 | 401 | 0 | 401 | 0 | 0.00% | 2.49377 | 0.03 | 0 | 12 |
Item Information Request | 202 | 744 | 0 | 1472 | 64.75 | 0.00% | 0.67395 | 0.06 | 0 | 98.1 |
Item Checkin Request | 202 | 741 | 0 | 2684 | 142.24 | 0.00% | 0.6745 | 0.08 | 0 | 123.8 |
Close connection | 1 | 4 | 0 | 4 | 0 | 0.00% | 250 | 0 | 0 | 0 |
TOTAL | 409 | 735 | 0 | 2684 | 130.44 | 0.00% | 1.35827 | 0.15 | 0 | 109.8 |
Before optimization (Login per check-in - 5 min)
Label | # Samples | Average | Min | Max | Std. Dev. | Error % | Throughput | Received KB/sec | Sent KB/sec | Avg. Bytes |
|---|---|---|---|---|---|---|---|---|---|---|
Init connection | 149 | 190 | 0 | 234 | 7.21 | 0.00% | 0.49827 | 0.02 | 0 | 31 |
Login | 149 | 352 | 0 | 412 | 12.47 | 0.00% | 0.49804 | 0.01 | 0 | 12 |
Item Information Request | 149 | 735 | 0 | 995 | 47.95 | 0.00% | 0.49741 | 0.05 | 0 | 98 |
Item Checkin Request | 148 | 738 | 0 | 912 | 42.01 | 0.00% | 0.49782 | 0.06 | 0 | 123.7 |
Close connection | 148 | 0 | 0 | 9 | 1.04 | 0.00% | 0.49923 | 0 | 0 | 0 |
TOTAL | 745 | 402 | 0 | 995 | 295.76 | 0.00% | 2.48125 | 0.13 | 0 | 52.9 |
After optimization (Single Login - 5 min)
Label | # Samples | Average | Min | Max | Std. Dev. | Error % | Throughput | Received KB/sec | Sent KB/sec | Avg. Bytes |
|---|---|---|---|---|---|---|---|---|---|---|
Create SIP2 Session class | 1 | 8 | 0 | 8 | 0 | 0.00% | 125 | 4.03 | 0 | 33 |
Init connection | 1 | 197 | 0 | 197 | 0 | 0.00% | 5.07614 | 0.15 | 0 | 31 |
Login | 1 | 313 | 0 | 313 | 0 | 0.00% | 3.19489 | 0.04 | 0 | 12 |
Item Information Request | 373 | 329 | 0 | 635 | 29.5 | 0.00% | 1.24331 | 0.12 | 0 | 98.2 |
Item Checkin Request | 372 | 475 | 0 | 666 | 22.7 | 0.00% | 1.24353 | 0.15 | 0 | 124 |
Close connection | 1 | 6 | 0 | 6 | 0 | 100.00% | 166.6667 | 0 | 0 | 0 |
TOTAL | 749 | 400 | 0 | 666 | 80.57 | 0.13% | 2.49158 | 0.27 | 0 | 110.6 |
Before optimization (Login per check-in - 5 min)
Label | # Samples | Average | Min | Max | Std. Dev. | Error % | Throughput | Received KB/sec | Sent KB/sec | Avg. Bytes |
|---|---|---|---|---|---|---|---|---|---|---|
Init connection | 213 | 188 | 0 | 201 | 3.98 | 0.00% | 0.71073 | 0.02 |