...

• 61 back-end modules deployed in 110 ECS services
• 3 okapi ECS services
• 8 m5.large  EC2 instances
• 2 db.r5.xlarge AWS RDS instance
• INFO logging level

High Level Summary

• Check-out: Honeysuckle is slower by 9%-28% than Goldenrod
• Check-in: 4%-22% slower than Goldenrod
• APIs turned slower in Honeysuckle: GET /automated-patron-blocks/{id} (150% slower) and  GET /circulation/loans (60%). These are covered by MODPATBLK-70 and CIRC-1014, respectively
• Okapi v4.3.3 seem to be using 2x-3x CPU cycles than in v1.3.2 (Goldenrod).  Potential issue found with the logging methods OKAPI-964 
• mod-pubsub has a memory leak that would drag down performance under high loads (see section on longevity test): MODPUBSUB-136
• Caching Okapi tokens in Okapi reduced mod-authtoken's CPU usage by over 90%
• Database's memory usage improved dramatically from Goldenrod's - little memory consumptions observed.

Test Runs

...

A 24-hour longevity test was performed (with 20 concurrent users and running with Okapi 4.2.2). Key observations:

• Performance degraded after 2 hours.
  • Average response time in the first two hours: 
    • Check-in:  2.28 sec
    • Check-out: 4.76 sec
  • Average response time in the first 12 hours:
    • Check-in: 2.709 sec
    • Check-out: 7.898 sec
  • Average response time in the last 12 hours:
    • Check-in:  4.605 sec
    • Check-out: 17.297 sec
  • Together the average response time for the entire test run:
    • Check-in: 3.414 sec
    • Check-out: 11.419 sec
• Throughput degrades over time, unsurprisingly
  • 

...

WARNING: Doing LIKE search without index for jsonb->'noticeConfig'>>'timing', CQL >>> SQL: noticeConfig.timing == After >>> lower(f_unaccent(jsonb>'noticeConfig'->>'timing')) LIKE lower(f_unaccent('After'))
WARNING: Doing LIKE search without index for jsonb->>'loanId', CQL >>> SQL: loanId == 671233fd-5c15-4f9e-8fab-f86330c389bd >>> lower(f_unaccent(jsonb->>'loanId')) LIKE lower(f_unaccent('671233fd-5c15-4f9e-8fab-f86330c389bd'))
WARNING: Doing LIKE search without index for jsonb->>'triggeringEvent', CQL >>> SQL: triggeringEvent == "Due date" >>> lower(f_unaccent(jsonb->>'triggeringEvent')) LIKE lower(f_unaccent('Due date'))

mod-feesfines missing indexes:

WARNING: Doing LIKE search without index for accounts.jsonb->>'userId', CQL >>> SQL: userId == e96618a9-04ee-4fea-aa60-306a8f4dd89b >>> lower(f_unaccent(accounts.jsonb->>'userId')) LIKE lower(f_unaccent('e96618a9-04ee-4fea-aa60-306a8f4dd89b'))
WARNING: Doing LIKE search without index for accounts.jsonb->'status'>>'name', CQL >>> SQL: status.name <> Closed >>> lower(f_unaccent(accounts.jsonb>'status'->>'name')) NOT LIKE lower(f_unaccent('Closed'))
WARNING: Doing LIKE search without index for manualblocks.jsonb->>'userId', CQL >>> SQL: userId == a79b533d-8f29-4be1-9415-5f5cd936623b >>> lower(f_unaccent(manualblocks.jsonb->>'userId')) LIKE lower(f_unaccent('a79b533d-8f29-4be1-9415-5f5cd936623b'))

...