PTF-Test Hybrid DB Deployment on multiple tenants[Galileo environment]. Provisioned primary DB db.r7g.2xlarge paired with Dedicated Offload DB db.r8g.2xlarg. Part 3
Overview
This report details the performance evaluation of a Dual-Provisioned Database architecture within a multi-tenant FOLIO environment (Galileo release). The tested architecture features a Provisioned Primary DB (db.r7g.2xlarge) paired with a Dedicated Offload DB (db.r8g.2xlarge). 7 highly intensive, I/O-heavy modules - mod-source-record-manager, mod-source-record-storage, mod-inventory-storage, mod-orders-storage, mod-data-export, mod-lists, and mod-fqm-manager were routed to the Dedicated Offload DB(db.r8g.2xlarge). The objective of this testing phase is to determine if a dual-provisioned hardware baseline canuccessfullytabilize heavy Data Import, Data Export) without degrading real-time Circulation (CICO) and ListApp workflows across 60 tenants.
Ticket: PERF-1334: Multi-Tenants Concurrent Tests (60 tenants) with a Provisioned Secondary DBClosed
Summary:
Test 3 -tandard Load: Under thetandard load profile, the Dual-Provisionedetup easily outperforms the previouserverless baseline. Data Import (50K) matched its optimal 18-minute benchmark, Data Export completed in 17utes, and CICO operations reduced by about 1econd. This confirms that the r8g.2xlarge instance can handle the concurrent load of all 7 heavy modules under normal operating conditions.
Test 4 applied a 3x load multiplier totress-test the architecture's breaking point. CICO achieved its fastest times yet (Check-In at 2.40econds; Check-Out at 3.50econds). Data Import execution degraded to 57utes. Based on the Performance Insight graph foregcon-perf1334-2nddb (Dedicated Offload DB, db.r8g.2xlarge), the primary bottleneck identified is IO: DataFileRead, which dominated the wait events throughout the test, accounting for approximately 70–80% of Average Activeessions (AAS). CPU utilization remained negligible throughout, confirming that compute was not the limiting factor. The degradation observed in Test 4 results — particularly the Data Import regression (18 → 57) is consistent with this I/Oaturation pattern on the Offload DB.
Test 5 applied a verticallycaled Dedicated Offload DB (db.r8g.4xlarge) under a high load profile (60 tenants) to determine if doubling the hardware capacity could mitigate the bottlenecks observed in Test 4. Despite the increased CPU and memory, the verticalcaling yielded no meaningful performance recovery. CICO remainedtable (Check-In at 3.12econds; Check-Out at 4.37econds), also Data Import duration 59utes, and ListApp didin`t fail and latency reached 8utes. This confirms that throwing more physical hardware at the Offload DB does not resolve the degradation.
Recommendations & Jiras
Authorization cache thrashing issue:
MODSIDECAR-185: Latency In High Tenant Count FOLIO ClusterIn Refinement
Previious report:
PTF-Test Hybrid DB Deployment on multiple(30) tenants[Galileo environment]. Provision DB + Serverless V2. Part 2 ,
PTF-Test Hybrid DBDeployment on multiple tenants[Galileo enviroment].
Test Runs
The first two tests (Test 1 and Test 2),hown in the test results table, are from previous testing. Link to the report:
Test 1. Baseline r7g 4xl;
Test 2. Baseline r7g 2xl +L 5-64ACU;
Test 3.
Architecture: Provisioned primary DB
db.r7g.2xlargepaired with Dedicated Offload DBdb.r8g.2xlarge.Load model: Testcenario 1: Medium load.
Routing: 7 modules (
srm,rs,inv-strg,ord-strg,data-export,lists, andfqm-manager) routed to Dedicated Offload DBdb.r8g.2xlarge.Number of tenants: 60;
Test 4.
Architecture: Provisioned primary DB
db.r7g.2xlargepaired with Dedicated Offload DBdb.r8g.2xlarge.Load model: Testcenario 2: High load.
Routing: 7 modules (
srm,rs,inv-strg,ord-strg,data-export,lists, andfqm-manager) routed to Dedicated Offload DBdb.r8g.2xlarge.Number of tenants: 60;
Test 5.
Architecture: Provisioned primary DB
db.r7g.2xlargepaired with Dedicated Offload DBdb.r8g.2xlarge.Load model: Testcenario 2: High load.
Routing: 7 modules (
srm,rs,inv-strg,ord-strg,data-export,lists, andfqm-manager) routed to Dedicated Offload DBdb.r8g.2xlarge.Number of tenants: 60;
Test 6.
Architecture: Provisioned primary DB
db.r7g.4xlargeLoad model: Testcenario 3: MOBIUS Test.
Test 7.
Architecture: Provisioned primary DB
db.r7g.2xlargepaired with Dedicated Offload DBdb.r8g.2xlarge.Load model: Testcenario 3: MOBIUS Test.
Routing: 7 modules (
srm,rs,inv-strg,ord-strg,data-export,lists, andfqm-manager) routed to Dedicated Offload DBdb.r8g.2xlarge.
Test 8.
Architecture: Provisioned primary DB
db.r7g.2xlargepaired with Dedicated Offload DBdb.r8g.2xlarge.Load model: Testcenario 3: MOBIUS Test.
Routing: 7 modules (
srm,rs,earch,ord-strg,data-export,lists, andfqm-manager) routed to Dedicated Offload DBdb.r8g.2xlarge.
Test 9.
Architecture: Provisioned primary DB
db.r7g.2xlargepaired with Dedicated Offload DBdb.r8gd.2xlarge.Load model: Testcenario 3: MOBIUS Test.
Routing: 7 modules (
srm,rs,inv-strg,ord-strg,data-export,lists, andfqm-manager) routed to Dedicated Offload DBdb.r8gd.2xlarge.
Test 10.
Architecture: Provisioned primary DB
db.r7g.2xlargepaired with Dedicated Offload DBdb.r8gd.2xlarge.Load model: Testcenario 3: MOBIUS Test.
Routing: 7 modules (
srm,rs,inv-strg,ord-strg,data-export,lists,mod-search andfqm-manager) routed to Dedicated Offload DBdb.r8gd.2xlarge.
Test 11.
Architecture: Provisioned primary DB
db.r8gd.4xlargeLoad model: Testcenario 3: MOBIUS Test.
Testcenarios:
Testcenario 1: Medium load
8 Tenants: Check-In/Check-Out (CICO);
3 Tenants: Executed Data Import (50K, Profile: PTF-Create-3);
3 Tenants: Bulk Edit for holdings, users, and items with upload and edit operations;
2 Tenants: Executed Data Export workflows with Custom and Default profiles;
2 Tenants: Executed Harvesting workflows(OAI - PMH);
2 Tenants: Executed Refresh Lists workflows.
Testcenario 2: High load(x3)
24 Tenants: Check-In/Check-Out (CICO);
9 Tenants: Executed Data Import (50K, Profile: PTF-Create-3);
9 Tenants: Bulk Edit for holdings, users, and items with upload and edit operations;
6 Tenants: Executed Data Export workflows with Custom and Default profiles;
6 Tenants: Executed Harvesting workflows(OAI - PMH);
2 Tenants: Executed Refresh Lists workflows.
Testcenario 3: MOBIUS Test:
Previous testing data:
PTF - MOBIUS (Sunflower) Performance Testing Report 18.02.2026
Results
Test 3 - Test 5
SCENARIO | Baseline r7g 4xl | Baseline r7g 2xl +L 5-64ACU | Test 3 | Test 4 | Test 5 | 1 VU | |
Check | In | 5,4sec | 4,2sec | 3.02 | 2.40sec | 3.12 | 3.3 sec |
Out | 7sec | 5,89sec | 4.67 | 3.50ec | 4.37 | 4.1 sec | |
Data Import | 35 | 26 | 18 | 57 | 59 | ||
Data export | Custom profile | 23 | 17 | 17 | 9 | 9 | 8.6 |
Default profile | 5 | 4,2 | 5 | 4,6 | 3,8 | 4,37 | |
OAI - PMH | 0,5ec | 0,456ec | 0,34ec | 0,536ec | 0,55ec | 0.372ec | |
Bulk Edit Upload | holdings | 66ec | 95ec | 71ec | 96ec | 93ec | 66ec |
users | 31ec | 40ec | 36ec | 53ec | 45ec | 32ec | |
items | 76ec | 120ec | 95ec | 152ec | 174ec | 231ec | |
Bulk Edit edit | holdings | 250ec | 192ec | 212 | 234ec | 227ec | 141ec |
users | 57ec | 39ec | 42ec | 42ec | 44ec | 37ec | |
items | 112ec | 84ec | 97ec | 96ec | 104ec | 53ec | |
ListApp | 4ec - 120ec | 3ec - 9 | 3ec - 4 | failed | 14ec-8 | The comparison table is below | |