PTF - Performance Testing of Exporting Circulation Logs

PTF - Performance Testing of Exporting Circulation Logs

Overview

  • This document contains performance testing results for Exporting Circulation Log on ECS Central and Member tenants.

  • The primary focus is to evaluate system behavior under increasing data volumes and identify the threshold at which performance degradation occurs.

  • Jira https://folio-org.atlassian.net/browse/PERF-1420

Summary

  • Circulation Log Export scales linearly up to 50K, indicating the inflection point begins around 100K records on both Central and Member tenants.

  • Export duration on Central tenant is ~10–15% faster than Member, but comparison is influenced by different dataset sizes (1M vs 7M logs).

  • 1K–50K exports complete within ~4 sec–2.5 min and scale near-linearly.

  • Exports above 100K become heavy — 100K ~4-6 min, 200K takes ~14–18 min, and 300K ~27–31 min.

  • CICO is impacted by concurrent exports, but the impact of Circulation Log exports on CICO does not increase proportionally with export size.

  • Check-In/Check-Out response times increase (mostly under 20% on average), with the most notable spikes at the 95th percentile during 30K (37%) and 200K+ (27% - 64%) exports.

 

Test Runs 

Test #

Test Conditions

Status

1

Central tenant – Export: 1K, 10K, 30K, 50K, 100K, 200K, 300K records

Completed

2

Member tenant – Export: 1K, 10K, 30K, 50K, 100K, 200K, 300K records

Completed

3

Member tenant – Export (1K–300K) + CICO background load (8 users)

Completed

Results

Test №1

This table contains Circulation Log Export durations across multiple export sizes on the Central tenant.
Export scales linearly up to 50K but degrades at 100K. 100K marks the inflection point where per-record cost begins accelerating non-linearly.

 
Export size

Central tenant

Duration

sec/1K

1K

3.9 sec

3.9 sec

10K

14.1 sec

1.4 sec

30K

44.4 sec

1.4 sec

50K

1 min 25 sec

1.7 sec

100K

4 min 33 sec

2.7 sec

200K

14 min 20 sec

4.3 sec

300K

27 min 16 sec

5.4 sec


Test №2

This table contains Circulation Log Export durations across multiple export sizes on the Member tenant.
On member tenant follows the same pattern, the inflection point remains at 100K.


Export size

Member tenant 

Duration

sec/1K

1K

18.8 sec

18.7 sec

10K

28.5 sec

2.8 sec

30K

1 min 17 sec

2.5 sec

50K

2 min 24 sec

2.8 sec

100K

5 min 51 sec

3.5 sec

200K

17 min 55 sec

5.3 sec

300K

30 min 52 sec

6.1 sec


Test №3

This table contains Circulation Log Export durations across multiple export sizes on the Member tenant with CICO background load (8 users).
The average impact on CICO ~20%. However, pct95 reveals significant spikes: Check-Out Controller pct95 reaches 1874 ms during 30K export (+37%) and Check-In Controller pct95 hits 984 ms during 200K export (+64%).
The impact of Circulation Log exports on CICO does not increase proportionally with export size.

CICO

Response time, ms

Baseline 

Circulation Log Export


Transaction

no Export

1K

 10K

30K

50K

100K

200K

300K

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Check-In Controller

386

600

435

579

472

720

451

816

409

674

420

684

430

984

403

882

Check-Out Controller

852

1366

961

1396

1014

1553

1022

1874

861

1365

1000

1716

990

1677

926

1736

POST_circulation/check-in-by-barcode (Submit_barcode_checkin)

196

311

218

333

214

335

212

361

199

383

203

340

206

348

185

327

POST_circulation/check-out-by-barcode (Submit_barcode_checkou)

224

324

247

373

236

409

260

584

210

439

237

375

292

444

220

405

GET_circulation/loans (Submit_barcode_checkout)

112

146

114

167

119

197

122

290

102

238

121

208

115

207

113

198

Impact of Circulation Log Export on Check-In and Check-Out:

CICO

Delta, ms

Circulation Log Export

Transaction

1K

 10K

30K

50K

100K

200K

300K

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Check-In Controller

49

-21

86

120

65

216

23

74

34

84

44

384

17

282

Check-Out Controller

109

30

162

187

170

508

9

-1

148

350

138

311

74

370

POST_circulation/check-in-by-barcode (Submit_barcode_checkin)

22

22

18

24

16

50

3

72

7

29

10

37

-11

16

POST_circulation/check-out-by-barcode (Submit_barcode_checkout)

23

49

12

85

36

260

-14

115

13

51

68

120

-4

81

GET_circulation/loans (Submit_barcode_checkout)

2

21

7

51

10

144

-10

92

9

62

3

61

1

52

 

CICO

 Delta, %

Circulation Log Export

Transaction

1K

 10K

30K

50K

100K

200K

300K

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Avg

pct95

Check-In Controller

13%

-4%

22%

20%

17%

36%

6%

12%

9%

14%

11%

64%

4%

47%

Check-Out Controller

13%

2%

19%

14%

20%

37%

1%

0%

17%

26%

16%

23%

9%

27%

POST_circulation/check-in-by-barcode (Submit_barcode_checkin)

11%

7%

9%

8%

8%

16%

2%

23%

4%

9%

5%

12%

-6%

5%

POST_circulation/check-out-by-barcode (Submit_barcode_checkout)

10%

15%

5%

26%

16%

80%

-6%

35%

6%

16%

30%

37%

-2%

25%

GET_circulation/loans (Submit_barcode_checkout)

2%

14%

6%

35%

9%

99%

-9%

63%

8%

42%

3%

42%

1%

36%


Resources Utilization for Test 1 - 2

Service CPU Utilization

The main CPU consumer is mod-data-export-worker. CPU utilization scales with export size, with sustained high usage observed during 200K–300K exports. Utilization was 31% for 1K exports, peaking at ~150% during the 300K export.

image-20260610-121442.png

 

Service Memory Utilization

Memory utilization increases with higher export load and drops after each peak. No signs of memory accumulation or memory leak were observed. Overall memory behavior remains stable across both tests.

On the central tenant, higher peaks are observed (~75–80%) during large exports (200K–300K), for the member tenant remains (~55–60%).

image-20260610-131608.png

 

DB CPU Utilization

CPU utilization shows clear scaling with export size in both tenants. At low load (1K–10K), CPU remains minimal (~9% on Central, ~21% on Member), while at high load (300K) it increases to ~18% on Central and up to ~40–48% on Member on spikes.
Member tenant consistently shows higher % and more pronounced spikes. However, this comparison should be interpreted in context of different dataset sizes (Central ~1M logs vs Member ~7M logs).

 

image-20260610-131811.png

 

DB Connections

Database connection counts remain stable - on average 560.

image-20260610-131913.png

 

DB load

DB load during test 1 and test 2 sliced by SQL:

image-20260610-132150.png
image-20260610-132222.png

 

Resources Utilization for Test 3

Service CPU Utilization

With background CICO load, mod-data-export-worker CPU utilization patterns remain consistent.
CICO-related modules do not show additional CPU consumption during exports.

image-20260610-132933.png

 

Service Memory Utilization

Memory utilization under combined export + CICO load remained stable and comparable to the export-only.

image-20260610-133050.png

 

DB CPU Utilization

Database metrics (CPU, connections, load) under concurrent export + CICO workload show slight increases compared to the export-only tests.
At low load (1K–10K), CPU is ~35%, while at 300K it reaches ~29–52% with intermittent peaks.

image-20260610-133543.png

DB Connections

image-20260610-133642.png

DB load

image-20260610-133741.png
image-20260610-133834.png

 

 

 

Appendix

Infrastructure

PTF - environment asptc2

  • 8 r7i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1

  • 1 instance of db.r7g.xlarge database instance: Writer instance

  • MSK ptf-loc

    • 6 m7g.large brokers in 2 zones

    • Apache Kafka version 3.9.x

    • EBS storage volume per broker 300 GiB

  • OpenSearch 2.19 ptf-test cluster

    • r7g.2xlarge.search 4 data nodes

    • r7g.large.search 3 dedicated master nodes

 

 

Module Task Definition Revision Module Version Task Count Mem Hard Limit Mem Soft Limit CPU Units Xmx Metaspace Size Max Metaspace Size mod-remote-storage 3 mod-remote-storage:4.1.0-SNAPSHOT.204 2 4920 4472 64 3960 512 512 mod-remote-storage - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-ebsconet 2 mod-ebsconet:3.0.0-SNAPSHOT.100 2 1248 1024 64 700 128 256 mod-ebsconet - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-mosaic 3 mod-mosaic:2.0.0-SNAPSHOT.21 2 1024 896 64 768 88 128 mod-mosaic - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 edge-sip2 2 edge-sip2:4.1.0-SNAPSHOT.396.tls 2 1024 896 64 768 88 128 mod-consortia-keycloak 2 mod-consortia-keycloak:2.0.0-SNAPSHOT.86.tls 2 5136 4776 512 4416 384 512 mod-consortia-keycloak - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-tags 4 mod-tags:3.1.0-SNAPSHOT.163 2 1024 896 64 768 88 128 mod-tags - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 edge-courses 3 edge-courses:1.8.0-SNAPSHOT.1188.tls 2 1024 896 64 768 88 128 mod-inventory-update 2 mod-inventory-update:5.1.1-SNAPSHOT.141 2 1024 896 64 768 88 128 mod-inventory-update - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-notify 2 mod-notify:3.6.0-SNAPSHOT.287 2 1024 896 64 768 88 128 mod-notify - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-configuration 1 mod-configuration:5.13.0-SNAPSHOT.381 2 1024 896 64 768 88 128 mod-configuration - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 folio-keycloak 3 folio-keycloak:26.5.1.tls 3 2750 2250 1536 1536 0 0 edge-caiasoft 2 edge-caiasoft:3.1.0-SNAPSHOT.139.tls 2 1024 896 64 768 88 128 mod-licenses 2 mod-licenses:6.4.0-SNAPSHOT.201 2 2480 2312 64 1792 384 512 mod-licenses - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-gobi 2 mod-gobi:4.0.0-SNAPSHOT.285 2 1024 896 64 768 88 128 mod-gobi - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-graphql 3 mod-graphql:1.15.1-SNAPSHOT.480 2 1024 896 64 768 88 128 mod-graphql - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-erm-usage 2 mod-erm-usage:5.3.0-SNAPSHOT.166 2 2800 2550 64 1800 384 512 mod-erm-usage - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-batch-print 3 mod-batch-print:1.4.1-SNAPSHOT.35 2 1024 896 64 768 88 128 mod-batch-print - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-copycat 2 mod-copycat:1.10.0-SNAPSHOT.254 2 1024 896 64 768 88 128 mod-copycat - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-entities-links 3 mod-entities-links:5.1.0-SNAPSHOT.152 2 2592 2483 64 1440 0 1024 mod-entities-links - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 pub-edge 1 pub-edge:2025.09.19 2 1024 896 64 768 0 0 mod-orders 2 mod-orders:14.0.0-SNAPSHOT.495 2 2048 1740 64 1024 384 512 mod-orders - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 kong 3 kong:3.9.2 2 6144 6144 64 768 88 128 edge-patron 5 edge-patron:5.5.0-SNAPSHOT.237.tls 2 1024 896 64 768 88 128 edge-inn-reach 3 edge-inn-reach:4.1.0-SNAPSHOT.97.tls 2 1024 896 64 768 88 128 mod-users-bl 2 mod-users-bl:8.1.0-SNAPSHOT.366 2 1440 1152 64 922 88 128 mod-users-bl - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-oa 6 mod-oa:3.3.0-SNAPSHOT.82 0 4024 3024 65 2024 96 160 mod-oa - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 65 256 0 96 mod-inventory-storage 5 mod-inventory-storage:30.1.0-SNAPSHOT.895 2 4096 3690 512 3067 384 512 mod-inventory-storage - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 768 512 256 0 128 mod-invoice 2 mod-invoice:7.0.0-SNAPSHOT.261 2 1440 1152 64 922 88 128 mod-invoice - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-user-import 1 mod-user-import:3.11.0-SNAPSHOT.228 2 1024 896 64 768 88 128 mod-user-import - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-sender 2 mod-sender:1.16.0-SNAPSHOT.121 2 1024 896 64 768 88 128 mod-sender - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mgr-tenant-entitlements 3 mgr-tenant-entitlements:4.0.0-SNAPSHOT.1469.tls 2 1024 896 65 512 88 256 mod-login-keycloak 2 mod-login-keycloak:4.1.0-SNAPSHOT.1120.tls 2 1024 896 128 512 88 256 mod-login-keycloak - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-data-export-worker 3 mod-data-export-worker:3.6.0-SNAPSHOT.323 2 3072 2800 64 2048 384 512 mod-data-export-worker - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 768 64 256 0 128 mod-circulation-storage 2 mod-circulation-storage:17.6.0-SNAPSHOT.456 2 2880 2592 64 1814 384 512 mod-circulation-storage - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-calendar 2 mod-calendar:4.1.0-SNAPSHOT.259 2 2048 1536 64 1024 96 256 mod-calendar - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-event-config 2 mod-event-config:2.11.0-SNAPSHOT.130 2 1024 896 64 768 88 128 mod-event-config - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-courses 1 mod-courses:1.6.0-SNAPSHOT.175 2 1024 896 64 768 88 128 mod-courses - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-circulation-item 3 mod-circulation-item:2.1.0-SNAPSHOT.43 2 1024 896 64 0 0 0 mod-circulation-item - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-email 2 mod-email:1.21.0-SNAPSHOT.151 2 2800 2550 64 1800 384 512 mod-email - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-di-converter-storage 2 mod-di-converter-storage:2.6.0-SNAPSHOT.162 2 1024 896 64 768 88 128 mod-di-converter-storage - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-pubsub 2 mod-pubsub:2.18.0-SNAPSHOT.258 2 1536 1440 1024 922 384 512 mod-pubsub - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 512 256 0 96 edge-orders 5 edge-orders:4.0.0-SNAPSHOT.200.tls 2 1024 896 64 768 88 128 edge-rtac 5 edge-rtac:3.2.0-SNAPSHOT.294.tls 2 1024 896 64 768 88 128 mod-linked-data-import 4 mod-linked-data-import:2.0.0-SNAPSHOT.13 2 4500 4000 64 3000 128 512 mod-linked-data-import - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-rtac-cache 2 mod-rtac-cache:1.2.0-SNAPSHOT.18 2 1024 896 64 768 88 128 mod-rtac-cache - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-users 2 mod-users:19.7.0-SNAPSHOT.752 2 1024 896 64 768 88 128 mod-users - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-template-engine 2 mod-template-engine:1.24.0-SNAPSHOT.144 2 1024 896 64 768 88 128 mod-template-engine - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-audit 3 mod-audit:3.1.0-SNAPSHOT.246 2 1024 896 64 768 88 128 mod-audit - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-source-record-manager 3 mod-source-record-manager:4.1.0-SNAPSHOT.442 2 4500 4000 64 3000 384 512 mod-source-record-manager - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-feesfines 2 mod-feesfines:19.5.0-SNAPSHOT.229 2 1024 896 64 768 88 128 mod-feesfines - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 edge-users 3 edge-users:2.3.0-SNAPSHOT.1380.tls 2 1024 896 64 768 88 128 mod-service-interaction 1 mod-service-interaction:4.3.0-SNAPSHOT.158 2 2048 1844 64 1290 384 512 mod-service-interaction - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-dcb 2 mod-dcb:2.1.0-SNAPSHOT.107 2 2048 1844 64 1292 384 512 mod-dcb - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-patron 2 mod-patron:6.5.0-SNAPSHOT.215 2 1024 896 64 768 88 128 mod-patron - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 edge-connexion 5 edge-connexion:1.5.1-SNAPSHOT.115.tls 2 1024 896 64 768 88 128 mod-data-export-spring 3 mod-data-export-spring:3.7.0-SNAPSHOT.208 1 2048 1844 64 1536 384 512 mod-data-export-spring - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-organizations-storage 2 mod-organizations-storage:5.1.0-SNAPSHOT.169 2 1024 896 64 700 88 128 mod-organizations-storage - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-ncip 5 mod-ncip:1.18.0-SNAPSHOT.306 2 1024 896 64 768 88 128 mod-ncip - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-agreements 2 mod-agreements:7.4.0-SNAPSHOT.288 2 4096 4096 64 4096 0 0 mod-agreements - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-organizations 2 mod-organizations:2.3.0-SNAPSHOT.130 2 1024 896 64 768 88 128 mod-organizations - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-serials-management 2 mod-serials-management:2.2.0-SNAPSHOT.90 2 2780 2312 64 1792 384 896 mod-serials-management - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-settings 2 mod-settings:1.4.0-SNAPSHOT.94 2 1024 896 64 768 88 128 mod-settings - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-search 3 mod-search:6.1.0-SNAPSHOT.407 2 2592 2480 64 1440 512 1024 mod-search - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 edge-dematic 2 edge-dematic:3.1.0-SNAPSHOT.173.tls 1 1024 896 64 768 88 128 mod-data-import 5 mod-data-import:3.5.0-SNAPSHOT.251 1 2048 1844 65 1292 384 512 mod-data-import - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 65 256 0 96 mod-record-specifications 2 mod-record-specifications:3.1.0-SNAPSHOT.60 2 2048 1592 64 1440 256 512 mod-record-specifications - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-inn-reach 2 mod-inn-reach:4.1.0-SNAPSHOT.140 2 3600 3240 64 2880 512 1024 mod-inn-reach - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-orders-storage 2 mod-orders-storage:15.0.0-SNAPSHOT.309 2 1024 896 64 700 88 128 mod-orders-storage - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-erm-usage-harvester 2 mod-erm-usage-harvester:5.3.0-SNAPSHOT.183 2 1024 896 64 768 88 128 mod-erm-usage-harvester - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-password-validator 2 mod-password-validator:4.1.0-SNAPSHOT.193 2 1440 1298 64 768 384 512 mod-password-validator - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 edge-dcb 3 edge-dcb:2.1.0-SNAPSHOT.72.tls 2 1024 896 64 768 88 128 mod-bulk-operations 3 mod-bulk-operations:2.4.0-SNAPSHOT.179 2 3072 2600 64 2048 384 512 mod-bulk-operations - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1500 768 64 256 0 128 mod-fqm-manager 2 mod-fqm-manager:4.2.0-SNAPSHOT.1263 0 3000 2600 64 2048 88 512 mod-fqm-manager - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-finance 2 mod-finance:6.0.0-SNAPSHOT.212 2 1024 896 64 768 88 128 mod-finance - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-tlr 2 mod-tlr:1.3.0-SNAPSHOT.55 2 1024 896 64 768 88 256 mod-tlr - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-lists 2 mod-lists:4.0.2-SNAPSHOT.1361 2 6000 2600 64 2048 88 512 mod-lists - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 768 715 64 256 0 96 mod-permissions 4 mod-permissions:6.8.0-SNAPSHOT.451 2 1684 1544 64 1024 384 512 mod-permissions - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-users-keycloak 2 mod-users-keycloak:4.1.0-SNAPSHOT.867.tls 4 4096 3688 128 3076 88 256 mod-users-keycloak - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-marc-migrations 2 mod-marc-migrations:3.1.0-SNAPSHOT.35 2 2880 2592 64 1814 384 512 mod-marc-migrations - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mgr-applications 2 mgr-applications:4.0.0-SNAPSHOT.1296.tls 2 1024 896 64 512 88 256 mgr-tenants 3 mgr-tenants:3.1.0-SNAPSHOT.1123.tls 2 1024 896 64 512 88 256 edge-oai-pmh 10 edge-oai-pmh:2.13.0-SNAPSHOT.204.tls 2 1512 1360 64 1440 384 512 mod-okapi-facade 3 mod-okapi-facade:4.1.0-SNAPSHOT.53.tls 2 1024 896 64 768 88 128 mod-okapi-facade - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-rtac 3 mod-rtac:3.10.0-SNAPSHOT.282 2 1024 896 64 768 88 128 mod-rtac - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-source-record-storage 2 mod-source-record-storage:6.1.0-SNAPSHOT.562 2 4500 4000 64 3000 384 512 mod-source-record-storage - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-scheduler 2 mod-scheduler:4.1.0-SNAPSHOT.1016.tls 2 1024 896 64 512 88 256 mod-scheduler - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-circulation-bff 3 mod-circulation-bff:1.3.0-SNAPSHOT.50 2 1024 896 64 768 88 128 mod-circulation-bff - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-inventory 2 mod-inventory:22.1.0-SNAPSHOT.659 2 4096 3688 512 3076 384 512 mod-inventory - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 768 512 256 0 128 mod-requests-mediated 2 mod-requests-mediated:1.4.0-SNAPSHOT.58 2 1500 1024 64 768 88 192 mod-requests-mediated - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-linked-data 4 mod-linked-data:3.0.0-SNAPSHOT.705 2 3048 2024 64 768 88 128 mod-linked-data - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-patron-blocks 2 mod-patron-blocks:1.14.0-SNAPSHOT.189 2 1024 896 64 768 88 128 mod-patron-blocks - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-reporting 2 mod-reporting:1.6.0-SNAPSHOT.47 2 1024 896 64 0 0 0 mod-reporting - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 edge-fqm 1 edge-fqm:4.1.0-SNAPSHOT.135.tls 2 1024 896 64 768 88 128 nginx-edge 1 nginx-edge:2025.09.19 2 1024 896 64 0 0 0 mod-invoice-storage 2 mod-invoice-storage:7.0.0-SNAPSHOT.173 2 1872 1536 64 1024 384 512 mod-invoice-storage - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-reading-room 3 mod-reading-room:2.1.0-SNAPSHOT.31 2 1024 896 64 768 88 128 mod-reading-room - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-data-export 2 mod-data-export:5.5.0-SNAPSHOT.288 1 2592 2480 64 1440 88 1024 mod-data-export - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-kb-ebsco-java 2 mod-kb-ebsco-java:7.1.0-SNAPSHOT.211 2 1024 896 64 768 88 128 mod-kb-ebsco-java - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-notes 2 mod-notes:8.1.0-SNAPSHOT.361 2 1024 896 64 952 384 512 mod-notes - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96 mod-eusage-reports 3 mod-eusage-reports:3.1.1-SNAPSHOT.109 2 1024 896 64 768 88 128 mod-eusage-reports - Sidecar 1 N/A folio-module-sidecar:3.0.18.tls N/A 1024 512 64 256 0 96

 

Tenant Circulation Log Records Count cs00000int 1093310 cs00000int_0001 7148498

 

 

Methodology/Approach

  1. Data preparation step:

Data was generated using the following query:

  • Prepared datasets of 1K, 10K, 30K, 50K, 100K, 200K, and 300K records for both Central and Member tenants.

  • Each dataset was tagged with a descriptive label indicating its size for easy filtering for export testing.

WITH bc AS ( SELECT jsonb->>'barcode' AS barcode, row_number() OVER () AS rn FROM cs00000int_0001_mod_inventory_storage.item WHERE NULLIF(TRIM(jsonb->>'barcode'), '') IS NOT NULL LIMIT 500000 ), bc_cnt AS ( SELECT count(*) AS cnt FROM bc ), gen AS ( SELECT gen_random_uuid() AS new_id, gs, (gs % (SELECT cnt FROM bc_cnt)) + 1 AS idx FROM generate_series(1, 300000) gs ) INSERT INTO cs00000int_0001_mod_audit.circulation_logs (id, jsonb) SELECT g.new_id, jsonb_set( jsonb_set( jsonb_set( '{ "id": "", "date": "", "items": [ { "itemId": "27efe92f-1b5c-5b32-b6c7-48e71539b52e", "loanId": "0af86ff6-bc0f-4a62-a5ea-42976f1dc91f", "holdingId": "e8cd4664-1b53-5cc2-a139-ee2bc8229eaa", "instanceId": "243ad997-f24d-5f46-89a9-ff8b8b59c390", "itemBarcode": "" } ], "action": "Checked out", "object": "Loan", "source": "EBSCOAdmin, EBSCOAdmin", "linkToIds": { "userId": "0001ef6e-216a-4436-8ee7-42121519afe6" }, "description": "Type: Page. 300K", "userBarcode": "813487757", "servicePointId": "9d1b77e8-f02e-4b7f-b296-3f2042ddac54" }'::jsonb, '{id}', to_jsonb(g.new_id::text) ), '{date}', to_jsonb( to_char( now() - (random() * interval '365 days'), 'YYYY-MM-DD"T"HH24:MI:SS.MS"Z"' ) ) ), '{items,0,itemBarcode}', to_jsonb(b.barcode) ) FROM gen g JOIN bc b ON b.rn = g.idx;

 

  1. Test Configuration:

  • Tested Circulation Log export on Central and Member tenants.

  • Executed exports of 1K, 10K, 30K, 50K, 100K, 200K, and 300K records were performed manually, using filtering by description to select the target dataset for export.

  • Ran background CICO workload (8 users) on the Member tenant during exports.