Page Comparison

Test #

Start time

End time

Instances number

Test Conditions

reindexing on Ramsons release, non-consortium environment

Duration *

Notes

Objective

1

2024-10-17T12:41:14

2024-10-17T14:06:46

10,099,620

Sequential: fs07000001

1 hour 25 min

• mod-search:

1. task count = 4

2. Mem Hard Limit = 2592

3. Mem Soft Limit = 2480

4. Xmx = -XX:MaxRAMPercentage=85.0

• mod-inventory-storage task count = 4

• open search Data nodes instance scaled up to r6g.4xlarge.search

We tested mod-search version 4.0.0-SNAPSHOT.281 with the baseline configuration, and it delivered very good results. However, the database size is quite large (using db.r6g.8xlarge), so we plan to experiment with other types of database instances to explore potential optimizations.

2

2024-10-17T14:35:59

2024-10-17T19:49:26

27,957,839

Sequential: fs09000000

5 hours 14 min

3

2024-10-17T19:58:07

2024-10-17T20:12:24

1,210,000

Sequential: fs07000002

14 min

4

2024-10-17T20:21:23

2024-10-17T22:46:34

In parallel: 3 tenants

All tenants reindex FAILED in
2 hours 25 min "MERGE_FAILED"

5

2024-10-16T14:38:08

2024-10-16T15:53:37

10,099,620

Sequential: fs07000001

1 hour 15 min

• mod-search:

1. task count = 4

2. Mem Hard Limit = 4592

3. Mem Soft Limit = 4480

4. Xmx = -XX:MaxRAMPercentage=85.0

• mod-inventory-storage task count = 4

• open search Data nodes instance scaled up to r6g.4xlarge.search

We tested mod-search version 4.0.0-SNAPSHOT.281 with the following increased memory parameters to assess potential performance improvements:

• Mem Hard Limit: 4592 MB

• Mem Soft Limit: 4480 MB

However, the results showed no significant improvement in performance, indicating that increasing the memory parameters is not beneficial.

6

2024-10-16T16:11:53

2024-10-16T21:08:00

27,957,839

Sequential: fs09000000

4 hours 57 min

7

2024-10-17T06:20:22

2024-10-17T06:34:04

1,210,000

Sequential: fs07000002

14 min

8

2024-10-17T06:40:22

2024-10-17T09:12:00

In parallel: 3 tenants

reindex FAILED for 1 tenant [tenantId: fs09000000, error: java.util.concurrent.CompletionException: org.folio.search.exception.FolioIntegrationException: Failed to publish reindex records range]

9

2024-10-15T10:16:50

2024-10-15T12:04:51

10,099,620

Sequential: fs07000001

1 hour 48 min

• mod-search:4.0.0-SNAPSHOT.278

For this version, we encountered out-of-memory (OOM) issues with mod-search and attempted to resolve them by increasing the memory parameters:

• Mem Hard Limit: 4592 MB

• Mem Soft Limit: 4480 MB

However, these adjustments did not resolve the issue, so we updated mod-search to version 4.0.0-SNAPSHOT.281.

10

2024-10-16T08:01:14

2024-10-16T12:32:19

10,099,620

Sequential: fs07000001

4 hours 31 min

• Data nodes instance scaled down to r6g.large.search

This test was performed to check if it is possible to reindex with regular open search size without scaling it.
It is possible to reindex but it will take longer to reindex (approx 4 times longer for r6g.large.search compared to r6g.4xlarge.search)

11

2024-10-21T09:49:10

2024-10-21T12:14:38

10,099,620

Sequential: fs07000001

2 hours 25 min

• 2 instances of database db.r6g.xlarge (reader and writer)

The baseline database size, using the db.r6g.8xlarge instance, is quite large. A performance test was conducted to assess whether this size is optimal. The results indicate that it would be more efficient to use smaller instances, such as db.r6g.4xlarge or db.r6g.2xlarge, instead.

12

2024-10-21T16:36:18

2024-10-21T18:26:15

10,099,620

Sequential: fs07000001

1 hour 50 min

• 1 instance of database db.r6g.2xlarge

13

2024-10-22T09:16:23

2024-10-22T10:38:20

10,099,620

Sequential: fs07000001

1 hour 22 minutes

• 1 instance of database db.r6g.4xlarge