Versions Compared

Old Version 17

changes.mady.by.user Olga Kondratenko

Saved on

compared with

New Version 18

changes.mady.by.user Olga Kondratenko

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents

...

Test #

Start time

End time

Instances number

Test Conditions

reindexing on Poppy Ramsons release, non-consortium environment

Duration *

Notes


1

2024-10-17T12:41:14

2024-10-17T14:06:46

10,099,620

Sequential: fs07000001

1 hour 25 min

  • mod-search:

  1. task count = 4

  2. Mem Hard Limit = 2592

  3. Mem Soft Limit = 2480

  4. Xmx = -XX:MaxRAMPercentage=85.0

  • mod-inventory-storage task count = 4

  • open search Data nodes instance scaled up to r6g.4xlarge.search

2

2024-10-17T14:35:59

2024-10-17T19:49:26

27,957,839

Sequential: fs09000000

5 hours 14 min

3

2024-10-17T19:58:07

2024-10-17T20:12:24

1,210,000

Sequential: fs07000002

14 min

4

2024-10-17T20:21:23

2024-10-17T22:46:34

In parallel: 3 tenants

All tenants reindex FAILED in
2 hours 25 min "MERGE_FAILED"

5

2024-10-16T14:38:08

2024-10-16T15:53:37

10,099,620

Sequential: fs07000001

1 hour 15 min

  • mod-search:

  1. task count = 4

  2. Mem Hard Limit = 4592

  3. Mem Soft Limit = 4480

  4. Xmx = -XX:MaxRAMPercentage=85.0

  • mod-inventory-storage task count = 4

  • open search Data nodes instance scaled up to r6g.4xlarge.search

6

2024-10-16T16:11:53

2024-10-16T21:08:00

27,957,839

Sequential: fs09000000

4 hours 57 min

7

2024-10-17T06:20:22

2024-10-17T06:34:04

1,210,000

Sequential: fs07000002

14 min

8

2024-10-17T06:40:22

2024-10-17T09:12:00

In parallel: 3 tenants

reindex FAILED for 1 tenant [tenantId: fs09000000, error: java.util.concurrent.CompletionException: org.folio.search.exception.FolioIntegrationException: Failed to publish reindex records range]

9

2024-10-15T10:16:50

2024-10-15T12:04:51

10,099,620

Sequential: fs07000001

1 hour 48 min

  • mod-search:4.0.0-SNAPSHOT.278

10

2024-10-16T08:01:14

2024-10-16T12:32:19

10,099,620

Sequential: fs07000001

4 hours 31 min

  • Data nodes instance scaled down to r6g.large.search

11

2024-10-21T09:49:10

2024-10-21T12:14:38

10,099,620

Sequential: fs07000001

2 hours 25 min

  • 2 instances of database db.r6g.xlarge (reader and writer)

12

2024-10-21T16:36:18

2024-10-21T18:26:15

10,099,620

Sequential: fs07000001

1 hour 50 min

  • 1 instance of database db.r6g.2xlarge

13

2024-10-22T09:16:23

2024-10-22T10:38:20

10,099,620

Sequential: fs07000001

1 hour 22 minutes

  • 1 instance of database db.r6g.4xlarge

...

Database use the same average amount of connections

...

Open Search metrics

CPU utilization percentage for all data nodes (Average).

...

Memory usage percentage for all data nodes (Average).

...

chrome_tuP3wGC9cE.png

Test #11 2 instances of db.r6g.xlarge database: writer and reader instances.

...

Infrastructure

PTF-environment rcp1

  • 10 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 

  • 1 instance of db.r6g.8xlarge database, writer instance.

  • MSK - fse-tenant
    4 kafka.m7g.xlarge brokers in 2 zones

    • Apache Kafka version 3.7.x

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true

    • log.retention.minutes=480

    • default.replication.factor=3

  • OpenSearch ptf-reindex-testcluster

    • OpenSearch version 2.13;

    • Data nodes

      • Availability Zone(s) - 2-AZ without standby

      • Instance type - r6g.4xlarge.search

      • Number of nodes - 4

      • EBS volume size (GiB) - 300

      • Provisioned IOPS - 3000IOPS

      • Provisioned Throughput (MiB/s) - 250 MiB/s

    • Dedicated master nodes
      Enabled - No

...