Versions Compared

Version Old Version 12 New Version 13
Changes made by

Hanna Tuhuzbayeva

Hanna Tuhuzbayeva

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
Overview

The purpose of the document is to assess reindexing performance on a consortium environment. Calculate reindex time and size of index.

Recommendations & Jiras

Test Summary

  • Reindexing process duration for consortuim environment takes:
    • 3 hours for 3 tenants in parallel (1.7M instances)
    • 2 hours for central tenant reindexing (1.2M instances)
    • 1 hour for secondary tenant (353K instances)
    • 20 minutes for secondary tenant (202K instances)

...

    • )

...


Test Runs /Results

Test #

Instances number

Test Conditions

reindexing on Poppy release, consortium environment

Duration *

Notes


1. 2023-11-28 09:20-12:20 UTC

1766108

In parallel: 3 tenants

3 hours
  • mod-search task count =8
  • mod-inventory-storage task count = 2
  • mod-okapi task count = 3
  • open search instance scaled up to r6g.4xlarge.search
  • without configuration of number_of_replicas and refresh_interval values of ES/OpenSearch

2. 2023-11-29 08:50 - 10:50 UTC

1212927

Sequential: cs00000int

2 hours

3. 2023-11-29 14:05 - 15:05 UTC

353179

Sequential: cs00000int_0001

1 hour
4. 2023-11-2915:22 - 15:42 UTC200002Sequential: cs00000int_000220 min

...

Infrastructure

PTF-environment pcon

  • 10 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 
  • 2 instances of db.r6g.xlarge database instances, one reader and one writer 
  • MSK ptf-kakfa-3
    • 4 m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true
    • log.retention.minutes=480
    • default.replication.factor=3

...