Table of Contents

Steps:

Create a namespace with Bugfest dataset

...

Pic. 1 Example "Kafka UI topics & partitions"

Adjust Kafka messages retention (OPTIONAL)

Before starting of indexation find log.retention.minutes (if log.retention.ms is null) property and set it to 24 hours (1440 minutes). Could be set on broker level.

If decided to do this on a topic level only - log.retention.ms should be changed because it has higher precedence over ..minutes and it's usually already set to some value.

Tune mod-search config(REQUIRED)

KAFKA_EVENTS_CONCURRENCY (default - 2) with higher value could increase instances reindex.

...

Considering that there should always be more subjects/contributors than instances - only subjects/contributors should be tuned. If there's an observation that subjects/contributors are read from topic faster than published - then we may want to also tune instances topic.

Scale-up backend modules(REQUIRED)

For better performance, please scale up backend modules.

...

For ECS Consortia tenants

I In pgadmin ran this query to identify current value and change the value to false as in the screenshot

...

After completion of all pre-required steps, trigger index with POST Postman request.

URI: /search/index/inventory/reindex

Headers: X-Okapi-Tenant & X-Okapi-Token

Body: 

{
  "recreateIndex": true,
  "resourceName": "instance"
}

...

After the indexation process is finished, do not forget to scale down the shared OpenSearch AWS service

r6g.2xlarge → r6g.xlarge (If no other conditions/constraints)

Adjust Kafka messages retention back

Return previous value to log.retention property (usually 8 hours).

Tune mod-search config back

Return modified env variables to default values