Table of Contents
Overview

...

Original ticket to test:
Jira Legacy
server System JiraJIRA
serverId 01505d01-b853-3c2e-90f1-ee9b165564fc
key PERF-635
Additional info ElasticSearch Reindex Performance Recommendations

Test Summary

Reindexing process for consortium environment takes:
- 3 hours for 3 tenants in parallel (1.7M instances);
- 2 hours for central tenant reindexing (1.2M instances);
- 1 hour for secondary tenant reindexing (353K instances);
- 20 minutes for secondary tenant reindexing (202K instances).
Duration depends not only on instance number, but also its type (source). cs00000int_0001 tenant has a lot of shared instances (343K) but much less unshared ones (10K) compared to cs00000int_0002 (200K unshared). Data details can be found here: Datastructure
High CPU utilization is observed on nginx-okapi module - up to 413% during 3 tenants test.
CPU utilization for mod-inventory-storage reached 102% during test on central tenant. mod-search CPU utilization was about 13-27% during all the tests.
No memory leaks suspected.

...

Test #	Instances number	Test Conditions reindexing on Poppy release, consortium environment	Duration *	Notes
1. 2023-11-28 09:20-12:20 UTC	1766108	In parallel: 3 tenants	3 hours	mod-search task count =8 mod-inventory-storage task count = 2 mod-okapi task count = 3 open search instance scaled up to r6g.4xlarge.search without configuration of number_of_replicas and refresh_interval values of ES/OpenSearch
2. 2023-11-29 08:50 - 10:50 UTC	1212927	Sequential: cs00000int	2 hours
3. 2023-11-29 14:05 - 15:05 UTC	353179	Sequential: cs00000int_0001	1 hour
4. 2023-11-2915:22 - 15:42 UTC	200002	Sequential: cs00000int_0002	20 min

...

Results from get-request for reindex monitoring:

...

PTF-environment pcon

...