Overview
The purpose of the document is to assess reindexing performance on a consortium environment. Calculate reindex time and size of index.
...
Test # | Instances number | Test Conditions reindexing on Poppy release, consortium environment | Duration | Notes |
1. 2023-11-28 09:20-12:20 UTC | 1766108 | In parallel: 3 tenants | 3 hours |
|
2. 2023-11-29 08:50 - 10:50 UTC | 1212927 | Sequential: cs00000int | 2 hours | |
3. 2023-11-29 14:05 - 15:05 UTC | 353179 | Sequential: cs00000int_0001 | 1 hour | |
| 4. 2023-11-2915:22 - 15:42 UTC | 200002 | Sequential: cs00000int_0002 | 20 min |
...
| Tenant | Reindex id | Get request reindex |
|---|---|---|
| cs00000int | bb944cf4-b99f-4aa3-b13e-f5c92dc630ed | 1212891 |
| cs00000int_001 | cf943d63-50db-4085-9629-783d7acdc67b | 353179 |
| cs00000int_002 | c62e2662-6a21-47fd-9bb8-eea11364c2c1 | 200002 |
| SUM | 1766072 |
Service CPU Utilization
Test #1 (3 tenants in parallel)
Test #2 (cs00000int maint tenant, sequental)
Test #3 and #4 (cs00000int_0001, cs00000int_0002 secondary tenants, sequental)
Memory Utilization
Test #1 (3 tenants in parallel)
Test #2 (cs00000int maint tenant, sequental)
Test #3 and #4 (cs00000int_0001, cs00000int_0002 secondary tenants, sequental)
DB CPU Utilization
Test #1 (3 tenants in parallel)
Test #2 (cs00000int maint tenant, sequental)
Test #3 and #4 (cs00000int_0001, cs00000int_0002 secondary tenants, sequental)
DB Connections
Test #1 (3 tenants in parallel)
Test #2 (cs00000int maint tenant, sequental)
Test #3 and #4 (cs00000int_0001, cs00000int_0002 secondary tenants, sequental)
Open Search CPU
Test #1 (3 tenants in parallel)
Test #2 (cs00000int maint tenant, sequental)
Test #3 and #4 (cs00000int_0001, cs00000int_0002 secondary tenants, sequental)
Open Search Ibdexing Data Rate
Test #1 (3 tenants in parallel)
Test #2 (cs00000int maint tenant, sequental)
Test #3 and #4 (cs00000int_0001, cs00000int_0002 secondary tenants, sequental)
Open Search Ibdexing Latency
Test #1 (3 tenants in parallel)
Test #2 (cs00000int maint tenant, sequental)
Test #3 and #4 (cs00000int_0001, cs00000int_0002 secondary tenants, sequental)
Appendix
Infrastructure
PTF-environment pcon
- 10 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
- 2 instances of db.r6g.xlarge database instances, one reader and one writer
- MSK ptf-kakfa-3
- 4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
Number of instances
| Tenant | Source | Instance number | Instances sum |
|---|---|---|---|
| cs00000int | FOLIO | 115035 | 1212927 |
| MARC | 1097892 | ||
| cs00000int_0001 | CONSORTIUM-FOLIO | 38712 | 353179 |
| CONSORTIUM-MARC | 304467 | ||
| FOLIO | 1000 | ||
| MARC | 9000 | ||
| cs00000int_0002 | CONSORTIUM-MARC | 4 | 200002 |
| FOLIO | 30000 | ||
| MARC | 169998 |
Methodology/Approach
- Use consortium cluster for testing (pcon in our case).
- Configure the environment in accordance with Infrastructure parameters to the one that FSE commonly uses.
- Run reindex, get the results for indexing time, index size. Use Steps for testing process#Reindex for details.
- Compare results.