| Table of Contents |
|---|
Overview
This document contains the results of the testing Data Import (batch imports) on 62 tenants with the following scenarios onthe MCPT cluster:
Central tenant: Large update imports - 2K; 30% Member tenants: single record imports (50% create, 50% update)
Central tenant: Large update imports - 10K; 100% Member tenants: single record imports (50% create, 50% update)
Summary
The average duration of single record imports (50% create, 50% update) during :
...
All Data-import finished with the status “Completed“
Most CPU consuming services were mod-consortia-b , mod-quick-marc-b, mod-inventory-b, mod-di-converter-storage-b, mod-source-record-storage-b, mod-inventory-storage-b, okapi-b, mod-source-record-manager-b it`s values are in Service CPU Utilization section
During 2 tests just 2 services showed changes in memory utilization: mod-data-import-b, mod-search-b, mod-inventory-b, mod-source-record-manager-b, mod-inventory-storage-b,mod-source-record-storage-b its values are in Service Memory Utilization section. No memory leaks.
DB CaPU usage during DI_T1(CT-2k_MT-SRI-50/50) has increased up to 80% and during DI_T2(CT-10k_MT-SRI-50/50) increased up to 95%.
Test Results
DI_T1(CT-2k_MT-SRI-50/50). This section contains durations for Data-import for jobs with “PTF-Create-3” on Central tenant: with 2K records and single record imports on 21 member tenants (50% with “Inventory Single Record - Default Update Instance“ profile, 50% with “Inventory Single Record - Default Create Instance“ profile).
...
Dependency graph illustrating the change in data import duration across tenants for two profiles “Inventory Single Record - Default Update Instance” and ”Inventory Single Record - Default Create Instance”
...
Resource utilization
This section presents a comprehensive collection of key graphs and metrics necessary for understanding system performance during tests. This area serves as a portal for visualizing and decoding how the various resources have been used and impacted during different stages of performance testing.
Service CPU Utilization
Service | DI_T1(CT-2k_MT-SRI-50/50). | DI_T2(CT-10k_MT-SRI-50/50). |
mod-consortia-b | 179% | 189% |
mod-quick-marc-b | 152% | 80% |
mod-inventory-b | 99% | 73% |
mod-di-converter-storage-b | 95% | 105% |
mod-source-record-storage-b | 49% | 48% |
mod-inventory-storage-b | 36% | 32% |
okapi-b | 28% | 37% |
mod-source-record-manager-b | 26% | 29% |
...
Memory Utilization
During testing, the memory usage increases only on the services that are in the table below, also Memory usage values before test 1 and test2 and after the tests are in the corresponding columns
Service | MU before the T1, T2 | MU after the T1, T2 |
mod-data-import-b | 36 | 50 |
mod-search-b | 24 | 31 |
mod-inventory-b | 15 | 28 |
mod-source-record-manager-b | 14 | 22 |
mod-inventory-storage-b | 13 | 23 |
mod-source-record-storage-b | 12 | 16 |
...
DB CPU Utilization
CPU usage before tets was about 3%, during DI_T1(CT-2k_MT-SRI-50/50) CPU usage has increased up to 80% and during DI_T2(CT-10k_MT-SRI-50/50) CPU usage has increased up to 95%.
...
DB Connections
The number of DB connections before the tests were about 750 during DI_T1(CT-2k_MT-SRI-50/50) maximal number was about 1750 and during DI_T2(CT-10k_MT-SRI-50/50) maximal number was about 2300.
...
DB Load
SQL queries
...
Methodology/Approach
All tests were run using the JMeter script. Data has been pre-generated for all tenants
Infrastructure
PTF -environment mcpt
11 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 database instance, writer
Name Memory GIBv CPUsmax_connections
db.r6g.4xlarge 128 GiB; 16 vCPU
MSK tenant
2 m5.2xlarge brokers in 1 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
...