...
The number of DB connections before the tests were about 750 during DI_T1(CT-2k_MT-SRI-50/50) maximal number was about 1750 and during DI_T2(CT-10k_MT-SRI-50/50) maximal number was about 2300.
...
DB Load
SQL queries
...
Methodology/Approach
All tests were run using the JMeter script.
Appendix
DI_T1(CT-2k_MT-SRI-50/50). This section contains durations for Data-import for jobs with “PTF-Create-3” on Central tenant: with 2K records and single record imports on 21 member tenants (50% with “Inventory Single Record - Default Update Instance“ profile, 50% with “Inventory Single Record - Default Create Instance“ profile).
...
Tenant
...
Duration
...
Central Tenant: DI_Import MARC BIB create 2K records
...
1 minute
...
SDIU_(Update) cs00000001_0022
...
12 sec
...
SDIU_(Update) cs00000001_0016
...
13 sec
...
SDIU_(Update) cs00000001_0013
...
13,9 sec
...
SDIU_(Update) cs00000001_0010
...
13,3 sec
...
SDIU_(Update) cs00000001_0008
...
11 sec
...
SDIU_(Update) cs00000001_0006
...
11 sec
...
SDIU_(Update) cs00000001_0005
...
11 sec
...
SDIU_(Update) cs00000001_0002
...
11,5 sec
...
SDIC_(Create) cs00000001_0021
...
12 sec
...
SDIC_(Create) cs00000001_0020
...
12 sec
...
SDIC_(Create) cs00000001_0019
...
11 sec
...
SDIC_(Create) cs00000001_0018
...
12 sec
...
SDIC_(Create) cs00000001_0017
...
12 sec
...
SDIC_(Create) cs00000001_0015
...
12 sec
...
SDIC_(Create) cs00000001_0014
...
12,7 sec
...
SDIC_(Create) cs00000001_0012
...
12 sec
...
SDIC_(Create) cs00000001_0011
...
12 sec
...
SDIC_(Create) cs00000001_0009
...
10 sec
...
SDIC_(Create) cs00000001_0007
...
5 sec
...
SDIC_(Create) cs00000001_0004
...
5,3 sec
...
SDIC_(Create) cs00000001_0003
...
5,8 sec
DI_T2(CT-10k_MT-SRI-50/50). This section contains durations for Data-import for jobs with “PTF-Create-3” on Central tenant: with 10K records and single record imports on 62 member tenants (50% with “Inventory Single Record - Default Update Instance“ profile, 50% with “Inventory Single Record - Default Create Instance“ profile).
...
Tenant
...
Duration
...
Central Tenant: DI_Importing MARC records workflow Transaction
...
5 minutes
...
SDIC_Single Record Import (Create) cs00000001_0002
...
5 sec
...
SDIC_Single Record Import (Create) cs00000001_0003
...
5 sec
...
SDIC_Single Record Import (Create) cs00000001_0006
...
5,6 sec
...
SDIC_Single Record Import (Create) cs00000001_0007
...
4,2 sec
...
SDIC_Single Record Import (Create) cs00000001_0008
...
4,1 sec
...
SDIC_Single Record Import (Create) cs00000001_0010
...
10 sec
...
SDIC_Single Record Import (Create) cs00000001_0012
...
6 sec
...
SDIC_Single Record Import (Create) cs00000001_0015
...
10 sec
...
SDIC_Single Record Import (Create) cs00000001_0017
...
11,3 sec
...
SDIC_Single Record Import (Create) cs00000001_0018
...
11,6 sec
...
SDIC_Single Record Import (Create) cs00000001_0021
...
12 sec
...
SDIC_Single Record Import (Create) cs00000001_0022
...
11 sec
...
SDIC_Single Record Import (Create) cs00000001_0023
...
16 sec
...
SDIC_Single Record Import (Create) cs00000001_0024
...
16 sec
...
SDIC_Single Record Import (Create) cs00000001_0025
...
16,5 sec
...
SDIC_Single Record Import (Create) cs00000001_0026
...
16,1 sec
...
SDIC_Single Record Import (Create) cs00000001_0030
...
15,6 sec
...
SDIC_Single Record Import (Create) cs00000001_0031
...
15,7 sec
...
SDIC_Single Record Import (Create) cs00000001_0032
...
15,7 sec
...
SDIC_Single Record Import (Create) cs00000001_0033
...
15,1 sec
...
SDIC_Single Record Import (Create) cs00000001_0034
...
15,5 sec
...
SDIC_Single Record Import (Create) cs00000001_0036
...
14,8 sec
...
SDIC_Single Record Import (Create) cs00000001_0039
...
15,6 sec
...
SDIC_Single Record Import (Create) cs00000001_0041
...
15,8 sec
...
SDIC_Single Record Import (Create) cs00000001_0047
...
15,1 sec
...
SDIC_Single Record Import (Create) cs00000001_0048
...
14,9 sec
...
SDIC_Single Record Import (Create) cs00000001_0049
...
15,4 sec
...
SDIC_Single Record Import (Create) cs00000001_0052
...
15,8 sec
...
SDIC_Single Record Import (Create) cs00000001_0055
...
15,3 sec
...
SDIC_Single Record Import (Create) cs00000001_0061
...
15,7 sec
...
SDIC_Single Record Import (Create) cs00000001_0062
...
15,8 sec
...
SDIU_Single Record Import (Update) cs00000001_0004
...
11,6 sec
...
SDIU_Single Record Import (Update) cs00000001_0005
...
10,3 sec
...
SDIU_Single Record Import (Update) cs00000001_0009
...
11,7 sec
...
SDIU_Single Record Import (Update) cs00000001_0011
...
11,8 sec
...
SDIU_Single Record Import (Update) cs00000001_0013
...
11,4 sec
...
SDIU_Single Record Import (Update) cs00000001_0014
...
11,6 sec
...
SDIU_Single Record Import (Update) cs00000001_0016
...
12,1 sec
...
SDIU_Single Record Import (Update) cs00000001_0019
...
11,3 sec
...
SDIU_Single Record Import (Update) cs00000001_0020
...
12,1 sec
...
SDIU_Single Record Import (Update) cs00000001_0027
...
16,1 sec
...
SDIU_Single Record Import (Update) cs00000001_0028
...
16,9 sec
...
SDIU_Single Record Import (Update) cs00000001_0029
...
17,1 sec
...
SDIU_Single Record Import (Update) cs00000001_0035
...
16,9 sec
...
SDIU_Single Record Import (Update) cs00000001_0037
...
16,5 sec
...
SDIU_Single Record Import (Update) cs00000001_0038
...
16 sec
...
SDIU_Single Record Import (Update) cs00000001_0040
...
18 sec
...
SDIU_Single Record Import (Update) cs00000001_0042
...
16,8 sec
...
SDIU_Single Record Import (Update) cs00000001_0043
...
17 sec
...
SDIU_Single Record Import (Update) cs00000001_0044
...
17,1 sec
...
SDIU_Single Record Import (Update) cs00000001_0045
...
16,5 sec
...
SDIU_Single Record Import (Update) cs00000001_0046
...
17,1 sec
...
SDIU_Single Record Import (Update) cs00000001_0050
...
16,5 sec
...
SDIU_Single Record Import (Update) cs00000001_0051
...
16,5 sec
...
SDIU_Single Record Import (Update) cs00000001_0053
...
16 sec
...
SDIU_Single Record Import (Update) cs00000001_0054
...
16,3 sec
...
SDIU_Single Record Import (Update) cs00000001_0056
...
15,5 sec
...
SDIU_Single Record Import (Update) cs00000001_0057
...
16,3 sec
...
SDIU_Single Record Import (Update) cs00000001_0058
...
16,6 sec
...
SDIU_Single Record Import (Update) cs00000001_0059
...
16,2 sec
...
SDIU_Single Record Import (Update) cs00000001_0060
...
15,9 sec
Infrastructure
PTF -environment mcpt
...
11 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
...
1 database instance, writer
Name Memory GIBv CPUsmax_connections
db.r6g.4xlarge 128 GiB; 16 vCPU
MSK tenant
...
2 m5.2xlarge brokers in 1 zones
...
Apache Kafka version 2.8.0
...
The graph on Database Load provides an illustration of the resource utilization database over a period of time when 2 tests were running. The key SQL queries that were sent are located in the right section of the figure
...
SQL queries
The SQL Queries section provides detailed information about the SQL queries executed on mcpt database. It includes metrics like the number of times each query was executed, the average time taken to execute each query, and the utilization of resources by each query.
...
Methodology/Approach
All tests were run using the JMeter script for single record import, large data import on the central tenant was started manually.
Appendix
DI_T1(CT-2k_MT-SRI-50/50). This section contains durations for Data-import for jobs with “PTF-Create-3” on Central tenant: with 2K records and single record imports on 21 member tenants (50% with “Inventory Single Record - Default Update Instance“ profile, 50% with “Inventory Single Record - Default Create Instance“ profile).
Tenant | Duration |
Central Tenant: DI_Import MARC BIB create 2K records | 1 minute |
SDIU_(Update) cs00000001_0022 | 12 sec |
SDIU_(Update) cs00000001_0016 | 13 sec |
SDIU_(Update) cs00000001_0013 | 13,9 sec |
SDIU_(Update) cs00000001_0010 | 13,3 sec |
SDIU_(Update) cs00000001_0008 | 11 sec |
SDIU_(Update) cs00000001_0006 | 11 sec |
SDIU_(Update) cs00000001_0005 | 11 sec |
SDIU_(Update) cs00000001_0002 | 11,5 sec |
SDIC_(Create) cs00000001_0021 | 12 sec |
SDIC_(Create) cs00000001_0020 | 12 sec |
SDIC_(Create) cs00000001_0019 | 11 sec |
SDIC_(Create) cs00000001_0018 | 12 sec |
SDIC_(Create) cs00000001_0017 | 12 sec |
SDIC_(Create) cs00000001_0015 | 12 sec |
SDIC_(Create) cs00000001_0014 | 12,7 sec |
SDIC_(Create) cs00000001_0012 | 12 sec |
SDIC_(Create) cs00000001_0011 | 12 sec |
SDIC_(Create) cs00000001_0009 | 10 sec |
SDIC_(Create) cs00000001_0007 | 5 sec |
SDIC_(Create) cs00000001_0004 | 5,3 sec |
SDIC_(Create) cs00000001_0003 | 5,8 sec |
DI_T2(CT-10k_MT-SRI-50/50). This section contains durations for Data-import for jobs with “PTF-Create-3” on Central tenant: with 10K records and single record imports on 62 member tenants (50% with “Inventory Single Record - Default Update Instance“ profile, 50% with “Inventory Single Record - Default Create Instance“ profile).
Tenant | Duration |
Central Tenant: DI_Importing MARC records workflow Transaction | 5 minutes |
SDIC_Single Record Import (Create) cs00000001_0002 | 5 sec |
SDIC_Single Record Import (Create) cs00000001_0003 | 5 sec |
SDIC_Single Record Import (Create) cs00000001_0006 | 5,6 sec |
SDIC_Single Record Import (Create) cs00000001_0007 | 4,2 sec |
SDIC_Single Record Import (Create) cs00000001_0008 | 4,1 sec |
SDIC_Single Record Import (Create) cs00000001_0010 | 10 sec |
SDIC_Single Record Import (Create) cs00000001_0012 | 6 sec |
SDIC_Single Record Import (Create) cs00000001_0015 | 10 sec |
SDIC_Single Record Import (Create) cs00000001_0017 | 11,3 sec |
SDIC_Single Record Import (Create) cs00000001_0018 | 11,6 sec |
SDIC_Single Record Import (Create) cs00000001_0021 | 12 sec |
SDIC_Single Record Import (Create) cs00000001_0022 | 11 sec |
SDIC_Single Record Import (Create) cs00000001_0023 | 16 sec |
SDIC_Single Record Import (Create) cs00000001_0024 | 16 sec |
SDIC_Single Record Import (Create) cs00000001_0025 | 16,5 sec |
SDIC_Single Record Import (Create) cs00000001_0026 | 16,1 sec |
SDIC_Single Record Import (Create) cs00000001_0030 | 15,6 sec |
SDIC_Single Record Import (Create) cs00000001_0031 | 15,7 sec |
SDIC_Single Record Import (Create) cs00000001_0032 | 15,7 sec |
SDIC_Single Record Import (Create) cs00000001_0033 | 15,1 sec |
SDIC_Single Record Import (Create) cs00000001_0034 | 15,5 sec |
SDIC_Single Record Import (Create) cs00000001_0036 | 14,8 sec |
SDIC_Single Record Import (Create) cs00000001_0039 | 15,6 sec |
SDIC_Single Record Import (Create) cs00000001_0041 | 15,8 sec |
SDIC_Single Record Import (Create) cs00000001_0047 | 15,1 sec |
SDIC_Single Record Import (Create) cs00000001_0048 | 14,9 sec |
SDIC_Single Record Import (Create) cs00000001_0049 | 15,4 sec |
SDIC_Single Record Import (Create) cs00000001_0052 | 15,8 sec |
SDIC_Single Record Import (Create) cs00000001_0055 | 15,3 sec |
SDIC_Single Record Import (Create) cs00000001_0061 | 15,7 sec |
SDIC_Single Record Import (Create) cs00000001_0062 | 15,8 sec |
SDIU_Single Record Import (Update) cs00000001_0004 | 11,6 sec |
SDIU_Single Record Import (Update) cs00000001_0005 | 10,3 sec |
SDIU_Single Record Import (Update) cs00000001_0009 | 11,7 sec |
SDIU_Single Record Import (Update) cs00000001_0011 | 11,8 sec |
SDIU_Single Record Import (Update) cs00000001_0013 | 11,4 sec |
SDIU_Single Record Import (Update) cs00000001_0014 | 11,6 sec |
SDIU_Single Record Import (Update) cs00000001_0016 | 12,1 sec |
SDIU_Single Record Import (Update) cs00000001_0019 | 11,3 sec |
SDIU_Single Record Import (Update) cs00000001_0020 | 12,1 sec |
SDIU_Single Record Import (Update) cs00000001_0027 | 16,1 sec |
SDIU_Single Record Import (Update) cs00000001_0028 | 16,9 sec |
SDIU_Single Record Import (Update) cs00000001_0029 | 17,1 sec |
SDIU_Single Record Import (Update) cs00000001_0035 | 16,9 sec |
SDIU_Single Record Import (Update) cs00000001_0037 | 16,5 sec |
SDIU_Single Record Import (Update) cs00000001_0038 | 16 sec |
SDIU_Single Record Import (Update) cs00000001_0040 | 18 sec |
SDIU_Single Record Import (Update) cs00000001_0042 | 16,8 sec |
SDIU_Single Record Import (Update) cs00000001_0043 | 17 sec |
SDIU_Single Record Import (Update) cs00000001_0044 | 17,1 sec |
SDIU_Single Record Import (Update) cs00000001_0045 | 16,5 sec |
SDIU_Single Record Import (Update) cs00000001_0046 | 17,1 sec |
SDIU_Single Record Import (Update) cs00000001_0050 | 16,5 sec |
SDIU_Single Record Import (Update) cs00000001_0051 | 16,5 sec |
SDIU_Single Record Import (Update) cs00000001_0053 | 16 sec |
SDIU_Single Record Import (Update) cs00000001_0054 | 16,3 sec |
SDIU_Single Record Import (Update) cs00000001_0056 | 15,5 sec |
SDIU_Single Record Import (Update) cs00000001_0057 | 16,3 sec |
SDIU_Single Record Import (Update) cs00000001_0058 | 16,6 sec |
SDIU_Single Record Import (Update) cs00000001_0059 | 16,2 sec |
SDIU_Single Record Import (Update) cs00000001_0060 | 15,9 sec |
Infrastructure
PTF -environment mcpt
11 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 database instance, writer
Name Memory GIBv CPUsmax_connections
db.r6g.4xlarge 128 GiB; 16 vCPU
| Expand | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
MSK “ptf-mobius-testing2“
2 m5.2xlarge brokers in 1 zone
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
Task definitions of related modules were changed to use shared Kafka topics.
| Expand | ||
|---|---|---|
| ||
|
Kafka topics were created with the suffix “mcpt.Default.ALL.” and used 2 partitions for involved topics topics,
The table contains modules, memory, and CPU parameters
...