Page Comparison

Versions Compared

Old Version 12

changes.mady.by.user Leonid Kolesnykov

Saved on Jun 25, 2024

compared with

New Version Current

changes.mady.by.user Leonid Kolesnykov

Saved on Jun 25, 2024

Key

This line was added.
This line was removed.
Formatting was changed.

Table of Contents
Overview

...

Smoke Test finished successfully for 3 concurrent DI Create jobs of 50K each.

DI Create	# of test	Number of concurrent jobs	Main tenant (fs09000000)	Second tenant (fs07000001)	Third tenant (fs07000002)
10k	Baseline	1	00:05:01	00:05:01	00:05:00
	1	2	00:09:04	00:09:08
	2	3	00:14:25	00:15:12	00:15:22
25k	Baseline	1	00:10:43	00:11:15	00:11:22
	3	2	00:22:44	00:22:47
	4	3	00:40:32	00:40:17	00:40:25
DI Update
10k	Baseline	1	00:06:25
	5	2	00:14:08	00:13:44
	6	3	00:20:13	00:21:03	00:20:41
25k	Baseline	1	00:15:17
	7	2	00:35:27	00:35:34
	8	3	00:58:49	00:58:59	00:59:05
DI Create (Smoke test)
50K	9	1	22:44.2
	10	3	01:21:37	01:22:49	01:22:49

...

QUESNELIA						POPPY					COMPARISON Quesnelia/Poppy
DI Create	# of test	Number of concurrent jobs	Central tenant	Second tenant	Third tenant	DI Create	Number of concurrent jobs	Central tenant	Second tenant	Third tenant	Delta fs09000000	Delta fs07000001	Delta fs07000002
10k	Baseline	1	00:05:01	00:05:01	00:05:00	10k	1	00:04:56			00:00:05
	1	2	00:09:04	00:09:08			2	00:10:43	00:10:37		00:01:39	00:01:29
	2	3	00:14:25	00:15:12	00:15:22		3	00:21:12	00:21:06	00:20:57	00:06:47	00:05:54	00:05:35
25k	Baseline	1	00:10:43	00:11:15	00:11:22	25k	1	00:11:24			00:00:41
	3	2	00:22:44	00:22:47			2	00:23:44	00:23:30		00:01:00	00:00:43
	4	3	00:40:32	00:40:17	00:40:25		3	00:37:11	00:37:05	00:36:58	00:03:21	00:03:12	00:03:27
DI Update	# of test	Number of concurrent jobs	Central tenant	Second tenant	Third tenant	DI Update	Number of concurrent jobs	Central tenant	Second tenant	Third tenant	Delta fs09000000	Delta fs07000001	Delta fs07000002
10k	Baseline	1	00:06:25			10k	1	00:06:32			00:00:07
	5	2	00:14:08	00:13:44			2	00:09:47	00:11:26		00:04:21	00:02:18
	6	3	00:20:13	00:21:03	00:20:41		3	00:19:08	00:19:06	00:18:31	00:01:05	00:01:57	00:02:10
25k	Baseline	1	00:15:17			25k	1	00:15:13			00:00:04
	7	2	00:35:27	00:35:34			2	00:30:49	00:30:52		00:04:38	00:04:42
	8	3	00:58:49	00:58:59	00:59:05		3	00:47:47	00:48:17	00:47:54	00:11:02	00:10:42	00:11:11
50K	9	1	22:44.2			50K	1	00:22:31			00:00:13
	10	3	01:21:37	01:22:49	01:22:49		3	01:12:54	01:12:44	01:12:35	00:08:43	00:10:05	00:10:14

Compared with Data Import Creates + Updates Poppy pcp1

Service CPU Utilization

Expand

title	CPU utilization comparison

Module Quesnelia	CPU (Create 25k 2 tenants)	CPU (Update 25k 2 tenants)
mod-inventory-b	121.54	149.37
mod-quick-marc-b	92.33	92.09
mod-di-converter-storage-b	59.54	97.72
nginx-okapi	52.43	86.49
okapi-b	27.08	47.92
mod-source-record-storage-b	24.1	41.22
mod-inventory-storage-b	23.5	26.07
mod-permissions-b	8.32	32.46
mod-source-record-manager-b	15.54	19.42
mod-users-b	7.2	8.1
mod-pubsub-b	7.85	8.07
mod-password-validator-b	2.89	2.83
mod-feesfines-b	2.72	3.14
mod-configuration-b	2.38	3.65
mod-authtoken-b	2.19	2.86
mod-circulation-storage-b	1.88	2.14
mod-data-import-b	1.85	2.04
mod-circulation-b	0.41	0.4
pub-okapi	0.26	0.25

Service Poppy	CPU Create	CPU Update
mod-inventory-b	122.87	181.72
mod-di-converter-storage-b	78.94	75.21
mod-quick-marc-b	75.7	122.16
nginx-okapi	71.79	78.33
mod-source-record-storage-b	47.36	42.14
okapi-b	36.99	29.78
mod-source-record-manager-b	30.41	36.98
mod-inventory-storage-b	24.83	19.45
mod-users-b	19.33	5.61
mod-configuration-b	11.69	2.73
mod-permissions-b	9.19	18.71
mod-pubsub-b	6.97	6.85
mod-authtoken-b	6.51	3.44
mod-password-validator-b	3.27	2.75
mod-feesfines-b	2.29	2.5
mod-data-import-b	1.84	2.09
mod-circulation-storage-b	1.27	1.65
mod-circulation-b	0.33	0.34
pub-okapi	0.23	0.24

...

Infrastructure

PTF -environment qcp1

10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 database instance, writer

Name Memory GIB vCPUs max_connections
db.r6g.xlarge
32 GiB 4 vCPUs 2731
Number of records in DB:
- fs09000000
  - instances - 25901331
  - items - 27074913
  - holdings - 25871735
- fs07000001
  - nstances - 10100620
  - items - 1484850
  - holdings - 10522266
- fs07000002
  - nstances - 1161275
  - items - 1153548
  - holdings - 1153548
MSK tenant
- 4 m5.2xlarge brokers in 2 zones
- Apache Kafka version 2.8.0
- EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
Open Search ptf-test
- version OpenSearch_2_7_R20240502
- Data nodes
  - Instance type - r6g.2xlarge.search
  - Number of nodes - 4
  - Storage type - EBS
  - EBS volume size (GiB) - 500
- Dedicated master nodes
  - Instance type - r6g.large.search
  - Number of nodes - 3

...