Following resources are used:

• 6 m4large EC2 spot instances for kubernetes cluster;
• 1 db.r5.xlarge instance for RDS service (writer)
• one m5.large per 2 zones for kafks on MSK

Previous Lotus testing performance results:

Lotus Snapshot Performance testing

Modules:

Data Import Module (mod-data-import-2.5.0-SNAPSHOT.231)

Source Record Manager Module (mod-source-record-manager-3.4.0-SNAPSHOT.621)

Source Record Storage Module (mod-source-record-storage-5.4.0-SNAPSHOT.426)

Inventory Module (mod-inventory-18.2.0-SNAPSHOT.537) -  mod-inventory-18.0.0

Inventory Storage Module (mod-inventory-storage-23.1.0-SNAPSHOT.692)

Data Import Converter Storage (mod-data-import-converter-storage-1.14.0-SNAPSHOT.202)

Invoice business logic module (mod-invoice-5.4.0-SNAPSHOT.306)

Data Export Module (mod-data-export-4.5.0-SNAPSHOT.319)

Performance-optimized configuration:

MAX_REQUEST_SIZE = 4000000 (for all modules)

2 items for all items (except mod-data-import)

2 partition for all DI kafka topics:

Examples:

delete old topic
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --delete --topic perf-eks-folijet.Default.fs09000000.DI_ERROR

recreate topic with "--partitions 2 --replication-factor 1"
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --create --topic perf-eks-folijet.Default.fs09000000.DI_ERROR --partitions 2 --replication-factor 1
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Created topic perf-eks-folijet.Default.fs09000000.DI_ERROR.

get topic info
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --describe --topic perf-eks-folijet.Default.fs09000000.DI_ERROR
Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR PartitionCount: 2   ReplicationFactor: 1    Configs: min.insync.replicas=1,message.format.version=2.6-IV0,unclean.leader.election.enable=true
    Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 0    Leader: 1   Replicas: 1 Isr: 1
    Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 1    Leader: 2   Replicas: 2 Isr: 2 (edited) 

Tests: