...
...
...
...
...
...
...
...
...
...
...
The following resources are used:
- 6 m4large EC2 spot instances for kubernetes Kubernetes cluster;
- 1 db.r5.xlarge instance for RDS service (writer)
- one m5.large per 2 zones for kafks on MSK
...
2 items for all items (except mod-data-import)
2 partition for all DI kafka topics:
Examples:
delete old topic
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --delete --topic perf-eks-folijet.Default.fs09000000.DI_ERROR
recreate topic with "--partitions 2 --replication-factor 1"
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --create --topic perf-eks-folijet.Default.fs09000000.DI_ERROR --partitions 2 --replication-factor 1
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Created topic perf-eks-folijet.Default.fs09000000.DI_ERROR.
get topic info
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --describe --topic perf-eks-folijet.Default.fs09000000.DI_ERROR
Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR PartitionCount: 2 ReplicationFactor: 1 Configs: min.insync.replicas=1,message.format.version=2.6-IV0,unclean.leader.election.enable=true
Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 0 Leader: 1 Replicas: 1 Isr: 1
Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 1 Leader: 2 Replicas: 2 Isr: 2
Tests:
| env | profile | records number | time | kafkaKafka partition number | module instance number | cpuCPU | decriptiondescription |
|---|---|---|---|---|---|---|---|
| MG Perf Rancher | PTF Create - 2 | 5000 | 8 min | 2 | 2 | 128 | 2022-05-24T14:24:20.945+00:00 2022-05-24T14:32:39.051+00:00 |
| MG Perf Rancher | PTF Update - 1 | 5000 | 25 min | 2 | 2 | 128 | 2022-05-24T14:39:07.476+00:00 2022-05-24T15:04:01.426+00:00 |
...