The following resources are used:
...
2 items for all items (except mod-data-import)
2 partition for all DI kafka topics:
Examples:
delete old topic
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --delete --topic perf-eks-folijet.Default.fs09000000.DI_ERROR
recreate topic with "--partitions 2 --replication-factor 1"
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --create --topic perf-eks-folijet.Default.fs09000000.DI_ERROR --partitions 2 --replication-factor 1
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Created topic perf-eks-folijet.Default.fs09000000.DI_ERROR.
get topic info
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --describe --topic perf-eks-folijet.Default.fs09000000.DI_ERROR
Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR PartitionCount: 2 ReplicationFactor: 1 Configs: min.insync.replicas=1,message.format.version=2.6-IV0,unclean.leader.election.enable=true
Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 0 Leader: 1 Replicas: 1 Isr: 1
Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 1 Leader: 2 Replicas: 2 Isr: 2
mod-data-import: -XX:MaxRAMPercentage=85.0 -XX:+UseG1GC / cpu: 128m/192m | memory: 1Gi/1Gi
...
mod-inventory-storage: -XX:MaxRAMPercentage=80 -XX:MetaspaceSize=120M -XX:+UseG1GC / DB_MAXPOOLSIZE = 15 / cpu: 512m/512m | memory: 778Mi/846Mi
Tests:
| env | profile | records number | time | time in Lotus | Kafka partition number | module instance number | CPU | description |
|---|---|---|---|---|---|---|---|---|
| MG Perf Rancher | PTF Create - 2 | 5000 | 7 min | 8 min | 2 | 2 | 512/1024 | 2022-05-27T12:58:30.331+00:00 2022-05-27T13:05:08.683+00:00 |
| MG Perf Rancher | PTF Update - 1 | 5000 | 10 min | 13 min | 2 | 2 | 512/1024 | 2022-05-27T13:22:35.123+00:00 2022-05-27T13:32:35.344+00:00 |
| MG Perf Rancher | PTF Create - 2 | 10`000 |
| 21 min | 19 min | 2 | 2 | 512/1024 | 2022-05- |
30T09: |
51: |
13. |
876+00:00 2022-05- |
30T10: |
12: |
33. |
982+00:00 | ||||||||
| MG Perf Rancher | PTF Update - 1 | 10`000 | 1h 23 min | 25 min | 2 | 2 | 512/1024 | 2022-05-25T17:57:06.429+00:00 2022-05-25T19:20:08.545+00:00 |
| MG Perf Rancher | PTF Create - 2 | 30`000 | 1h 6 min | 45 min | 2 | 2 | 512/1024 | 2022-05-27T13:37:12.980+00:00 2022-05-27T14:31:52.595+00:00 |
| MG Perf Rancher | PTF Update - 1 | 30`000 | 1h 26min | - | 2 | 2 | 512/1024 | 2022-05-27T15:37:33.580+00:00 2022-05-27T17:03:15.702+00:00 |