Versions Compared

Version Old Version 38 New Version 39
Changes made by

Serhii_Nosko

Aliaksandr Fedasiuk

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The following resources are used:

...

2 items for all items (except mod-data-import)

2 partition for all DI kafka topics:

Examples:

delete old topic
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --delete --topic perf-eks-folijet.Default.fs09000000.DI_ERROR

recreate topic with "--partitions 2 --replication-factor 1"
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --create --topic perf-eks-folijet.Default.fs09000000.DI_ERROR --partitions 2 --replication-factor 1
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Created topic perf-eks-folijet.Default.fs09000000.DI_ERROR.

get topic info
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --describe --topic perf-eks-folijet.Default.fs09000000.DI_ERROR
Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR PartitionCount: 2   ReplicationFactor: 1    Configs: min.insync.replicas=1,message.format.version=2.6-IV0,unclean.leader.election.enable=true
    Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 0    Leader: 1   Replicas: 1 Isr: 1
    Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 1    Leader: 2   Replicas: 2 Isr: 2 (edited) 


mod-data-import: -XX:MaxRAMPercentage=85.0 -XX:+UseG1GC / cpu: 128m/192m | memory: 1Gi/1Gi

...

envprofile

records

number

timetime in Lotus

Kafka

partition

number

module

instance

number

CPUdescription
MG Perf RancherPTF Create - 2
50007 min8 min22512/1024

2022-05-27T12:58:30.331+00:00

2022-05-27T13:05:08.683+00:00

MG Perf RancherPTF Create - 2
50007 min8 min22512/1024

NEW

2022-06-10T07:44:27.576+00:00

2022-06-10T07:51:11.140+00:00

MG Perf RancherPTF Create - 2
50007 min8 min22512/1024

NEW

-Ddi.flow.control.enable=false

2022-06-14T10:24:44.093+00:00

2022-06-14T10:31:54.725+00:00

MG Perf RancherPTF Update - 1
500010 min13 min22512/1024

2022-05-27T13:22:35.123+00:00

2022-05-27T13:32:35.344+00:00

MG Perf RancherPTF Create - 2
10`00021 min | 27min
19 min22512/1024
-Ddi.flow.control.enable=false

2022-05-30T09:51:13.876+00:00 | 2022-05-31T18:13:05.977+00:00

2022-05-30T10:12:33.982+00:00 | 2022-05-31T18:40:58.928+00:00

MG Perf RancherPTF Create - 2
10`00016 min
19 min22512/1024
NEW

2022-06-10T07:54:23.720+00:00

2022-06-10T08:08:48.484+00:00

MG Perf RancherPTF Create - 2
10`00016 min
19 min22512/1024
NEW
-Ddi.flow.control.enable=false

2022-06-14T10:36:41.482+00:00,
2022-06-14T10:53:03.556+00:00"

MG Perf RancherPTF Update - 1
10`00030 min25 min22512/1024
-Ddi.flow.control.enable=false

2022-05-31T19:19:46.296+00:00

2022-05-31T19:49:59.651+00:00

MG Perf RancherPTF Create - 2
10`00021 min
19 min22512/1024
-Ddi.flow.control.enable=true

2022-05-31T20:02:06.368+00:00

2022-05-31T20:23:19.490+00:00

MG Perf RancherPTF Update - 110`00031 min
25 min22512/1024
-Ddi.flow.control.enable=true

2022-06-01T19:08:11.563+00:00

2022-06-01T19:39:58.803+00:00

MG Perf RancherPTF Create - 2
10`00017 min
19 min22512/1024
-Ddi.flow.control.enable=true
-Ddi.flow.control.max.simultaneous.records=100 
-Ddi.flow.control.records.threshold=50

2022-06-03T09:20:07.654+00:00

2022-06-03T09:37:51.631+00:00

MG Perf RancherPTF Create - 2
30`0001h 6 min45 min22512/1024

2022-05-27T13:37:12.980+00:00

2022-05-27T14:31:52.595+00:00

MG Perf RancherPTF Update - 1
30`000 1h 26min-22512/1024

2022-05-27T15:37:33.580+00:00

2022-05-27T17:03:15.702+00:00

MG Perf RancherPTF Create - 2
50`0002h 37 min1h 25min22512/1024

3 errors: io.netty.channel.StacklessClosedChannelException

2022-06-01T19:48:33.977+00:00

2022-06-01T22:25:59.700+00:00

MG Perf RancherPTF Create - 250`00059 min1h 25min22512/1024

NEW

2022-06-10T08:12:29.178+00:00

2022-06-10T09:11:34.642+00:00

MG Perf RancherPTF Create - 2
100`000 2h 56min2h 24min (22 errors)22512/1024

NEW

2022-06-13T09:30:35.574+00:00

2022-06-13T12:26:52.484+00:00

MG Perf RancherPTF Create - 2
500`00014h 46min (60 errors)15h 37min (31 errors)22512/1024

NEW

60 errors

2022-06-13T14:27:40.568+00:00

2022-06-14T05:14:27.458+00:00



60 errors (500K - PTF Create - 2):

...

Almost all errors with mod-inventory

...

storage related to not having enough memory for instances (memory: 778Mi/846Mi).  Instances of mod-inventory-storage

...

were restarted 2 times.

Image Added


io.vertx.core.impl.NoStackTraceThrowable: {"errors":[{"message":"must not be null","type":"1","code":"javax.validation.constraints.NotNull.message","parameters":[{"key":"contributors[0].name","value":"null"}]}]}

...

io.vertx.core.impl.NoStackTraceThrowable: {"errors":[{"message":"must not be null","type":"1","code":"javax.validation.constraints.NotNull.message","parameters":[{"key":"contributors[2].name","value":"null"}]}]}


io.vertx.core.impl.NoStackTraceThrowable: proxyClient failure: mod-inventory-storage-23.1.0-SNAPSHOT.692 http://mod-inventory-storage: Connection was closed: POST /holdings-storage/holdings
io.vertx.core.impl.NoStackTraceThrowable: proxyClient failure: mod-inventory-storage-23.1.0-SNAPSHOT.692 http://mod-inventory-storage: Connection was closed: POST /instance-storage/instances
io.vertx.core.impl.NoStackTraceThrowable: proxyClient failure: mod-inventory-storage-23.1.0-SNAPSHOT.692 http://mod-inventory-storage: finishConnect(..) failed: Connection refused: mod-inventory-storage.folijet.svc.cluster.local/172.20.250.48:80: POST /holdings-storage/holdings

...