The following resources are used:

...

2 items for all items (except mod-data-import)

2 partition for all DI kafka topics:

Examples:

delete old topic
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --delete --topic perf-eks-folijet.Default.fs09000000.DI_ERROR

recreate topic with "--partitions 2 --replication-factor 1"
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --create --topic perf-eks-folijet.Default.fs09000000.DI_ERROR --partitions 2 --replication-factor 1
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Created topic perf-eks-folijet.Default.fs09000000.DI_ERROR.

get topic info
# ./kafka-topics.sh --bootstrap-server=<kafka-ip>:9092 --describe --topic perf-eks-folijet.Default.fs09000000.DI_ERROR
Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR PartitionCount: 2   ReplicationFactor: 1    Configs: min.insync.replicas=1,message.format.version=2.6-IV0,unclean.leader.election.enable=true
    Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 0    Leader: 1   Replicas: 1 Isr: 1
    Topic: perf-eks-folijet.Default.fs09000000.DI_ERROR Partition: 1    Leader: 2   Replicas: 2 Isr: 2 (edited)

mod-data-import: -XX:MaxRAMPercentage=85.0 -XX:+UseG1GC / cpu: 128m/192m | memory: 1Gi/1Gi

...

6210527T1258303310527T130508683 Create 27 8 -Ddi.flow.control.enable=false14T10244409314T103154725 Update 1500011 13 20T06184674820T063002991 Update 1500010 13

2022-05-27T13:22:35.123+00:00

2022-05-27T13:32:35.344 Create 221 | 27min19 -Ddi.flow.control.enable=false0530T095113.876+:00 | 2022-05-31T18:13:05.9770530T101233982 | 2022-05-31T18:40:58.928+00:0010`00016 19 min10T07542372010T080848484 Create 210`00016 19 min-Ddi.flow.control.enable=false14T103648214T105303556 Update - 1
20T07070059420T07285490525 min-Ddi.flow.control.enable=false0531T1919462960531T19495965119 min-Ddi.flow.control.enable=true0531T2002063680531T202319490

env	profile	records number	time	time in Lotus	Kafka partition number	module instance number	CPU	description
MG Perf Rancher	PTF Create - 2	5000	7 min	8 min	2	2	512/1024	mod-source-record-manager-3.4.0-SNAPSHOT.	659+ 2022-	06-	10T07:	44:	27.	576+00:00 2022-	06-	10T07:	51:	11.	140+00:00
MG Perf Rancher	PTF Create - 2	5000	7 min	8 min	2	2	512/1024	mod-source-record-manager-3.4.0-SNAPSHOT.659+ -Ddi.flow.control.enable=false 2022-06-10T0714T10:24:44:27.576093+00:00 2022-06-10T0714T10:5131:1154.140725+00:00
MG Perf Rancher	PTF	Update -	1	5000	11 min	13 min	2	2	512/1024	mod-source-record-manager-3.4.0-SNAPSHOT.659+	2022-06-	20T06:	18:	46.	748+00:00 2022-06-	20T06:	30:	02.	991+00:00
MG Perf Rancher	PTF	Create -	2	10`000	16 min	19 min	2	2	512/1024	mod-source-record-manager-3.4.0-SNAPSHOT.659+ 2022-06-	10T07:	54:	23.	720+00:00 2022-06-	10T08:	08:	48.	484+00:00
MG Perf Rancher	PTF	Create -	2	10`000	16 min	19 min	2	2	512/1024	mod-source-record-manager-3.4.0-SNAPSHOT.659+ -Ddi.flow.control.enable=false 2022-06-14T10:36:41.482+00:00 2022-06-14T10:53:03.556+00:00
MG Perf Rancher	PTF	Update -	1	10`000	22 min		25 min	2	2	512/1024	mod-source-record-manager-3.4.0-SNAPSHOT.659+ 2022-	06-	20T07:	07:	00	.594+00:00 2022-	06-	20T07:	28:	54.	905+00:00
MG Perf Rancher	PTF Create - 2	50`000	59 min	1h 25min	2	2	512/1024	mod-source-record-manager-3.4.0-SNAPSHOT.659+ 2022-06-	10T08:	12:	29.	178+00:00 2022-06-	10T09:	11:	34.	642+00:00
MG Perf Rancher	PTF	Update -	1	50`000	1h 42 min	2h 17min	2	2	512/1024	mod-source-record-manager-3.4.0-SNAPSHOT.659+	2022-06-	20T09:	11:41.	701+00:00 2022-06-	20T10:	54:	29.	378+00:00
MG Perf Rancher	PTF	10`000	22 min	25 min	Create - 2	100`000	2h 20min	2h 24min (22 errors)	2	2	512/1024	mod-source-record-manager-3.4.0-SNAPSHOT.659+ 2022-06-	13T09:	30:	35.	574+00:00 2022-06-	13T12:	26:	52.	484+00:00
MG Perf Rancher	PTF Update - 1	10`000	30 min	100`000	2h 49min	4h 40min (tests were made for 1 instance number and partition number	2	2	512/1024	mod-source-record-manager-3.4.0-SNAPSHOT.659+ 2022-	06-	21T11:	46:	43.	175+00:00 2022-	06-	21T14:	36:	05.	532+00:00	MG Perf Rancher	PTF Create - 2	10`000	21 min	57 errors Inventory/Inventory-storage errors: io.netty.channel.StacklessClosedChannelException, io.vertx.core.impl.NoStackTraceThrowable: Connection is not active now, current status: CLOSED io.vertx.core.impl.NoStackTraceThrowable: Timeout
MG Perf Rancher	PTF Create - 2	500`000	14h 46min (60 errors)	15h 37min (31 errors)	2	2	512/1024	mod-source-record-manager-3.4.0-SNAPSHOT.659+ 60 errors 2022-	06-	13T14:	27:	40.	568+00:00 2022-	06-	14T05:	14:	27.	458+00:00

Results before flow control fix: MODSOURMAN-811

env

profile

records

number

time

time in Lotus

Kafka

partition

number

module

instance

number

CPU

description

MG Perf Rancher

PTF

Update

Create -

1

2

10`000

5000

31

7 min

25

8 min

2

512/1024

-Ddi.flow.control.enable=true

mod-source-record-manager-3.4.0-SNAPSHOT.621

2022-

06

05-

01T19

27T12:

08

58:

11

30.

563

331+00:00

2022-

06

05-

01T19

27T13:

39

05:

58

08.

803

683+00:00
MG Perf Rancher	PTF

Create

Update -

2

1

10`000

5000

17

10 min

19

13 min

2

512/1024

-Ddi.flow.control.enable=true
-Ddi.flow.control.max.simultaneous.records=100
-Ddi.flow.control.records.threshold=50

2022-

06

05-

03T09

27T13:

20

22:

07

35.

654

123+00:00

2022-

06

05-

03T09

27T13:

37

32:

51

35.

631

344+00:00
MG Perf Rancher	PTF Create - 2

30`000

10`000

1h 6

21 min | 27min

45

19 min

2

512/1024

-Ddi.flow.control.enable=false

2022-05-

27T13

30T09:

37

51:

12

13.

980

876+00:00

| 2022-05-

27T14

31T18:

31:52.595+00:00

MG Perf Rancher

PTF Update - 1

30`000

1h 26min

-

2

512/1024

13:05.977+00:00

2022-05-

27T15

30T10:

37

12:33.

580

982+00:00

| 2022-05-

27T17

31T18:

03

40:

15

58.

702

928+00:00
MG Perf Rancher	PTF

Create

Update -

2

1

50`000

10`000

2h 37

30 min

1h 25min

25 min

2

512/1024

3 errors: io.netty.channel.StacklessClosedChannelException

-Ddi.flow.control.enable=false

2022-

06

05-

01T19

31T19:

48

19:

33

46.

977

296+00:00

2022-

06

05-

01T22

31T19:

25

49:59.

700

651+00:00
MG Perf Rancher	PTF Create - 2

50`000

10`000

59

21 min

1h 25min

19 min

2

512/1024

mod-source-record-manager-3.4.0-SNAPSHOT.659+

-Ddi.flow.control.enable=true

2022-

06

05-

10T08

31T20:

12

02:

29

06.

178

368+00:00

2022-

06

05-

10T09

31T20:

11

23:

34

19.

642

490+00:00
MG Perf Rancher	PTF Update - 1

50`000

10`000

1h 42

31 min

2h 17min

25 min

2

512/1024

mod-source-record-manager-3.4.0-SNAPSHOT.659+

-Ddi.flow.control.enable=true

2022-06-

20T09

01T19:08:11

:41

.

701

563+00:00

2022-06-

20T10

01T19:

54

39:

29

58.

378

803+00:00
MG Perf Rancher	PTF Create - 2

100`000

2h 20min

2h 24min

(22 errors)

10`000

17 min

19 min

2

512/1024

mod-source-record-manager-3.4.0-SNAPSHOT.659+

-Ddi.flow.control.enable=true
-Ddi.flow.control.max.simultaneous.records=100
-Ddi.flow.control.records.threshold=50

2022-06-

13T09

03T09:

30

20:

35

07.

574

654+00:00

2022-06-

13T12

03T09:

26

37:

52

51.

484

631+00:00
MG Perf Rancher	PTF

Update

Create -

1

2

100`0002h 49min

4h 40min

(tests were made for 1 instance number and partition number

30`000

1h 6 min

45 min

2

512

/1024mod-source-record-manager-3.4.0-SNAPSHOT.659+

/1024

2022-

06

05-

21T11

27T13:

46

37:

43

12.

175

980+00:00

2022-

06

05-

21T14

27T14:

36

31:

05

52.

532

595+00:00

57 errors Inventory/Inventory-storage errors:

io.netty.channel.StacklessClosedChannelException,

io.vertx.core.impl.NoStackTraceThrowable: Connection is not active now, current status: CLOSED

io.vertx.core.impl.NoStackTraceThrowable: Timeout

MG Perf Rancher

PTF Update - 1

30`000

1h 26min

-

2

512/1024

2022-05-27T15:37:33.580+00:00

2022-05-27T17:03:15.702+00:00

MG Perf Rancher

PTF Create -

2

500`000

14h 46min

(60 errors)

15h 37min

(31 errors)

2

50`000

2h 37 min

1h 25min

2

512/1024

mod-source-record-manager-3.4.0-SNAPSHOT.659+

60 errors

3 errors: io.netty.channel.StacklessClosedChannelException

2022-06-

13T14

01T19:

27

48:

40

33.

568

977+00:00

2022-06-

14T05

01T22:

14

25:

27

59.

458

700+00:00

60 errors (500K - PTF Create - 2):

...

Versions Compared

Old Version 49

New Version 50

Key

The following resources are used:

Results before flow control fix: MODSOURMAN-811

60 errors (500K - PTF Create - 2):

Page Comparison

Versions Compared

Old Version 49

New Version 50

Key

Results before flow control fix: MODSOURMAN-811

60 errors (500K - PTF Create - 2):