Modules:
Data Import Module (mod-data-import:2.8.0-SNAPSHOT.258)
...
Orders Storage Module (mod-orders-storage:13.6.0-SNAPSHOT.302)
Configuration:
Kafka: 2 partitions for each topic and the replication factor is 2
...
5k rec | 10k rec | 30k rec | 50k rec | 100k rec | |||
|---|---|---|---|---|---|---|---|
Profile: Default create instance & SRS MARC BIB | "startedDate" : "2023-03-15T17:22:44.416+00:00", "completedDate" : "2023-03-15T17:27:22.393+00:00", 3 min | "startedDate" : "2023-03-15T17:28:15.385+00:00", "completedDate" : "2023-03-15T17:39:07.001+00:00", 11 min | "startedDate" : "2023-03-15T17:40:05.862+00:00", "completedDate" : "2023-03-15T18:12:30.559+00:00", 32 min | a lot of StacklessClosedChannelException and Timeouts exceptions | a lot of StacklessClosedChannelException and Timeouts exceptions | ||
Profile: BF Default: MARC file import with creating of new instance, holding and item | "startedDate" : "2023-03-16T15:33:10.726+00:00", "completedDate" : "2023-03-16T15:38:55.351+00:00", 5 min/(MG: 7min) | "startedDate" : "2023-03-16T15:41:39.053+00:00", "completedDate" : "2023-03-16T15:55:22.105+00:00", 14 min/(MG: 16min) | "startedDate" : "2023-03-16T18:38:24.782+00:00", "completedDate" : "2023-03-16T19:24:57.711+00:00", 46 min(*) | a lot of StacklessClosedChannelException and Timeouts exceptions | a lot of StacklessClosedChannelException and Timeouts exceptions | ||
| Increase DB connection pool | |||||||
Profile: BF Default: MARC file import with creating of new instance, holding and item | "startedDate" : "2023-03-23T15:45:54.863+00:00", "completedDate" : "2023-03-23T15:51:36.077+00:00", 6 min/(MG: 7min) | "startedDate" : "2023-03-23T16:50:59.795+00:00", "completedDate" : "2023-03-23T17:04:49.625+00:00", 14 min/(MG: 16min) | not tested | "startedDate" : "2023-03-23T17:07:32.020+00:00", "completedDate" : "2023-03-23T18:01:08.354+00:00", 54 min/(MG: 59min) | "startedDate" : "2023-03-25T22:48:41.215+00:00", "completedDate" : "2023-03-26T02:25:15.039+00:00", 3h 37min/(MG: 2h 20min with errors) | ||
| Julian changes in inventory-storage | |||||||
Profile: BF Default: MARC file import with creating of new instance, holding and item | "startedDate" : "2023-03-16T20:05:48.739+00:00", "completedDate" : "2023-03-16T20:13:20.760+00:00", 8 min/(MG: 7min) | "startedDate" : "2023-03-16T20:13:52.860+00:00", "completedDate" : "2023-03-16T20:30:27.189+00:00", 17 min/(MG: 16min) | a lot of StacklessClosedChannelException and Timeouts exceptions | not tested | not tested | ||
| Increase DB connection pool (with inventory-storage changes) | |||||||
Profile: BF Default: MARC file import with creating of new instance, holding and item | "startedDate" : "2023-03-23T10:10:09.297+00:00", "completedDate" : "2023-03-23T10:23:39.168+00:00", 13 min/(MG: 7min) | "startedDate" : "2023-03-23T10:24:31.073+00:00", "completedDate" : "2023-03-23T10:43:41.657+00:00", 19 min/(MG: 16min) | "startedDate" : "2023-03-23T10:57:11.525+00:00", "completedDate" : "2023-03-23T11:39:07.009+00:00", 42 min | "startedDate" : "2023-03-23T11:58:47.557+00:00", "completedDate" : "2023-03-23T12:52:03.823+00:00", 54 min/(MG: 59min) | not tested | ||
Increase DB connection pool ( with record-storage changes) | |||||||
Profile: BF Default: MARC file import with creating of new instance, holding and item | "startedDate" : "2023-03-23T1523T19:4511:5430.863875+00:00", "completedDate" : "2023-03-23T1523T19:5119:3650.077519+00:00", 6 8 min/(MG: 7min) | "startedDate" : "2023-03-23T1624T09:5029:5952.795948+00:00", "completedDate" : "2023-03-23T1724T09:0447:49.625251+00:00", 14 18 min/(MG: 16min) | not tested | "startedDate" : "2023-03-23T1724T09:0750:3229.020560+00:00", "completedDate" : "2023-03-23T1824T11:0109:0818.354798+00:00", 54 min1h 19min/(MG: 59min) | Increase DB connection pool (with record-storage changes) | Profile: BF Default: MARC file import with creating of new instance, holding and item "startedDate" : "2023-03-23T1924T13:1133:3041.875732+00:00", "completedDate" : "2023-03-23T1924T16:1924:5053.519397+00:00", 8 min2h 51min/(MG: 7min) "startedDate" : "2023-03-24T09:29:52.948+00:00", "completedDate" : "2023-03-24T09:47:49.251+00:00", 18 min/(MG: 16min)59min) | not tested |
(*) mark: 1 error in Item creation: io.netty.channel.StacklessClosedChannelException
Performance degradation: needs to be investigated
| Jira Legacy | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
