Skip to end of banner
Go to start of banner

Folijet - Performance of importing orders on Orchid Snapshot

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 33 Current »

Modules:

Data Import Module (mod-data-import:2.8.0-SNAPSHOT.258)

Data Import Converter Storage (mod-di-converter-storage:2.1.0-SNAPSHOT.8)

Source Record Manager Module (mod-source-record-manager:3.7.0-SNAPSHOT.759)

Source Record Storage Module (mod-source-record-storage:5.7.0-SNAPSHOT.494)

Inventory Module (mod-inventory:20.1.0-SNAPSHOT.604)

Inventory Storage Module (mod-inventory-storage:26.1.0-SNAPSHOT.826)

Orders Module (mod-orders:12.7.0-SNAPSHOT.654)

Orders Storage Module (mod-orders-storage:13.6.0-SNAPSHOT.302)

Configuration:

Kafka: shared cluster with 2 nodes (replication factor is 2)


mod-data-import: CPU:256/128, mem:2048Mi/1844Mi, -XX:MaxMetaspaceSize=512m -Xmx1292m

mod-di-converter-storage: CPU:128/128, mem:1024Mi/896Mi,  -XX:MaxMetaspaceSize=128m -Xmx768m

mod-source-record-manager: CPU:1024/512, mem:2048Mi/1844Mi, -XX:MaxMetaspaceSize=800m -Xmx1024m

mod-source-record-storage: CPU:1024/512, mem:1536Mi/1440Mi, -XX:MaxMetaspaceSize=512m -Xmx1024m

mod-inventory: CPU:1024/512, mem:2880Mi/2592Mi, -XX:MaxMetaspaceSize=512m -Xmx1814m

mod-inventory-storage: CPU:1024/512, mem:2208Mi/1952Mi, -XX:MaxMetaspaceSize=512m -Xmx1440m

mod-orders: CPU:1024/512, mem:2048Mi/1440Mi, db_maxpoolsize: 5, -XX:MaxMetaspaceSize=256m -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/dump/mod-orders.hprof

mod-orders-storage: CPU:512/256, mem:1024Mi/896Mi, db_maxpoolsize: 5, -XX:MaxMetaspaceSize=128m -Xmx700m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/dump/mod-orders-storage.hprof

mod-finance: CPU:256/128, mem:1024Mi/896Mi, db_maxpoolsize: 5, -XX:MaxMetaspaceSize=128m -Xmx700m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/dump/mod-finance.hprof

mod-finance-storage: CPU:128/128, mem:1024Mi/896Mi, db_maxpoolsize: 5,  -XX:MaxMetaspaceSize=128m -Xmx700m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/dump/mod-finance-storage.hprof    

mod-configuration: CPU:128/128, mem:1024Mi/896Mi, -XX:MaxMetaspaceSize=128m -Xmx768m


3 poLines per order (1 item)

500 rec

5k rec10k rec

Profile:

pending orders only

"startedDate" : "2023-03-07T07:28:26.132+00:00",

"completedDate" : "2023-03-07T07:33:02.215+00:00",

5min

"startedDate" : "2023-03-13T13:26:11.088+00:00",

"completedDate" : "2023-03-13T14:01:28.452+00:00",

35min

"startedDate" : "2023-03-13T15:05:33.637+00:00",

 "completedDate" : "2023-03-13T16:13:50.101+00:00",

1h 11min

Profile:

open orders &

create instances

"startedDate" : "2023-03-07T17:50:50.265+00:00",

"completedDate" : "2023-03-07T17:56:21.468+00:00",

6min

"startedDate" : "2023-03-07T19:05:22.933+00:00",

"completedDate" : "2023-03-07T19:46:37.662+00:00",

41min (5 errors: NoSuchElementException)

"startedDate" : "2023-03-07T19:56:50.779+00:00",

"completedDate" : "2023-03-07T21:18:07.797+00:00",

1h 49min (12 errors: NoSuchElementException)

Profile:

open orders &

create instances, holdings, items

 "startedDate" : "2023-03-10T10:42:41.518+00:00",

"completedDate" : "2023-03-10T10:47:38.666+00:00",

6min*

"startedDate" : "2023-03-13T17:01:53.549+00:00",

"completedDate" : "2023-03-13T17:45:38.502+00:00",

44min

"startedDate" : "2023-03-10T11:54:02.095+00:00",

"completedDate" : "2023-03-10T13:17:55.613+00:00"

2h*

3 poLines per order (2 items)


Profile:

pending orders only

"startedDate" : "2023-03-15T18:49:46.897+00:00",

"completedDate" : "2023-03-15T18:52:34.898+00:00",

3 min

"startedDate" : "2023-03-15T18:53:40.506+00:00",

"completedDate" : "2023-03-15T19:13:41.364+00:00",

20 min


Profile:

open orders &

create instances

"startedDate" : "2023-03-15T19:15:11.179+00:00",

"completedDate" : "2023-03-15T19:18:07.061+00:00",

3 min

"startedDate" : "2023-03-15T19:24:15.628+00:00",

 "completedDate" : "2023-03-15T19:47:35.427+00:00",

23 min


Profile:

open orders &

create instances, holdings, items

"startedDate" : "2023-03-16T03:37:33.745+00:00",

"completedDate" : "2023-03-16T03:40:47.818+00:00",

3 min*

"startedDate" : "2023-03-16T03:41:57.577+00:00",

"completedDate" : "2023-03-16T04:09:48.357+00:00",

28 min



(*) mark: the 500 records import was finished with 35 errors and 10k records import was imported with 700 errors. All errors look like this:

HttpException: {"message":"ISBN value is invalid","code":"invalidISBN","parameters":[{"key":"isbn","value":"9719429267 (pbk.)"}]}

HttpException: {"message":"ISBN value is invalid","code":"invalidISBN","parameters":[{"key":"isbn","value":"9781405734240 (v.1)"}]}

HttpException: {"message":"ISBN value is invalid","code":"invalidISBN","parameters":[{"key":"isbn","value":"0585098646 (electronic bk.)"}]}



  MODORDERS-864 - Getting issue details... STATUS Task for fix NoSuchElementException


mod-orders 12h:mod-orders-storage 12h:




  • No labels