Table of Contents |
---|
Infrastructure
PTF -environment
- 10 m6i.2xlarge EC2 instances
- 2 instances of db.r6.xlarge database instances, one reader and one writer
- MSK
- 4 m5.2xlarge brokers in 2 zones
- auto.create-topics.enable = true
- log.retention.minutes=120
...
- mod-data-export-worker: 1.4.10
- mod-data-export-spring: 1.4.5
- mod-agreements: 5.2.2
- mod-notes: 3.1.2
- mod-users: 18.3.0
- mod-inventory-storage-24.1.0
- mod-inventory-18.2.2
- mod-orders-12.4.3
- mod-orders-storage-13.3.3
- okapi-4.14.2
Test Results
Test # | Jobs | Orders | Success rate | Time | |
---|---|---|---|---|---|
1 | 10 | 100 | 6/10 | ±10 sec | Failed login to FTP (FtpException) Socket closed (SocketException) |
2 | 10 | 500 | 8/10 | 55-57 sec | |
3 | 10 | 1000 | 7/10 | 3 min | Failed login to FTP (FtpException) Socket closed (SocketException) |
4 | 10 | 1000 | 9/10 | 2 min 17 s | Connection refused (Connection refused) (ConnectException) |
5 | 10 | 1000 | 8/10 | 2 min 48 s | Connection refused (Connection refused) (ConnectException) File upload failed. 450 PASV: data transfer in progress (FtpException) |
6 | 10 | 2000 | 10/10 | ±5 min | |
7 | 10 | 4000 | 10/10 | 9-10 min | |
8 | 10 | 6000 | 8/10 | 16 min | Connection closed without indication. (FTPConnectionClosedException) Connection is not open (IOException) |
...
- At this point we can conclude that more-or less safe limit is 8-10 jobs with 1000 orders per organisation
- Most of jobs failed with Connection refused (Connection refused). However it doesn’t looks like it's the matter of FTP server. We did observe same behavior previously with 3-4 parallel job, and it was fixed with increasing of memory parameters on the module side. At same moment we can't see significant resource usage on FTP server side.
- It looks like percentage of success if highly depended on orders number. The more orders number - the higher chance to complete all jobs successfully.
- we We can conclude that safe limit for EDIFACT is 10 jobs with 1000 - 4000 orders per job.
- Most of a test was dun subsequently and no memory leak was found.
Resources usage
test 1
test 2
...