...

• How long does it take to export 10K records?
• What happens to the job that is running, will it be able to resume and complete successfully when the new task is spun up? 
• Look for a memory trend and use it to decide on the number of concurrent jobs needed to reach the tipping point.  

Infrastructure

• 10 m6i.2xlarge EC2 instances  (changed. In Lotus it was m5.xlarge)
• 2 instances of db.r6.xlarge database instances, one reader and one writer
• MSK. (topics being used [cluster].[tenant].data-export.job.command ,  [cluster].[tenant].data-export.job.update)
  • 4 m5.2xlarge brokers in 2 zones
  • auto.create-topics.enable = true
  • log.retention.minutes=120
  • 2 partitions per DI topics

...

Below charts represents that system is working stable.

For the endurance testing scheduler for this particular test scheduler inside organisation integration was set to occur hourly over night.and left overnight, export performed at least 19 times. 

As you can see there is no growing of memory usage 

...