...
There’s currently a GitHub issue https://github.com/spring-projects/spring-batch/issues/4133 connected to simultaneous running of multiple jobs. Issue reproduces, confirmed with POC. Probably we’ll not be affected if one app instance runs only one job.
If chunks are submitted to kafka - parallel chunks processing in a scope of one instance would require concurrent consuming.
“manager“/”worker” are supposed to be separate app instances, f.e. using profiles for configuration. Probably will be ok to have 1 manager + one worker for each app instance. TODO: check/test how to route responses in such case, or if it’s possible to consume on manager different from one which created the job.
In case of one app instance remote chunking will most likely be slower than current solution, so probably some profile/env variable should be present to enable remote chunking only in case there’re multiple app instances, otherwise - use currently implemented approach.
...