...
The existing renewal process is too slow. However, having a batch perform one operation at a time has also proven problematic. The suggested design is to have a batch queue up a defined number of renewals and then perform the operation.
The business logic can perform a pre-process where it checks to see if any of the renewals would fail for business logic reasons. Queue all renewals that pass the business logic test and then when the queue size is reached perform the SQL operation on that queue. This is potentially problematic in that if something fails in the queue the entire queue might be considered a failure. If this is undesired then a queue size of 1 must be used.
This refers to queue as a subset of renewals that have passed the business logic processing. In this way, a queue is used to refer not to a batch but instead to a generic collection.
To restrict the amount of memory consumption there should be a max batch size of 10,000. When more than the maximum batch size is sent to the API endpoints then an HTTP 413 "Payload Too Large" should be immediately returned without processing anything.
...
The "error" represents technical failures, such as an SQL failure.
SVG: circ-573.svg
Example Multi-
...
update SQL
Postgresql can update multiple rows in a single statement, such as:
...