Add new parameters to support flow control
RCA Group
None
Description
Environment
None
Potential Workaround
None
defines
has to be done before
relates to
Checklist
hideTestRail: Results
Activity
Show:
Serhii_Nosko April 15, 2022 at 12:01 PM
Parameters were added in scope of PR https://github.com/folio-org/mod-source-record-manager/pull/620,
This PR would be merged after closing of the the main https://folio-org.atlassian.net/browse/MODSOURMAN-662 story for flow control implementation.
This particular story could be closed
Done
Details
Details
Assignee
Unassigned
UnassignedReporter
Aliaksandr FedasiukPriority
P2Story Points
1
Sprint
None
Development Team
Folijet
Fix versions
Release
Morning Glory (R2 2022)
TestRail: Cases
Open TestRail: Cases
TestRail: Runs
Open TestRail: Runs
Created December 28, 2021 at 1:06 PM
Updated July 5, 2022 at 1:25 PM
Resolved April 15, 2022 at 12:13 PM
TestRail: Cases
TestRail: Runs
Introduce two configuration parameters (names are to be altered):
di.flow.max_simultaneous_records – this parameter defines how many records can be processed by the system simultaneously. Default value is proposed to be set to 10*(kafka batch size)
di.flow.records_threshold – this parameter defines how many records from previous batch must be processed before throwing new records to pipeline. Default value is proposed to be 5*(kafka batch size)
Since Kafka works with batches, those parameters should be set to be evenly divided by Kafka batch size. a validation could also be added for this rule