KIWI | KIWI (with OL) | KIWI with partitions N# 2 | ||
|---|---|---|---|---|
5K MARC Create | PTF - Create 2 | 5 min, 8 min | 8 min | 5 min |
5K MARC Update | PTF - Updates Success - 1 | 11 min, 13 min | 6 min | 7 min |
10K MARC Create | PTF - Create 2 | 11 min , 14 min | 12 min | 10 min |
10K MARC Update | PTF - Updates Success - 1 | 22 min, 24 min | 15 min | 11 min |
| 25K MARC Create | PTF - Create 2 | 23 mins, 25 mins, 26 mins | 24 min | 23 min |
| 25K MARC Update | PTF - Updates Success - 1 | 1 hour 20 mins (completed with errors) *, 56 mins | 40 min | failed |
| 50K MARC Create | PTF - Create 2 | Completed with errors, 1 hr 40 mins | 43 min | - |
| 50K Update | PTF - Updates Success - 1 | 2 hr 32 mins (job stuck at 76% completion) | 1hr 4min | - |
number of partitions - 2
memory usage
org.apache.kafka.common.errors.RebalanceInProgressException: Offset commit cannot be completed since the consumer is undergoing a rebalance for auto partition assignment. You can try completing the rebalance by calling poll() and then retry the operation.
...