number of partitions - 2
Profile
Duration 1 partition
KIWI | KIWI (with OL) | KIWI with partitions N# 2 | |
|---|---|---|---|
5K MARC Create | PTF - Create 2 |
5 min, 8 min | 8 min | 5 min |
5K MARC Update | PTF - Updates Success - 1 |
|---|
11 min, 13 min | 6 min | 7 min |
10K MARC |
|---|
Create | PTF - Create 2 |
|---|
11 min , 14 min | 12 min | 10 min | ||
10K MARC Update | PTF - Updates Success - 1 | 22 min, 24 min | 15 min | 11 min |
|---|---|---|---|---|
| 25K MARC Create | PTF - Create 2 |
| 23 mins, 25 mins, 26 mins | 24 min | 23 min | |
| 25K MARC Update | PTF - Updates Success - 1 |
|---|
| 1 hour 20 mins (completed with errors) *, 56 mins | 40 min | failed | |
| 50K MARC Create | PTF - Create 2 |
|---|
| Completed with errors, 1 hr 40 mins | 43 min | - |
| 50K |
|---|
| Update | PTF - Updates Success - 1 |
|---|
| 2 hr 32 mins (job stuck at 76% completion) | 1hr 4min | - |
number of partitions - 2
org.apache.kafka.common.errors.RebalanceInProgressException: Offset commit cannot be completed since the consumer is undergoing a rebalance for auto partition assignment. You can try completing the rebalance by calling poll() and then retry the operation.
@ingestionTime
1642509297159
@log
054267740449:kcp1-folio-eis
@logStream
kcp1/mod-inventory/cb97f30af8b74ce88e5faf0895bb9650
@message
12:34:54 [] [] [] [] ERROR KafkaConsumerWrapper Consumer - id: 51 subscriptionPattern: SubscriptionDefinition(eventType=DI_SRS_MARC_BIB_RECORD_MATCHED_READY_FOR_POST_PROCESSING, subscriptionPattern=kcp1\.Default\.\w{1,}\.DI_SRS_MARC_BIB_RECORD_MATCHED_READY_FOR_POST_PROCESSING) Error while commit offset: 21596
@timestamp
1642509294406
12:34:54 [] [] [] [] ERROR KafkaConsumerWrapper Consumer - id: 51 subscriptionPattern: SubscriptionDefinition(eventType=DI_SRS_MARC_BIB_RECORD_MATCHED_READY_FOR_POST_PROCESSING, subscriptionPattern=kcp1\.Default\.\w{1,}\.DI_SRS_MARC_BIB_RECORD_MATCHED_READY_FOR_POST_PROCESSING) Error while commit offset: 21596