error from logs
WARN Fetcher [Consumer clientId=consumer-DI_INVENTORY_HOLDING_CREATED.mod-inventory-18.0.4-33, groupId=DI_INVENTORY_HOLDING_CREATED.mod-inventory-18.0.4] Received unknown topic or partition error in fetch for partition kcp1.Default.fs09000000.DI_INVENTORY_HOLDING_CREATED-0
@timestamp1642416090190
KIWI | KIWI (with OL) | KIWI with partitions N# 2 | ||
|---|---|---|---|---|
5K MARC Create | PTF - Create 2 | 5 min, 8 min | 8 min | 5 min |
5K MARC Update | PTF - Updates Success - 1 | 11 min, 13 min | 6 min | 7 min |
10K MARC Create | PTF - Create 2 | 11 min , 14 min | 12 min | 10 min |
10K MARC Update | PTF - Updates Success - 1 | 22 min, 24 min | 15 min | 11 min |
| 25K MARC Create | PTF - Create 2 | 23 mins, 25 mins, 26 mins | 24 min | 23 min |
| 25K MARC Update | PTF - Updates Success - 1 | 1 hour 20 mins (completed with errors) *, 56 mins | 40 min | failed |
| 50K MARC Create | PTF - Create 2 | Completed with errors, 1 hr 40 mins | 43 min | - |
| 50K Update | PTF - Updates Success - 1 | 2 hr 32 mins (job stuck at 76% completion) | 1hr 4min | - |
number of partitions - 2
memory usage
org.apache.kafka.common.errors.RebalanceInProgressException: Offset commit cannot be completed since the consumer is undergoing a rebalance for auto partition assignment. You can try completing the rebalance by calling poll() and then retry the operation.
@ingestionTime
1642509297159
@log
054267740449:kcp1-folio-eis
@logStream
kcp1/mod-inventory/cb97f30af8b74ce88e5faf0895bb9650
@message
12:34:54 [] [] [] [] ERROR KafkaConsumerWrapper Consumer - id: 51 subscriptionPattern: SubscriptionDefinition(eventType=DI_SRS_MARC_BIB_RECORD_MATCHED_READY_FOR_POST_PROCESSING, subscriptionPattern=kcp1\.Default\.\w{1,}\.DI_SRS_MARC_BIB_RECORD_MATCHED_READY_FOR_POST_PROCESSING) Error while commit offset: 21596
@timestamp
1642509294406
12:34:54 [] [] [] [] ERROR KafkaConsumerWrapper Consumer - id: 51 subscriptionPattern: SubscriptionDefinition(eventType=DI_SRS_MARC_BIB_RECORD_MATCHED_READY_FOR_POST_PROCESSING, subscriptionPattern=kcp1\.Default\.\w{1,}\.DI_SRS_MARC_BIB_RECORD_MATCHED_READY_FOR_POST_PROCESSING) Error while commit offset: 21596
number of partitions - 3
