Table of Contents

Overview

...

• Data import create holdings job durations increased significantly in Quesnelia release. 4 times longer with 10k file. Failed to complete with 80k file because it was stopped after 4 hours of test run with only 46 committed jobs (total for the test was 81).
• Top CPU utilization: mod-inventory-b - 16%, nginx-okapi - 5%, mod-source-record-storage-b - 4%, mod-quick-marc-b - 7%. Such low resource utilization from modules side can be explained by DB queries huge average latency during INSERT and UPDATE processes which had lock on the same tuple.
• Top memory consumption: mod-inventory-storage-b - 85%, mod-data-import-b - 52%, mod-source-record-storage-b - 45%, mod-source-record-manager-b - 43%. Growing trend was defined in tests set #1 for mod-inventory-storage-b - 85%
• DI job duration for the same file size grew from test to test if to use the same instance HRID to create holdings
• DI perform faster if to use files with 1 unique instance HRID for every 1000 records. DI duration corresponds to file size with such approach. Memory utilized without growing trend. CPU and RDS utilization increased because there are less locks in DB.

Recommendations & Jiras

• Investigate growing trend for mod-inventory-storage in tests set #1 (using 1 instance HRID to create all Holdings)
• Define high number of Holdings associated with one instance HRID that's still realistic
• Consider limit the request /inventory/items-by-holdings-id with limit. Now limit=0. 

  Jira Legacy
  server System Jira serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key MODINVSTOR-1229

Errors

• error status for 32'd split job during 80k file importing- SNAPSHOT_UPDATE_ERROR

• Log message: 
  ERROR taImportKafkaHandler org.folio.inventory.dataimport.exceptions.CacheLoadingException: Error loading jobProfileSnapshot by id: 'aee287c2-0d40-4e8d-9879-4c1c54bcd819', status code: 503

...

• MSK tenant
  • 2 m5.2xlarge brokers in 2 zones
  • Apache Kafka version 2.8.0
  • EBS storage volume per broker 300 GiB
  • auto.create.topics.enable=true
  • log.retention.minutes=480
  • default.replication.factor=2

...