Skip to end of banner
Go to start of banner

Data Import Create MARC holdings records

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

UIDATIMP-1034 - Getting issue details... STATUS



Overview 

This document contains the results of testing Data Import Create MARC holdings records in pre-lotus to detect performance trends.

Infrastructure

  • 6 m5.xlarge EC2 instances 
  • 2 instances of db.r6.xlarge database instances, one reader and one writer
  • MSK
    • 4 m5.2xlarge brokers in 2 zones
    • auto.create-topics.enable = true
    • log.retention.minutes=120
  • mod-inventory memory
    • 256 CPU units, 1814MB mem
    • inventory.kafka.DataImportConsumerVerticle.instancesNumber=10
    • inventory.kafka.MarcBibInstanceHridSetConsumerVerticle.instancesNumber=10
    • kafka.consumer.max.poll.records=10
  • mod-inventory-storage
    • 128 CPU units, 544MB mem
  • mod-source-record-storage
    • 128 CPU units, 908MB mem
  • mod-source-record-manager
    • 128 CPU units, 1292MB mem
  • mod-data-import
    • 128 CPU units, 1024MB mem


Software versions

All of this snapshot versions were taken in the week of January 10

  • mod-data-import:2.3.0-SNAPSHOT.78
  • mod-data-import-converter-storage:1.13.0-SNAPSHOT.69
  • mod-source-record-storage:5.3.0-SNAPSHOT.132
  • mod-source-record-manager:3.3.0-SNAPSHOT.115
  • mod-inventory:18.1.0-SNAPSHOT.344
  • mod-inventory-storage:23.0.0-SNAPSHOT.573
  • mod-search:1.6.0-SNAPSHOT.452
  • mod-quick-marc:2.3.0-SNAPSHOT.47


Results




User Story Acceptance Criteria :

  • What is the recommended maximum file size a user can upload to create MARC holdings records and also FOLIO holdings records?
    • so far we have only 5K file for creation MARC holdings records.
  • How many MARC holdings records and also FOLIO holdings records can be created in 30 minutes?
    • Theoretically it's 50K
  • How many MARC holdings records and also FOLIO holdings records can be created in an hour?
    • Theoretically it's 100K
  • How long will it take to create 1,000 MARC holdings records and also FOLIO holdings records via data import?
    • 25 sec.
  • How long will it take to create 5,000 MARC holdings records and also FOLIO holdings records via data import?
    • 2 min 20 sec


  • No labels