Versions Compared

Old Version 1

changes.mady.by.user Martin Tran

Saved on

compared with

New Version 2

changes.mady.by.user Varun Javalkar

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Currently one CREATE import job of 50K records can run reliably, takes around 2 hours. Multiple CREATE jobs may be run but depending on the timing of when the second (or n-th job) starts, the first job is slowed down and the second or n-th job takes a very long time to get started and finishes. This is because of the overwhelming number of messages that are queued up in the Kafka topics that the first job created.  
    • Consequently running concurrently DI jobs in a multi-tenants cluster is almost impossible when the number of records per each job are tens of thousands. 
    • To date 100K CREATE imports have not been done successfully, either more or less records than anticipated have been created.
    • Current hardware's resources cannot accommodate both DI and circulation load successfully.

Functionality

  • It is very time-consuming and requires a lot of manual effort to clean up old Kafka topics after upgrading DI modules to the new FOLIO release. Migration script to clean up old topics will greatly enhance upgrades.