Versions Compared

Old Version 3

changes.mady.by.user Martin Tran

Saved on

compared with

New Version 4

changes.mady.by.user Martin Tran

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Currently one CREATE import job of 50K records can run reliably, takes around 2 hours. Multiple CREATE jobs may be run but depending on the timing of when the second (or n-th job) starts, the first job is slowed down and the second or n-th job takes a very long time to get started and finishes. This is because of the overwhelming number of messages that are queued up in the Kafka topics that the first job created.  
    • Consequently running concurrently DI jobs in a multi-tenants cluster is almost impossible when the number of records per each job are tens of thousands. 
    • To date 100K CREATE imports have not been done successfully, either more or less records than anticipated have been created.
  • Current hardware's resources cannot accommodate both DI and circulation load successfully.
    • During CREATE imports, CPU utilization % of mod-srm is 600%, mod-srs is 400%, mod-inventory is over 250%. 
    • During UPDATE imports, CPU utilization % of mod-srm is 500%, mod-srs is 400%, mod-inventory for a long duration around 700%.
    • As a result circulation activities' response times are increased by 1/3.
  • Polling mechanism to get statuses on the DI landing page is slow
    • Polling mechanism executes a slow query on the DB side. Query needs to improve.
    • Polling may not be necessary if we move to the push model with websocket. 

Functionality

  • It is very time-consuming and requires a lot of manual effort to clean up old Kafka topics after upgrading DI modules to the new FOLIO release. Migration script to clean up old topics will greatly enhance upgrades.