Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

IN PROGRESS

...

The purpose of these set of tests is to measure performance of Kiwi release. Find possible issues, bottlenecks. 


Software versions


  • mod-oai-pmh:3.7.0-SNAPSHOT.188.  (mod-oai-pmh:3.6.1 for final test with bugfest like dataset)
  • edge-oai-pmh:2.4.0
  • mod-source-record-manager:3.2.3
  • mod-source-record-storage:5.2.1
  • mod-inventory-storage:22.0.1
  • okapi:4.9.0


Conclusions

  • Unstable parts of first tests was made by data set:
    • instances didn't have underlying records and this causes multiple repeating calls from mod-edge-oai-pmh to mod-oai-pmh.
    • this leads to end client to wait until oai-pmh will find records with underlying records. And often client fail with 504 getaway timeout (load balancer timeout 400 seconds).
  • Timeouts in DB fixed with changing of dataset to "bugfest" like.
  • Jira ticket created to handle client waits: MODOAIPMH-383

...

[resumptionToken] returning in initial call response and in each harvesting call until there is no more records to harvest. When all data has being harvested - resumptionToken will not return with the response.

Software versions

...


Issues detected during testing

...

Test 1

Records transferred  . - 4770043 (should be 8415303)


Records harvested - 20618 X 100 = 2 061 800.

Underlying records number: 1212039


We can see here unstable part of test. This spikes on chart showing extremely increased response times. which leads to throughput gaps. At this point we still not sure this does it happening we've checked:

...

  • While data transferring process is going on the background DB CPU usage has reached 70%-75%.
  • Data transferring process has failed in 10 minutes and transfer only 4770043 from 8M records.
  • Harvesting itself consumes 15% DB CPU.





Test 2

Records transferred . - 3815867 (should be 8415303)

Records harvested - 22305 X 100 = 2 230 500.

Underlying records number: 1212039














Final test (bugfest data set)

...