IN PROGRESS

The purpose of these set of tests is to measure performance of Kiwi release. Find possible issues, bottlenecks.

Conclusions

Unstable parts of first tests was made by data set:
- instances didn't have underlying records and this causes multiple repeating calls from mod-edge-oai-pmh to mod-oai-pmh.
- this leads to end client to wait until oai-pmh will find records with underlying records. And often client fail with 504 getaway timeout (load balancer timeout 400 seconds).
Timeouts in DB fixed with changing of dataset to "bugfest" like.

Test flow

Test consist of few calls:

...

Test 1

Records transferred . - 4770043 (should be 8415303)

Records harvested - 20618 X 100 = 2 061 800.

We can see here unstable part of test. This spikes on chart showing extremely increased response times. which leads to throughput gaps. At this point we still not sure this does it happening we've checked:

...

While data transferring process is going on the background DB CPU usage has reached 70%-75%.
Data transferring process has failed in 10 minutes and transfer only 4770043 from 8M records.
Harvesting itself consumes 15% DB CPU.