IN PROGRESS
...
The purpose of these set of tests is to measure performance of Kiwi release. Find possible issues, bottlenecks.
Conclusions
Test flow
Test consist of few calls:
...
2021-12-01T10:02:42,566 ERROR [vert.x-eventloop-thread-0] MarcWithHoldingsRequestHelper Save instance Ids failed: Timeout. |
io.vertx.core.impl.NoStackTraceThrowable: Timeout |
Fifed with changing of data set to "bugfest" like
Test results
Test 1
Records transferred . - 4770043 (should be 8415303)
Records harvested - 20618 X 100 = 2 061 800.
We can see here unstable part of test. This spikes on chart showing extremely increased response times. which leads to throughput gaps. At this point we still not sure this does it happening we've checked:
...
- While data transferring process is going on the background DB CPU usage has reached 70%-75%.
- Data transferring process has failed in 10 minutes and transfer only 4770043 from 8M records.
- Harvesting itself consumes 15% DB CPU.
Test 2
Records transferred . - 3815867 (should be 8415303)
Records harvested - 22305 X 100 = 2 230 500.
Final test (bugfest data set)
Records transferred - 8213392;
Records harvested - 78082 X 100 = 7 808 200
With new data set there is no "unstable parts" in this test
CPU usage is stable and without big spikes. As you can see there is higher CPU usage at the beginning of the test. It's data transferring process between mod-inventory-stirage and mod-oai-pmh.
We can see here that memory usage is stable and none of containers didn't fail.
