IN PROGRESS
...
Test 1
Records transferred . - 4770043 (should be 8415303)
Records harvested - 20618 X 100 = 2 061 800.
We can see here unstable part of test. This spikes on chart showing extremely increased response times. which leads to throughput gaps. At this point we still not sure this does it happening we've checked:
- RDS response times (logs);
- mod-oai-pmh (logs);
- nginx-oai-pmh (logs);
- edge-oai-pmh (logs);
- okapi (logs);
At each point we have good response times and we can't see correlation between logs and this chart.
Possible issues:
- AWS bandwidth limitations (on target environment side and/or on load generator side);
- Issue with load generator (not enough resources/ limitations/java heap).
Service CPU usage has reached ±200% while data transfer. And it's on 50-60% level during data harvesting. However during "unstable part" of test it has drop down to 20-25%.
...
- While data transferring process is going on the background DB CPU usage has reached 70%-75%.
- Data transferring process has failed in 10 minutes and transfer only 4770043 from 8M records.
- Harvesting itself consumes 15% DB CPU.
Test 2
Records transferred . - 4770043 (should be 8415303)
Records harvested - 22305 X 100 = 2 230 500.
