Table of Contents |
---|
...
- Kiwi release was able ho harvest 7,808,200 records in 19 hr 8 min (1M records per 2 hours and 15 min).
- Average response time per request with resumption token 0.874s.
- No memory or CPU issues were found (after the first couple of JIRAs below had been fixed)
- KPIs:
- mod-oai-pmh CPU usage 120% (on data transferring) 100% on harvesting.
- RDS CPU usage 80% on data transferring and ±15 % on harvesting
- Memory usage 105-107% on mod-source-record-manager. 35% on mod-oai-pmh. No signs of memory leaks on related modules.
- A few issues were found
- OutOfMemory exception:
Jira Legacy server System Jira serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key MODOAIPMH-374 - Thread block issue:
Jira Legacy server System Jira serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key MODOAIPMH-374 - When instances didn't have underlying MARC records, multiple repeating calls from mod-edge-oai-pmh to mod-oai-pmh were occurred, resulting in the end-client receiving an timeout, see
Jira Legacy server System Jira serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key MODOAIPMH-383
- OutOfMemory exception:
...
- Total Underlying SRS records: 1,212,039
- Duration: 4 hr 57 min
- Records transferred: 4,770,043 (should be 8,415,303)
- Records harvested - 20,618 X 100 = 2,061,800.
...
- While data transferring process is going on the background DB CPU usage has reached 70%-75%.
- Data transferring process has failed in 10 minutes and transfer only 4770043 from 8M records.
- Harvesting itself consumes 15% DB CPU.
Test 2
- Total Underlying SRS records: 1,212,039
- Duration: 4 hr 25 min
- Records transferred: 3,815,867 (should be 8,415,303)
- Records harvested - 22305 X 100 = 2,230,500.
...