Implement metrics for mod-oai-pmh
Description
Environment
Potential Workaround
Attachments
- 06 Feb 2022, 08:30 PM
Checklist
hideTestRail: Results
Activity
Viachaslau Khandramai March 11, 2022 at 12:23 PMEdited
Cannot be verified on Kiwi bugfest because DEBUG mode cannot be enabled. Previously verified on Firebird Rancher in DEBUG mode: https://folio-org.atlassian.net/browse/MODOAIPMH-377?focusedCommentId=157782&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel
Denis March 1, 2022 at 5:37 PM
Deployed to Kiwi Bugfest. Please proceed with verification
Viachaslau Khandramai February 6, 2022 at 8:30 PM
Verified on https://firebird.ci.folio.org.
1) Enable DEBUG mode for mod-oai-pmh by using pre-configured log4j file: -Dlog4j.configurationFile=/etc/log4j/log4j.properties.
2) Import some MARC records.
3) Make harvesting: GET https://firebird-edge-oai-pmh.ci.folio.org:443/oai?verb=ListRecords&apikey=eyJzIjoiblo1NkYzTGVBYSIsInQiOiJkaWt1IiwidSI6ImRpa3VfYWRtaW4ifQ==&metadataPrefix=marc21_withholdings
4) After 1 minute inspect logs to be sure that metrics are collected:
2022-02-06T20:27:52,812 DEBUG [pool-2-thread-1 ] MetricsCollectingService ----------------- 2022-02-06T20:27:52.8119 -----------------
06/02/2022 23:27:52 2022-02-06T20:27:52,812 DEBUG [pool-2-thread-1 ] MetricsCollectingService INSTANCES_PROCESSING ---> Avg. time: 65.0 ms, min: 65.0 ms, max: 65.0 ms, cnt: 1
06/02/2022 23:27:52 2022-02-06T20:27:52,812 DEBUG [pool-2-thread-1 ] MetricsCollectingService PARSE_XML ---> Avg. time: 2.1 ms, min: 1.0 ms, max: 6.0 ms, cnt: 19
06/02/2022 23:27:52 2022-02-06T20:27:52,812 DEBUG [pool-2-thread-1 ] MetricsCollectingService SRS_RESPONSE ---> Avg. time: 193.0 ms, min: 193.0 ms, max: 193.0 ms, cnt: 1
06/02/2022 23:27:52 2022-02-06T20:27:52,812 DEBUG [pool-2-thread-1 ] MetricsCollectingService INVENTORY_STORAGE_RESPONSE ---> Avg. time: 3.0 ms, min: 3.0 ms, max: 3.0 ms, cnt: 1
06/02/2022 23:27:52 2022-02-06T20:27:52,812 DEBUG [pool-2-thread-1 ] MetricsCollectingService SEND_REQUEST ---> Avg. time: 553.0 ms, min: 553.0 ms, max: 553.0 ms, cnt: 1
Magda Zacharska January 31, 2022 at 9:01 PM
Hi @Viachaslau Khandramai - how can I verify this story?
Details
Details
Assignee
Reporter
Priority
P3
Purpose/Overview:
Custom metrics should be implemented for mod-tai-pmh.
Requirements/Scope:
The metrics should be posted to separate logs. Metrics can be enabled/disabled via logging configuration. It should be done once per minute and only if there is a request during this minute. The needed metrics should cover:
The time, that is taken by the download instances call.
The average time of response from inventory-storage.
The average time of response from SRS.
The average time of marshaling of object to XML response.
The average time of the whole time of processing of one instance (from the instances table)
Approach:
Is described here: https://folio-org.atlassian.net/wiki/display/FOLIJET/%5BSpike%5D+mod-oai-pmh+performance+metrics
Acceptance criteria:
Metrics are implemented
Unit tests coverage is at least 80%
Regression testing:
MMZ: The initial harvest has been run three times in the kiwi-bugfest environment after the hotfix was deployed on March 4th, March 5th and March 6th. Each harvest takes ~16 hours and harvests 8,057,158 records as reported in the bugfest channel. No regression found. Performance and consistency has been improved.
For the smoke test please use tests that are less taxing on the system:
C13784 List records in marc21_withholdings format with start and end date