Unexpected character error

Description

While harvesting a customer data a following errors occur in the logs and causes the harvest to end prematurely:

ERROR oldingsRequestHelper Error has been occurred at JsonParser while reading data from response. Message: Unexpected character ('s' (code 115)): was expecting a colon to separate field name and value

ERROR oldingsRequestHelper Error has been occurred at JsonParser while reading data from response. Message: Unexpected character ('u' (code 117)): was expecting a colon to separate field name and value

ERROR oldingsRequestHelper Error has been occurred at JsonParser while reading data from response. Message: Still have 47 undecoded bytes, should not call 'feedInput'

While this is obviously a problem with the data, the system should handle it gracefully. The issue seems to be similar to MODOAIPMH-345.

Acceptance criteria

  • Investigate possible option of addressing the issue in RMB or outside

  • Document findings

  • Resolve the issue if possible or create development stories if solution is very involving.

Functional Verification:

Verify that "ERROR oldingsRequestHelper Error has been occurred at JsonParser while reading data from response. " doesn't appear in the logs. - VK

Regression testing:
MMZ: The initial harvest has been run three times in the kiwi-bugfest environment after the hotfix was deployed on March 4th, March 5th and March 6th. Each harvest takes ~16 hours and harvests 8,057,158 records as reported in the bugfest channel. No regression found. Performance and consistency has been improved.

For the smoke test please use tests that are less taxing on the system:
C13784 List records in marc21_withholdings format with start and end date

Environment

None

Potential Workaround

None

Attachments

3

Checklist

hide

TestRail: Results

Activity

Show:

Viachaslau Khandramai March 11, 2022 at 12:35 PM

Error didn't occur in the logs on Kiwi Bugfest:

Denis March 1, 2022 at 5:37 PM

Deployed to Kiwi Bugfest. Please proceed with verification

Aliaksei Harbuz February 21, 2022 at 12:16 PM
Edited

While logs were investigated issues related to error while converting record to xml representation were found, the ticket  was created.  That errors could impact on total numbers of records.

Aliaksei Harbuz February 21, 2022 at 12:11 PM
Edited

Data was saved to the instances table of mod-oai-pmh module with json parser version from the  branch without errors at concorde5c environment :

Harvesting did not produce json errors.

Done

Details

Assignee

Reporter

Priority

Story Points

Sprint

Development Team

Firebird

Fix versions

Release

Kiwi (R3 2021) Hot Fix #2

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs
Created January 26, 2022 at 3:43 PM
Updated March 26, 2022 at 2:09 PM
Resolved February 25, 2022 at 2:46 PM
TestRail: Cases
TestRail: Runs