OAI-PMH Support (UXPROD-993)

[MODOAIPMH-73] Default-loaded SRS records won't show up in mod-oai-pmh Created: 08/Jul/19  Updated: 03/Jan/20  Resolved: 11/Jul/19

Status: Closed
Project: mod-oai-pmh
Components: None
Affects versions: 1.0.2
Fix versions: 1.1.0
Parent: OAI-PMH Support

Type: Bug Priority: P2
Reporter: Theodor Tolstoy (One-Group.se) Assignee: Craig McNally
Resolution: Done Votes: 0
Labels: back-end
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: PNG File screenshot-1.png     File srs_recs.7z    
Sprint: ACQ Sprint 67
Story Points: 2
Development Team: Thunderjet
Tester Assignee: Craig McNally
Epic Link: OAI-PMH Support

 Description   

Overview:
Connection between mod-oai-pmh and SRS seem broken.

Steps to Reproduce:

  1. Invoke the loading of test records into SRS (https://s3.amazonaws.com/foliodocs/api/mod-source-record-storage/p/source-record-storage.html#source_storage_populate_test_marc_records_post)
  2. make a HTTP GET call to {OKAPI_URL}

    oai/records?metadataPrefix=marc21&set=all.

Expected Results:
You get a MARC_XML formatted OAI-PMH response back.

Actual Results:
A HTTP 500 with the response "Sorry, we can't process your request. Please contact administrator(s)."

Additional Information:
I have tried the same thing in Chalmers tenant with other records. Same result. These records are attached to this Bug report.

Log from mod-oai

08 Jul 2019 17:17:08:882 ERROR AbstractGetRecordsHelper [2150141365eqId] Sorry, we can't process your request. Please contact administrator(s).
java.util.concurrent.CompletionException: java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to java.lang.CharSequence
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_151]
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_151]
at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943) ~[?:1.8.0_151]
at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[?:1.8.0_151]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_151]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_151]
at me.escoffier.vertx.completablefuture.VertxCompletableFuture.lambda$supplyBlockingAsync$11(VertxCompletableFuture.java:334) ~[ms.jar:?]
at io.vertx.core.impl.FutureImpl.setHandler(FutureImpl.java:79) ~[ms.jar:?]
at io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:289) ~[ms.jar:?]
at io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:339) ~[ms.jar:?]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [ms.jar:?]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) [ms.jar:?]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463) [ms.jar:?]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886) [ms.jar:?]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [ms.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to java.lang.CharSequence
at io.vertx.core.json.JsonObject.getString(JsonObject.java:111) ~[ms.jar:?]
at org.folio.oaipmh.helpers.storage.SourceRecordStorageHelper.getInstanceRecordSource(SourceRecordStorageHelper.java:33) ~[ms.jar:?]
at org.folio.oaipmh.helpers.AbstractGetRecordsHelper.buildRecords(AbstractGetRecordsHelper.java:167) ~[ms.jar:?]
at org.folio.oaipmh.helpers.AbstractGetRecordsHelper.lambda$buildRecordsResponse$5(AbstractGetRecordsHelper.java:132) ~[ms.jar:?]
at me.escoffier.vertx.completablefuture.VertxCompletableFuture.lambda$supplyBlockingAsync$10(VertxCompletableFuture.java:327) ~[ms.jar:?]
at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:273) ~[ms.jar:?]
at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76) ~[ms.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_151]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_151]
... 2 more
08 Jul 2019 17:17:08:884 INFO LogUtil [2150141367eqId] org.folio.rest.RestVerticle start invoking getOaiRecords
08 Jul 2019 17:17:08:905 INFO LogUtil [2150141388eqId] 127.0.0.1:49284 GET /oai/records metadataPrefix=marc21 HTTP_1_1 500 70 783 tid=fs00001000 Internal Server Error 


 Comments   
Comment by Craig McNally [ 08/Jul/19 ]

Here's the root cause: https://github.com/folio-org/data-import-raml-storage/commit/6df0e8803456131caa5bbb003dea2baaaed080a1#diff-543ff91985065a4003dbbff5bcca8b45

in short, parsedRecord.content changed from string to object, but mod-oai-pmh's SourceRecordStorageHelper was never updated and still expects to find a string there.

Also impacted: https://github.com/folio-org/mod-oai-pmh/blob/8af90672682fc3ea65052a5c04ca7e697647a64f/src/main/java/org/folio/oaipmh/helpers/AbstractGetRecordsHelper.java#L200

Comment by Craig McNally [ 08/Jul/19 ]

Making this a P2 since a fix is needed for Chalmers go-live

Comment by Ann-Marie Breaux (Inactive) [ 08/Jul/19 ]

Hi Craig McNally I'm sorry about that. Folijet introduced the breaking change. Is there a dependency that Folijet missed that we should document someplace?

cc: Oleksii Kuzminov and Kateryna Senchenko

Theo mentioned this at the Data Migration mtg this morning, but Folijet was never involved in the OAI-PMH work, so may not have realized this was a breaking change.

Comment by Piotr Kalashuk [ 11/Jul/19 ]

The changes have been verified on folio-testing environment via API Tests PR#250.
The base URL to access edge modules is https://folio-testing.aws.indexdata.com:8000 (please refer to https://dev.folio.org/guides/automation/) so postman environment variables needs to be updated accordingly.

Comment by Ann-Marie Breaux (Inactive) [ 11/Jul/19 ]

Thank you Piotr Kalashuk. Once all is sorted, we should also alert Theodor Tolstoy (One-Group.se) so that he can check it in the Chalmers environment

Comment by Craig McNally [ 11/Jul/19 ]

note to other testers - working on verifying this.

Comment by Craig McNally [ 11/Jul/19 ]

Verified this on folio-testing:

 curl "$OKAPI/oai/records?metadataPrefix=marc21&set=all" -H "X-Okapi-Token: $TOKEN"
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><OAI-PMH xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd" xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:marc="http://www.loc.gov/MARC21/slim" xmlns:oai-identifier="http://www.openarchives.org/OAI/2.0/oai-identifier" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><responseDate>2019-07-11T17:04:46Z</responseDate><request set="all" metadataPrefix="marc21" verb="ListRecords">http://folio-api-tests.org/oai</request><ListRecords><record><header><identifier>oai:folio-api-tests.org:diku/6b4ae089-e1ee-431f-af83-e1133f8e3da0</identifier><datestamp>1970-01-01T00:00:00Z</datestamp><setSpec>all</setSpec></header><metadata><marc:record><marc:leader>01922nma a2200325 c 4500</marc:leader><marc:controlfield tag="001">1011273942 ...elided...
Comment by Craig McNally [ 11/Jul/19 ]

Closing this since it was verified against folio-testing. Once the chalmers site is updated we will let Theodor Tolstoy (One-Group.se) know and he can give it a test there

Comment by Craig McNally [ 12/Jul/19 ]

FYI this is closed but I haven't had a chance to test and merge the API test PR yet... Got distracted but will do so tomorrow.

Comment by Craig McNally [ 12/Jul/19 ]

API tests pass against folio-testing and have been merged

Comment by Ann-Marie Breaux (Inactive) [ 12/Jul/19 ]

Thank you so much, Piotr Kalashuk and Craig McNally, for jumping on this. Theodor Tolstoy (One-Group.se) is there a way that you can check this with regards to Chalmers data?

Comment by Theodor Tolstoy (One-Group.se) [ 12/Jul/19 ]

Yes,thank you. I will try to add the data and test in a couple of hours.

Comment by Ann-Marie Breaux (Inactive) [ 12/Jul/19 ]

Just a quick FYI, Theodor Tolstoy (One-Group.se). I think you'll only be interacting with SRS, not Inventory, when you test this - is that right? Currently you can load MARC records to SRS, but the ability to create instances from SRS is broken. We had to accommodate a schema change in Instances, and the SRS fix should go live later tonight or tomorrow in folio-testing.

Comment by Theodor Tolstoy (One-Group.se) [ 14/Jul/19 ]

Yes, you are correct Ann-Marie Breaux I never got around to do it but I hope to have it tested in time for when the us folks wakes up on Monday

Comment by Theodor Tolstoy (One-Group.se) [ 14/Jul/19 ]

I am adding records to SRS now. It looks good

Comment by Theodor Tolstoy (One-Group.se) [ 14/Jul/19 ]

but I will attempt a larger harvest once I have added a couple of thousand records.

Generated at Fri Feb 09 00:19:03 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.