add option to store source marc as json

Description

MODINV-44 will create a way to store the original MARC input in a separate table (outside of the `Instance`) and will let the inventory module offer a GET API to get this original input file as JSON.

Now, as the data-importer (mod-data-loader) is responsible for handling MARC input files, the option the store the file as JSON should be implemented here, as well as the conversion from MARC binary to MARC JSON. It might be helpful to check whether marc4j offers the latter functionality.

Environment

None

Potential Workaround

None

Checklist

hide

TestRail: Results

Activity

Show:

Alfons SeelenJuly 10, 2018 at 8:42 AM

Done, but hasn't been tested yet to `mod-inventory-storage`. Might be that query has to be adapted

Alfons SeelenJune 21, 2018 at 2:16 PM
Edited

Don't forget to put a reference to the source record in the instance!
Or at least a corresponding ID for both (e.g. UUID).

Alfons SeelenJune 20, 2018 at 11:40 AM

Currently, mod-data-loader, when receiving a POST request on /load/marc-data, it will...

create a POST request that it submits to a storage URL (required param, see API) at /admin/importSQL.
This request consist of an SQL query with all (limited to the bulk size, if larger, multiple requests) records.

What I will probably do is add a second similar outgoing POST request for every current, containing the source strings (JSON) of the records. A corresponding table should be created in mod-inventory-storage.

Also, the API (POST /load/marc-data) of the mod-data-loader should be extended with a boolean that tells whether the source records should be stored.

As discussed in the back-end core team meeting, in the future, we might want to change the way the data loader interacts with the/a storage module, but not now.

Aside from this functionality I will probably add some small unit tests that check whether the SQL strings are created correctly. I will not continue to use the "test" API, as all current unit tests are implemented, but implement tests through the isTest==false path, to be able to test the whole code chain within the 'unit'. Forget this last remark if that might be confusing. It's the least important.

Done

Details

Assignee

Reporter

Priority

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created May 9, 2018 at 12:54 PM
Updated July 27, 2018 at 7:33 AM
Resolved July 10, 2018 at 8:42 AM
TestRail: Cases
TestRail: Runs