add option to store source marc as json
Description
Environment
Potential Workaround
blocks
Checklist
hideTestRail: Results
Activity

Alfons SeelenJuly 10, 2018 at 8:42 AM
Done, but hasn't been tested yet to `mod-inventory-storage`. Might be that query has to be adapted

Alfons SeelenJune 21, 2018 at 2:16 PMEdited
Don't forget to put a reference to the source record in the instance!
Or at least a corresponding ID for both (e.g. UUID).

Alfons SeelenJune 20, 2018 at 11:40 AM
Currently, mod-data-loader, when receiving a POST request on /load/marc-data
, it will...
create a POST request that it submits to a storage URL (required param, see API) at /admin/importSQL
.
This request consist of an SQL query with all (limited to the bulk size, if larger, multiple requests) records.
What I will probably do is add a second similar outgoing POST request for every current, containing the source strings (JSON) of the records. A corresponding table should be created in mod-inventory-storage
.
Also, the API (POST /load/marc-data
) of the mod-data-loader
should be extended with a boolean that tells whether the source records should be stored.
As discussed in the back-end core team meeting, in the future, we might want to change the way the data loader interacts with the/a storage module, but not now.
Aside from this functionality I will probably add some small unit tests that check whether the SQL strings are created correctly. I will not continue to use the "test" API, as all current unit tests are implemented, but implement tests through the isTest==false
path, to be able to test the whole code chain within the 'unit'. Forget this last remark if that might be confusing. It's the least important.
MODINV-44 will create a way to store the original MARC input in a separate table (outside of the `Instance`) and will let the inventory module offer a GET API to get this original input file as JSON.
Now, as the data-importer (mod-data-loader) is responsible for handling MARC input files, the option the store the file as JSON should be implemented here, as well as the conversion from MARC binary to MARC JSON. It might be helpful to check whether marc4j offers the latter functionality.