Migration Tools (UXPROD-850)

[UXPROD-1500] Create a command line interface to use for initial load of data into the system Created: 28/Jan/19  Updated: 03/May/21  Resolved: 29/Apr/21

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: None
Parent: Migration Tools

Type: New Feature Priority: P3
Reporter: patty.wanninger Assignee: Ian Walls
Resolution: Done Votes: 0
Labels: round_iv
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Epic Link: Migration Tools
Front End Estimate: Small < 3 days
Front End Estimator: Hkaplanian
Front-End Confidence factor: Low
Back End Estimate: Large < 10 days
Back End Estimator: Hkaplanian
Kiwi Planning Points (DO NOT CHANGE): 7
PO Rank: 73
PO Ranking Note: As written, it looks like this need is already met by existing, external tools like curl. This would be particularly redundant with bulk APIs being able to receive multiple JSON objects in a single POST
Rank: Chalmers (Impl Aut 2019): R5
Rank: Chicago (MVP Sum 2020): R5
Rank: Cornell (Full Sum 2021): R5
Rank: Duke (Full Sum 2021): R5
Rank: 5Colleges (Full Jul 2021): R1
Rank: FLO (MVP Sum 2020): R1
Rank: GBV (MVP Sum 2020): R2
Rank: hbz (TBD): R1
Rank: Hungary (MVP End 2020): R1
Rank: Lehigh (MVP Summer 2020): R2
Rank: Leipzig (Full TBD): R1
Rank: Leipzig (ERM Aut 2019): R1
Rank: TAMU (MVP Jan 2021): R2
Rank: U of AL (MVP Oct 2020): R5

 Description   

Loader should be a command-line app that can post file-based json data to all modules to which data will be migrated using the module APIs, and, optionally, for inventory, a file of marcxml for processing of bib data.



 Comments   
Comment by Dale Arntson [ 28/Jan/19 ]

I don't think we need the last sentence. Scheduling can happen with crontab, and job interuption, failure is handled by DataLoaderRequirements # 6 ( UXPROD-1504 Closed ).

Comment by patty.wanninger [ 29/Jan/19 ]

Final sentence removed per Dale's comment.

Comment by Anya [ 29/Mar/19 ]

Comment from March meeting: Should exclude MARC and anything loading into SRS

Comment by Ian Walls [ 29/Jul/19 ]

If we're excluding MARC data from the scope of this issue, then it seems to me that there is already a host of pre-existing tools we can use to POST JSON to an API endpoint (such as curl or Postman).

Comment by Ian Walls [ 30/Jul/19 ]

Apologies; misunderstood the order of PO ranks. Re-ranking

Comment by Holly Mistlebauer [ 11/Sep/19 ]

Ian Walls and patty.wanninger: Two institutions are asking for this feature to be added to the MVP, but the Cap Plan Team believes this is being done currently with curll--this is being demo'd by TAMU and others Please close or update this issue appropriately. Thanks much!

Comment by Theodor Tolstoy (One-Group.se) [ 14/Dec/20 ]

We have recently share a tool that can load JSON data to various APIs both using a GUI and a CLI: https://github.com/FOLIO-FSE/service_tools. Take a look and see if this is enough. It is very easy to extend. Error handling of failed batches is a bit sparse.
The relevant functionality is located in https://github.com/FOLIO-FSE/service_tools/blob/develop/service_tasks/batch_poster.py

Comment by Julian Ladisch [ 29/Apr/21 ]

Closing because curl, wget, Postman and https://github.com/FOLIO-FSE/service_tools exist.

https://github.com/FOLIO-FSE/service_tools is listed on https://folio-org.atlassian.net/wiki/display/SYSOPS/Documentation

Generated at Fri Feb 09 00:16:03 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.