Support for migration of open order data into FOLIO
Problem statement
Currently, order migration is a highly manual process. ICs do have some tools available, but when it comes to handling orders, the process often relies on direct access to the database. Additionally, the tools currently in use don't support the use of templates, which further limits efficiency.
Accordingly, there is a need for a tool for ICs that would automate the described process of migrating orders created in other systems into FOLIO. This tool should provide a user-friendly mechanism for configuring templates and field mappings to streamline the migration process.
Related information
Requirements and Expectations
This is not an ongoing runtime migration or integration but rather a one-time migration. Once the migration is completed (e.g., from Sierra or other systems), no further migration will be required. “Once it's done, it's done”.
The input data can only be provided in the form of a file, in one of the specified formats: JSON, CSV, or TSV.
It is expected that the number of records in a single file for migration will be low thousands, up to 10K or 15K.
There are no specific performance requirements defined (common sense applies).
No need for dry-run functionality - ICs would try a small sample to pre-test.
After the migration is complete, a report with statistics and errors must be generated and provided.
Migration templates are configured in Settings -> Orders -> Order templates (the same way as, say, for MOSAIC). Note that there’s no option to export order templates from the FOLIO into a file.
Outstanding questions
How should this tool be launched? For example, from command-line interface (CLI) - just specify the path to the source file, the path to the template file and where to write the result (database)?
Where to locate the template configuration? E.g., somewhere under Settings -> Orders? Note that there’s no option to export the template from the FOLIO into a file.
In the UI, templates are located under Settings → Orders → Order templates
The API will know where to look for templates internally
Comments
@Raman Auramau You could probably look into the approach we implemented for marc records migrations. We implemented the mod-marc-migration module, which is deployed as a usual module into the environment when such migrations are needed.
Regarding how tool should be launched, my preference would be that we POST whatever the tools needs from our standalone migration machines and that errors, successes, etc be returned much as they are with the other API endpoints. If the process takes long enough that awaiting a real time response isn’t feasible, returning some kind of job ID that we can use to retrieve the outcome later along the lines of what’s done with Data Import would be ideal.
For template config, Settings--Orders strikes me the natural home, the idea being that our data payload would specify the applicable template which would automatically be applied to the data before preforming additional processing. The goal is supply as little data as possible and have the template and tool perform complex object construction.