Purpose

The purpose of this document is to enumerate steps that should occur that will allow import & export job profiles between FOLIO tenants. Most changes will occur in mod-data-import-converter-storage(mod-data-import-cs). MODDATAIMP-577 - Getting issue details... STATUS

Approach

1. Mapping Field Origin Definition

Fields in a mapping profile can be constructed via the FOLIO UI. Some level of validation exists in the FOLIO UI but much less on Data Import APIs. The FOLIO settings UI is responsible for enumerating valid values that a field can contain. This list of "accepted value" is saved into mod-data-import-converter-storage as is. Incorrect accepted values can be saved in mod-data-import-cs. mod-data-import-cs needs to be augmented to apply validation steps similar to the FOLIO UI. As it stands, critical business logic lives only in the FOLIO UI when it should be present in the FOLIO module.

A representation of a mapping field should exist in mod-data-import-cs. This representation should also have a link to its origin; FOLIO domain areas like Inventory, Orders, Invoices. With the mapping field origin present, this will allow mod-data-import-cs to be able to do the same validation that the FOLIO UI performs.

2. Profile Validation

Profile validation should start occurring on mod-data-import-cs. Upon creation or update of a job profile, validation should occur with data sourced from the mapping field origin. Good error messages should be returned to allow easier troubleshooting. Adding this to mod-data-import-cs will make it independent from the FOLIO UI.

mod-data-import-cs will validate every Match, Action, Mapping & Job profile that is persisted within. mod-data-import-cs will be aware of the origin of a profile, e.g. action profile for "create instance", belongs to inventory. Inventory will validate the profile and send a response back to mod-data-import-cs. Some preliminary validations should occur at mod-data-import-cs. Here is a sequence diagram of the example

mod-inventory can be replaced with other FOLIO domain areas for validation of components of a Job Profile. mod-data-import-cs will determine the destination of a Profile object.

The endpoint should be similar to /inventory/data-import-profiles/validate. The endpoint will accept an array of Profiles and return an array of results detailed a valid Profile or error messages signaling reason why a profile is not valid. Order of the results could match the corresponding input OR profile identifiers from the input is referenced in the validation results.. This means that mod-data-import-cs will delegate validation to corresponding modules that own a FOLIO domain. New endpoints will be implemented in those modules that will accept profiles for validation.

3. Profile Transformation

Having mapping field origins defined and validation occurring in mod-data-import-cs, it will set the stage to allow imports of job profiles from other tenants. Transformations would occur by comparing incoming reference data with existing reference data. This will usually be by name or code. If no match can be found then an error is returned. Job profiles returned by mod-data-import-cs API should be accepted as input for the transformation without any modification.

A GET can be made to /data-import-profiles/jobProfiles with a withRelations query parameter to export a job profile.

A POST can be made to /data-import-profiles/jobProfiles with a transform query parameter to import a job profile

Import

POST /data-import-profiles/jobProfiles?transform=true HTTP/1.1
Host: folio-snapshot-okapi.dev.folio.org
x-okapi-tenant: diku
x-okapi-token: {{token}}
Content-Type: application/json

{
	"id": "e34d7b92-9b83-11eb-a8b3-0242ac130003",
    "name": "Default - Create instance and SRS MARC Bib",
    "description": "This job profile creates SRS MARC Bib records and corresponding Inventory Instances using the library's default MARC-to-Instance mapping. It can be edited, duplicated, or deleted.",
    "dataType": "MARC",
    "deleted": false,
    "userInfo": {
          "firstName": "System",
          "lastName": "System",
          "userName": "System"
    },
    "parentProfiles": [],
    "childProfiles": [### Match, Action profiles etc ###],
    "hidden": false,
    "metadata": {
        "createdDate": "2021-04-13T14:00:00.000+00:00",
        "createdByUserId": "00000000-0000-0000-0000-000000000000",
        "updatedDate": "2021-04-13T15:00:00.462+00:00",
        "updatedByUserId": "00000000-0000-0000-0000-000000000000"
   }
}

During transformation, mod-data-import-cs will convert incoming reference data identifiers/values into reference data identifiers/values that exists in the tenant. If a suitable value cannot be found, details errors will be returned to the API client. Errors should include valid values that the system will accept. Every detail will be validated before errors are returned, the process should not fail on the first exception.

Details like "userInfo" and "metadata" will be translated/updated to the user context that is making the API call.

There is an option to have mod-data-import-cs create the missing reference data. If it is a trivial create, this will be very doable, otherwise errors should be returned to the user.

Deployment

Import/export can be enabled on certain job profile types on a case by case basis.For example, starting with MARC BIB profiles and then working on EDIFACT next. Additionally, introduction of mapping field origins means that mod-data-import-cs will have more module dependencies. These new dependencies should be optional and mod-data-import-cs should fail gracefully when reference data of a module, that is not enabled on the tenant, has been requested.

Browser not supported