Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Jira Legacy
serverSystem JIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODDATAIMP-567

For any library storing MARC Bibliographic records in Source Record Storage (SRS), FOLIO contains a default MARC-to-Instance map. FOLIO uses that map when creating or updating Inventory Instances with Source = MARC. The map identifies the Instance fields that are controlled by the underlying MARC Bibliographic record, and which fields and subfields of MARC data populate into which Instance fields. Occasionally libraries may adjust their default map, either because of 1) updated default mappings delivered in a new FOLIO release or 2) local fields or mapping decisions that differ from the default map. When a library's default map is updated, those updated mappings will affect any Instances created or updated after the map change. However, some libraries would like to update all of their existing Instances to reflect the updated mappings. This script allows for that to happen. 


Jira Legacy
serverSystem JIRA
serverId

NOTE: Add some info about how the script works, approximate timing, when to run it so as not to interfere with regular library operations

Examples of library-specific mapping changes:

  • Add a mapping for the previous system's legacy bib ID. Usually it's something like a 9xx field (e.g. 907$a) mapping to a special identifier type (defined in Inventory Settings)
  • Include 690 fields in the Instance subjects

01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODDATAIMP-653

Based on testing in Lotus environment, refreshing Inventory Instances takes approximately 1/2 to 1 second per record, depending on the amount of other load on the tenant. Thus, for 1 million Instances, the time to refresh 1 million Instances would be approximately 6-12 days. Developers will continue to review to see if the timing can be decreased.


View file
nameUpdateMarcToInstanceMapping.sh
height250

NOTE!!! This Instance update script may affect performance of other imports, so it needs to be used during non-working hours or during off-peak hours!!!

NOTE!!! This script will only work for Kiwi and later releases.



Before launching Script, you need to download jq library to allow script work:

https://stedolan.github.io/jq/download/


In order to run the script, you need:

  1. Open terminal (Mac OS or UNIX Systems) or Power shell (Windows) 
  2. Go to the folder where the script is located.
  3. Run Script with or without parameters:
    • e.g. for launch without parameters - sh UpdateMarcToInstanceMapping.sh
      Image Added
    • e.g. for launch with parameters - sh UpdateMarcToInstanceMapping.sh url tenant username password limit release
      Where:
      • url (your library okapi url) - can be seen in the app settings → software versions → okapi services → okapi
        Image Added
        Image Added
      • tenant (admin tenant)
      • username (admin user name)
      • password (admin user name password)
      • limit (number of instances to parse where all sources are counted - FOLIO/MARC)
      • release (release version: 0 is for Kiwi, 1 is for Lotus and future releases)

After the script completes its work, it shows how long the script has been running and number of processed instances:

Total count: 1, Total succeeded count: 1, Total error count: 0
0 hours, 0 minutes and 4 seconds elapsed.

Also, as a result you will get 2 files: offset.txt and results.txt

offset file only will contain OFFSET value (the position in the dataset to start from, where all sources are counted - FOLIO/MARC). Script starts with the offset value from file.

results file will contain all processed instance ids with update status in format (if the results are presented):

   instanceId - status (e.g. 708979eb-c732-4dfa-8f51-8176c9dea6bf - ok)



View file
nameUpdate MARC to Instance map.postman_collection.json
height250

NOTE!!! This Instance update Postman collection may affect performance of other imports, so it needs to be used during non-working hours or during off-peak hours!!!

NOTE!!! This Postman collection will only work for Kiwi and later releases.

NOTE!!! Postman collection can handle only about 1500 requests.


Before launch launching Postman collection you need to , set up the environment:

  • click Click on "eye" button and press "Edit/Add" on "Environment" variables
  • Name the environment as you want and put configure these mandatory variables:
    • protocol (http/https)
    ,
    • url (your library okapi url)
    ,
    • - can be seen in the app settings → software versions → okapi services → okapi
      Image Added
      Image Added
    • username (admin user name)
    ,
    • password (admin user name password)
    ,
    • xokapitenant (admin tenant).
  • There are is also an optional parameter "limit" to limit number of instances to parse. Both sources are counted (FOLIO/MARC), but only MARC source types will be processed. Aivar Iusupov If the user sets a limit, does the script start with instance HRID=1 and go until the limit is reached, or can the user specify the start instance HRID or UUID? Ann-Marie Breaux (Deactivated) - No, it will start from UUID (ordering is based only on UUIDs, like in mod_inventory_storage.instance table). And there are also 1 more problem with that: e.g. if we run our script with limit = 10000 and then during the day we import 1000 instances, we can face a problem that between processed entries could be unprocessed UUID because of ordering.
  • After setting up all the environment variables press "Reset all" and then "Save"
    Image RemovedImage Added
  1. Import collection by pressing "Import" button 
    Image RemovedImage Added
  2. Select collection file (Update MARC to Instance map.postman_collection.json) file from folder or drag it into opened open window and then press "Import" button.
    Image RemovedImage Added
  3. Go to the "Collections" tab and select "Update MARC to Instance map"
    Image RemovedImage Added
  4. Then select environment you'd set up
    Image RemovedImage Added
  5. To launch collection press "Run" button below environment selection and in the opened window press "Run Update MARC to Instance map"
    Image Removed Aivar Iusupov Just to confirm - this is updating the Instances, not the MARC-to-Instance map. Is that correct? Ann-Marie Breaux (Deactivated) This will apply all the MARC-to-Instance mapping rules on SRS Record and will update Instance.
    Image Added