Scripts for Inventory, Source Record Storage, and Data Import Cleanup

 The following scripts may be used for identifying and cleaning up records in FOLIO's Source Record Storage and Inventory. More technical details are available via the title hotlink for each script.

1. Retrieving Inventory Holdings Records and Update/Cleanup then if needed
   1. Release: Kiwi or later
   2. Release notes: Kiwi
   3. Use cases:
      1. Inventory Holdings source is anything other than FOLIO or MARC 
      2. If the Inventory Holdings source = MARC, check to see if there is a linked SRS MARC Holdings record
         1. If yes, then keep the Holdings source as MARC, and ensure that View source and Edit via QM in the Actions menu work for the Inventory Holdings record.
         2. If no, then change the Holdings source to FOLIO, and ensure that View source/Edit in QM do not display in the Inventory Holdings Action menu.
   4. Timing: On demand
   5. Related Jira: MODSOURMAN-627: Data integrity issues with instances/holdings records
   6. NOTE: These scripts do not update the Inventory Holdings. They only retrieve holdings that should be reviewed and then possibly subsequently updated
2. Scripts for identifying and fixing invalid instanceIDs in SRS
   1. Release: Lotus or later
   2. Release notes: Lotus
   3. Use case: Inventory instance source = MARC and "View source" is not available but there is MARC Bibliographic record that has id of this instance at 999 ff $i subfield and has not in "records_lb.external_id" column
   4. Timing: On demand
   5. Related Jira: MODSOURCE-300: Create script for fixing invalid SRS instanceIDs
3. Script to refresh Instances against an updated MARC Bib-to-Inventory Instance map
   1. Release: Kiwi or later
   2. Release notes: Lotus
   3. Use case: To refresh some or all Inventory Instances, after the library's default MARC Bib-to Inventory Instance map has been updated
   4. Timing: On demand
   5. Related Jira: MODDATAIMP-567: Script to refresh Instances against an updated MARC Bib-to-Inventory Instance map
4. Script to delete duplicate mapping rules for each record type
   1. Release: Kiwi or later
   2. Release notes: Kiwi
   3. Use case: To clear duplicate mapping_rules records in the db 
   4. Timing: On demand
   5. Related Jira: MODSOURMAN-632: Create a script to delete duplicate mapping rules for each record type
5. Script to identify "bad URLs" (broken links, etc.) in records linking to electronic items
   1. Release: Juniper or later (and maybe earlier... tested on Juniper)
   2. Use cases:
      1. To detect Page Not Found (404) and any other unsuccessful status code
      2. To detect permanent redirects that could be updated to the new URL to improve link stability and loading time
      3. To detect insecure (non-https) links and upgrade them to https if possible
   3. Timing: On demand
6. Script for setting the SRS record "order" field so that Inventory Single Record Imports do not display blank log information
   1. Release: Kiwi or later
   2. Release notes: Kiwi, Lotus
   3. Use case: To set the "order" field in SRS records, so that Inventory Single Record imports will display proper information in the log
   4. Timing: On demand
   5. Related Jiras: 
      1. MODSOURMAN-422: Add file name info and record sequence number for records posted direct via API
      2. UIDATIMP-1136: No log displaying for (some? all?) of the single record imports
7. Script to remove identifiers with invalid identifier type from instance records
   1. Release: Kiwi or later
   2. Release notes: Kiwi, Lotus
   3. Use case: Remove identifiers with invalid identifier type from instance records
   4. Timing: On demand
   5. Related Jiras:
      1. BF-318 : Remove identifiers with invalid identifier type from instance records
8. Script to change the labels of Inventory reference values
   1. Release: Kiwi or later
   2. Use case: To change the labels (names) of reference values in the Inventory domain and keep the value's existing UUID.
   3. Timing: On demand
9. Script to identify item records with invalid itemLevelCallNumberTypeIds
   1. Release: Nolana or later
   2. Use case: Identify records with invalid call number types so that they can be cleaned up
   3. Timing: On Demand
10. Script to remove SRS MARC Bib records with no links to InstanceIds
    1. Release: prior to Orchid
    2. Use case: Multiple match errors for MARC-to-MARC match by 0XX or 9XX fields, except 001 or 999 ff
    3. Timing: On Demand
    4. Related Jiras:
       1. MODSOURCE-585 Data import matching takes incorrect SRS records into consideration
11. Script to remove child and parent elements from data import profiles
    1. Release: prior to Orchid
    2. Use case: slow performance of mod-data-import-converter-storage, OOM crashes of mod-source-record-manager, RecordTooLargeException from Kafka
    3. Timing: On Demand
    4. Related Jiras:
       1. MODDICONV-293 Add validation for creating profiles to prevent saving child and parent elements
       2. UIDATIMP-1369 "parentProfiles" and "childProfiles" properties are populated on profile update
12. Scripts to permanently delete irrelevant Data Import profiles
    1. Release: prior to Poppy
    2. Use case: a particular profile cannot be deleted from the UI, even though the associated Job Profile was deleted
    3. Timing: On Demand
    4. Related Jiras:
       1. MODDICONV-335 Action Profile can not be deleted
          
13. Scripts to detect profiles with mirroring associations
    1. Release: prior to Poppy
    2. Use case: profiles with mirroring associations (invalid profiles, for which details cannot be displayed on UI) will fail to migrate to use wrappers
    3. Timing: On Demand
    4. Related Jiras:
       1. MODDICONV-331 Create a script to detect mirroring associations to run before migration to Poppy

14.Scripts to detect profiles with mirroring associations

1. 1. Release: prior to Poppy
   2. Use case: profiles with mirroring associations (invalid profiles, for which details cannot be displayed on UI) will fail to migrate to use wrappers
   3. Timing: On Demand
   4. Related Jiras:
      1. MODDICONV-331 Create a script to detect mirroring associations to run before migration to Poppy

15.Clean up records with inconsistent matched id values

1. 1. Release: prior to Poppy
   2. Use case: If more than 1 ACTUAL record with the same matched_id - set status OLD to all the other records.
   3. Timing: On Demand
   4. Related Jiras:
      1. MODSOURCE-666 Create script to clean up records with inconsistent matched id values