Scripts for Inventory, Source Record Storage, and Data Import Cleanup
The following scripts may be used for identifying and cleaning up records in FOLIO's Source Record Storage and Inventory. More technical details are available via the title hotlink for each script.
Retrieving Inventory Holdings Records and Update/Cleanup then if needed
Release: Kiwi or later
Release notes: Kiwi
Use cases:
Inventory Holdings source is anything other than FOLIO or MARC
If the Inventory Holdings source = MARC, check to see if there is a linked SRS MARC Holdings record
If yes, then keep the Holdings source as MARC, and ensure that View source and Edit via QM in the Actions menu work for the Inventory Holdings record.
If no, then change the Holdings source to FOLIO, and ensure that View source/Edit in QM do not display in the Inventory Holdings Action menu.
Timing: On demand
Related Jira: MODSOURMAN-627: Data integrity issues with instances/holdings records
NOTE: These scripts do not update the Inventory Holdings. They only retrieve holdings that should be reviewed and then possibly subsequently updated
Scripts for identifying and fixing invalid instanceIDs in SRS
Release: Lotus or later
Release notes: Lotus
Use case: Inventory instance source = MARC and "View source" is not available but there is MARC Bibliographic record that has id of this instance at 999 ff $i subfield and has not in "records_lb.external_id" column
Timing: On demand
Related Jira: MODSOURCE-300: Create script for fixing invalid SRS instanceIDs
Script to refresh Instances against an updated MARC Bib-to-Inventory Instance map
Release: Kiwi or later
Release notes: Lotus
Use case: To refresh some or all Inventory Instances, after the library's default MARC Bib-to Inventory Instance map has been updated
Timing: On demand
Related Jira: MODDATAIMP-567: Script to refresh Instances against an updated MARC Bib-to-Inventory Instance map
Script to delete duplicate mapping rules for each record type
Release: Kiwi or later
Release notes: Kiwi
Use case: To clear duplicate mapping_rules records in the db
Timing: On demand
Related Jira: MODSOURMAN-632: Create a script to delete duplicate mapping rules for each record type
Script to identify "bad URLs" (broken links, etc.) in records linking to electronic items
Release: Juniper or later (and maybe earlier... tested on Juniper)
Use cases:
To detect Page Not Found (404) and any other unsuccessful status code
To detect permanent redirects that could be updated to the new URL to improve link stability and loading time
To detect insecure (non-https) links and upgrade them to https if possible
Timing: On demand
Release: Kiwi or later
Use case: To set the "order" field in SRS records, so that Inventory Single Record imports will display proper information in the log
Timing: On demand
Related Jiras:
Script to remove identifiers with invalid identifier type from instance records
Release: Kiwi or later
Use case: Remove identifiers with invalid identifier type from instance records
Timing: On demand
Related Jiras:
Script to change the labels of Inventory reference values
Release: Kiwi or later
Use case: To change the labels (names) of reference values in the Inventory domain and keep the value's existing UUID.
Timing: On demand
Script to identify item records with invalid itemLevelCallNumberTypeIds
Release: Nolana or later
Use case: Identify records with invalid call number types so that they can be cleaned up
Timing: On Demand
Script to remove SRS MARC Bib records with no links to InstanceIds
Release: prior to Orchid
Use case: Multiple match errors for MARC-to-MARC match by 0XX or 9XX fields, except 001 or 999 ff
Timing: On Demand
Related Jiras:
Script to remove child and parent elements from data import profiles
Release: prior to Orchid
Use case: slow performance of mod-data-import-converter-storage, OOM crashes of mod-source-record-manager, RecordTooLargeException from Kafka
Timing: On Demand
Related Jiras:
Scripts to permanently delete irrelevant Data Import profiles
Release: prior to Poppy
Use case: a particular profile cannot be deleted from the UI, even though the associated Job Profile was deleted
Timing: On Demand
Related Jiras:
Scripts to detect profiles with mirroring associations
Release: prior to Poppy
Use case: profiles with mirroring associations (invalid profiles, for which details cannot be displayed on UI) will fail to migrate to use wrappers
Timing: On Demand
Related Jiras:
Clean up records with inconsistent matched id values
Release: prior to Poppy
Use case: If more than 1 ACTUAL record with the same matched_id - set status OLD to all the other records.
Timing: On Demand
Related Jiras:
MODSOURCE-666 Create script to clean up records with inconsistent matched id values
Scripts for receiving id's and wrapper id's for selected jobProfile
Release: Poppy
Use case: to get job profile information in case it migrated with issues
Timing: After migration, on demand
Related Jiras:
MODDICONV-360 Investigate result of migration profiles and profiles snapshots to Poppy
Script to identify invalid associations between Match profile linked to Modify Action profile
Release: Quesnelia
Use case: to get the unique job profile IDs in case it has a Match Profile linked to the Modify Action Profile.
Timing: After migration, on demand
Related Jiras:
MODDICONV-375 Create script to identify invalid associations.
MODDATAIMP-1008 Document scenarios for Modify used with Update action and verify the behavior
Scripts to populate fund code in imported order lines and existing invoice lines
Release: Orchid or later
Use case: fund code is not shown in the imported order lines list, created invoice lines list, and in their CSV export
Timing: On Demand
Related Jiras:
MODDATAIMP-1057 Create scripts to populate fund code in imported order lines and invoice lines
MODORDERS-1102 Order lines search results do not display fund code for orders created by data import
MODINVOSTO-173 [RRT] Invoices do not display fund codes
MODDATAIMP-1045 [RRT] Fund code not appearing in invoice CSV export
Scripts to find and fix invalid index(es) for marc_indexers job
Release: Indexes were created on Quesinalia, but the issue can be observed on subsequent releases
Use case: High spikes of CPU usage observed for SRS (after Quesnelia CSP #10, or Ramsons releases)
Timing: On demand
Related Jiras:
MODSOURCE-834 - Code implementation for increasing performance for the marc_indexers job
BF-1042 - Fixing indexes on the Bugfest environment
MODSOURCE-855 - Additional code improvements