Settings - Reference data decisions and load sequence for migration

Settings - Reference data decisions and load sequence for migration

Scope

This is not intended to be a complete introduction to reference data. It is aimed at providing a list of reference data decisions that should be made prior to migration and documenting the order in which records should be migrated from legacy systems. Also, because this list is based on Texas A&M University's (TAMU) migration, it is not exhaustive, as TAMU did not migrate all possible record types from its previous system.

Data records – Prerequisite reference data/data records, other instructions

Data record

Application

Prerequisite reference data or other data records (settings/... entries are relative to FOLIO baseurl)

Schema or other documentation

Notes

Data record

Application

Prerequisite reference data or other data records (settings/... entries are relative to FOLIO baseurl)

Schema or other documentation

Notes

instances

inventory

  • Reference data

    • MARC bib-to-Instance map (optional; if desired to change default marc-to-instance mapping for all MARC records)

    • settings/inventory instances section (optional)

    • settings/inventory/ instances, holdings, items section statistical code types, statistical codes (optional)

https://github.com/folio-org/mod-source-record-manager/blob/master/mod-source-record-manager-server/src/main/resources/rules/rules.json

Institutions may wish to identify records with statistical codes for a variety of reasons. Example: grouping records supplied by a given ebook vendor.

holdings

inventory

  • Reference data:

    • settings/tenant-settings/location-institutions (required)

    • settings/tenant-settings/location-campuses (required)

    • settings/tenant-settings/location-libraries (required)

    • settings/tenant-settings/servicePoints (required)

    • settings/inventory Holdings section (optional)

    • settings/inventory/ instances, holdings, items section statistical code types, statistical codes (optional)

  • Data records

    • instances (required)

Institutions may wish to identify records with statistical codes for a variety of reasons. Example: grouping records supplied by a given ebook vendor.

items

inventory

  • Reference data

    • settings/inventory/loantypes (required),

    • settings/inventory/materialtypes (required),

    • settings/inventory Items section (optional)

    • settings/inventory/ instances, holdings, items section statistical code types, statistical codes (optional)

  • Data records:

    • instances (required)

    • holdings and their prerequisites (required)

Schema: https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/item-storage.html#item_storage_items_post

Institutions may wish to identify records with statistical codes for a variety of reasons. Example: recording multiple item statuses from a previous system

users

users

  • Reference data:

    • settings/users/groups (required)

    • settings/users/addresstypes (required)

Schema: https://github.com/folio-org/mod-user-import/blob/master/ramls/schemas/userdataimport.json

 

proxies

users

  • Data records: users (required)

Schema: https://s3.amazonaws.com/foliodocs/api/mod-users/p/proxiesFor.html#proxiesfor_post

 

loans

 

  • Reference data:

    • settings/tenant-settings/servicePoints

    • settings/calendar

    • settings/circulation/loan policies

    • settings/circulation/request policies

    • settings/circulation/notices policies

    • settings/circulation/overdue fine policies

    • settings/circulation/lost item fee policies

    • settings/circulation/patron-notices

    • settings/circulation/rules

  • Data records:

    • users and their prerequisites (required)

    • instances, holdings, items and their prerequisites (required)

 

For migration purposes, calendar begin and end dates must include the earliest loan to be migrated and the latest possible due date based on loan policy.

Also for migration, institutions may choose to use storage endpoints or business logic endpoints to migrate loans. See Considerations for migrating loans for a comparison and contrast of these approaches and links to the relevant schema definitions.

requests (recall/hold/page)

requests

  • Data records:

    • instances, holdings, items and their prerequisites (required)

    • users and their prerequisites (required)

    • loans and their prerequisites (required)

https://s3.amazonaws.com/foliodocs/api/mod-circulation/p/circulation.html#circulation_requests_post

https://s3.amazonaws.com/foliodocs/api/mod-circulation-storage/p/request-storage.html#request_storage_requests_post

Institutions have differed in their approach to migrating requests, some choosing to use business logic modules and others choosing storage modules. Both schema urls are listed for that reason.

fees and fines

 

  • Reference records:

    • settings/users/owners (optional)

    • settings/users/feefinestable (optional)

  • Data records:

    • users and their prerequisites (required)

    • instances, holdings, items and their prerequisites (required)

https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/accounts.html#accounts_post

https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/feefineactions.html#feefineactions_post

  • The settings/users/owners and settings/users/feefinestable must be set up if the institution chooses to migrate existing fees/fines as "not automatic" i.e. manual.

  • Institutions have taken different approaches to migrating fees and fines, some creating only account records and others creating account records plus feefineaction records

 

contacts (organizations)

organizations

 

Schema: https://s3.amazonaws.com/foliodocs/api/mod-organizations-storage/p/contact.html#organizations_storage_contacts_post

  • urls in the contact record must begin with 'http://' or 'https://', otherwise the record will be rejected as invalid.

organizations

organizations

  • Reference data:

    • settings/organizations/category (optional),

  • Data records: contacts (optional)

Schema: https://s3.amazonaws.com/foliodocs/api/mod-organizations-storage/p/category.html#organizations_storage_categories_post

  • urls in the organizations record must begin with 'http://' or 'https://', otherwise the record will be rejected as invalid.

purchase orders

orders

  • Reference data: 

    • settings/tenant-settings/location-institutions (required)

    • settings/tenant-settings/location-campuses (required)

    • settings/tenant-settings/location-libraries (required)

    • settings/tenant-settings/servicePoints (required)

    • settings/acquisition-units required if orders/financials must be separated/permissioned differently for different users within the same tenant

  • Data records:

    • instances (optional)

    • organizations (vendors) (required)

    • finance structure (optional)

Schema: https://s3.amazonaws.com/foliodocs/api/mod-orders/p/order.html#orders_composite_orders_post

 

pieces

receiving

  • Data records

    • purchase orders

https://s3.amazonaws.com/foliodocs/api/mod-orders/p/pieces.html#orders_pieces_post

"Pieces" records may be created to contain loose issues that have been checked in but not yet bound and therefore lack an item record in inventory. The piece record requires a uuid link to the titleId that appears in the po line, but that uuid doesn't exist until the purchase order and its lines are POSTed to the /orders/composite-orders endpoint, so it's necessary to post the order po lines and get the title ids before creating the piece records.

As an alternative to creating piece records, some institutions have chosen to update the "receivingHistory" data element in the inventory holdings record to contain information about loose issues.

Reference data (elements are grouped by the url in the settings app and then listed alphabetically)

 

Data element (relative to base url for instance)

 

 

Description

 

 

Prerequisites

 

 

Schema

 

 

Notes

 

 

Data element (relative to base url for instance)

 

 

Description

 

 

Prerequisites

 

 

Schema

 

 

Notes

 

settings/calendar

sets the normal opening and closing hours of a given service point as well as exceptions to normal hours.

settings/tenant-settings/servicePoints

https://s3.amazonaws.com/foliodocs/api/mod-calendar/p/calendar.html#calendar_periods__servicepointid__period_post

As of the Honeysuckle release, circulation staff have experienced problems editing calendars in the settings UI. Institutions may wish to fill out the hours in a json record and post the calendar to the appropriate endpoint.

settings/circulation/fine-policies

define overdue fine rates, maximum overdue fine, and related settings

 

https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/overdue-fine-policy.html#overdue_fines_policies_post

 

settings/circulation/loan-policies

used in circulation module to define basic loan/renewal/recall configuration. If the local tenant wishes to migrate retrospective loans to FOLIO and link them to existing policies and rules, loan-policies must be created before migration.

 

https://s3.amazonaws.com/foliodocs/api/mod-circulation-storage/p/loan-policy-storage.html#loan_policy_storage_loan_policies_post

 

settings/circulation/lost-item-fee-policy

defines whether item "ages to lost" and when, default replacement fees and fines, and related policies

 

https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/lost-item-fee-policy.html#lost_item_fees_policies_post

 

settings/circulation/notice-policies

used in circulation module to define which types of notices are sent in various conditions and what their frequency will be.

  • settings/circulation/patron-notices

https://s3.amazonaws.com/foliodocs/api/mod-circulation-storage/p/patron-notice-policy.html#patron_notice_policy_storage_patron_notice_policies_post

 

settings/circulation/patron-notices

defines the html templates used to compose circulation notices

 

https://s3.amazonaws.com/foliodocs/api/mod-notify/p/patron-notice.html#patron_notice_post

 

settings/circulation/request-policies

defines which types of request (recall, hold, page) are allowed within a given policy

 

https://s3.amazonaws.com/foliodocs/api/mod-circulation-storage/p/request-policy-storage.html#request_policy_storage_request_policies_post

 

settings/circulation/rules

used in circulation to determine combination of  policies that apply to a given circulation situation. 

  • settings/circulation/loan-policies

  • settings/circulation/request-policies

  • settings/circulation/notices-policies

  • settings/circulation/fine-policies

  • settings/circulation/lost-item-fee policy

https://s3.amazonaws.com/foliodocs/api/mod-circulation/p/circulation-rules.html#circulation_rules_put

Rules can be created through the Settings/Circulation/Circulation rules UI or by "putting" a text file to the endpoint. Local tenants with complex circulation rules may find it more convenient to update rules via the text file, although that method requires substituting uuids for text labels. When PUTting the text file to the /circulation/rules endpoint, do not include the uuid in the file.

settings/inventory/callNumberTypes

used in inventory holdings records to identify type of call number. 

 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/call-number-type.html#call_number_types_post

As of Honeysuckle release, the default reference data does not include a call-number-type for 'blank' as defined by MARC 852 indicator 1. Local tenants may need to add the 'blank' value for migration purposes.

settings/inventory/formats

used in inventory instance records to identify record format. Identified in the inventory instance record UI and Settings/Inventory UI as "formats". If instance record is created via Data Import of a MARC record, format is assigned by mapping the MARC field 338 subfield $b. Default list provided in FOLIO based on RDA carrier type codes and names. 

 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/instance-format.html

Instructions on how to modify mapping rules are available at: How to work with mapping rules.

settings/inventory/holdingsTypes

used in inventory holdings records to identify type of holdings record.

 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/holdings-type.html#holdings_types_post

If the local tenant wishes to base their holdings-types codes on USMARC holdings records leader/06, then they will have to create an additional holding type for 'unknown'.

settings/inventory/holdingsNoteTypes

used in inventory holdings records to identify types of note included in the record. 

 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/holdings-note-type.html#holdings_note_types_post

Can be used, for instance, to define a type of note that contains "Latest volume or edition in alternate location".

settings/inventory/identifierTypes

used in inventory instance records to define type of identifier. 

 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/identifier-type.html#identifier_types_post

Local tenant may wish to define specific identifier type for USMARC field 001 to uniquely identify previous system record key.

settings/inventory/resourcetypes

used in inventory instance records to identify record type. Identified in the inventory instance record UI and Settings/Inventory UI as "resource type". If instance record is created via Data Import of a MARC record, instance-type is assigned by mapping the MARC field 336 subfield $b. Default list provided in FOLIO based on RDA content type codes and names. If the MARC record lacks field 336 subfield $b, the instance record is assigned a default instance-type of 'unspecified'.

 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/instance-type.html#instance_types_post

 

settings/inventory/loantypes

used in inventory item records as one of several elements that determine appropriate loan policy. 

 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/loan-type.html#loan_types_post

Local tenant may wish to delete default loan types provided with FOLIO before defining their own.

settings/inventory/materialtypes

required element used in item records to identify the item's type of material. Can be used as a defining data element in circulation-rules. 

 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/material-type.html#material_types_post

Local tenant may wish to delete default material-types provided with FOLIO.

settings/inventory/StatisticalCodeSettings

values for locally-defined codes that can be used in main inventory record types, instances, holdings, items

  • settings/inventory/statisticalCodeTypes

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/statistical-code.html#statistical_codes_post

Institutions may wish to identify records with statistical codes for a variety of reasons. Example: grouping instance or holdings records supplied by a given ebook vendor, identifying an item record data element migrated from a previous system that has no direct counterpart in FOLIO.

settings/inventory/statisticalCodeTypes

types of locally-defined codes that can be used in main inventory record types, instances, holdings, items

 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/statistical-code-type.html#statistical_code_types_post

 

settings/organizations/category (organizations)

used in organization records to identify classes of data, such as addresses, emails, urls, etc. 

 

https://s3.amazonaws.com/foliodocs/api/mod-organizations-storage/p/category.html#organizations_storage_categories_post

Local tenant may wish to delete default categories provided with FOLIO, using the Settings/organizations/category page in the UI.

settings/tenant-settings/addresses

used in purchase orders for Bill to and Ship to addresses.

 

https://s3.amazonaws.com/foliodocs/api/mod-configuration/p/config.html#configurations_entries_post

 

settings/tenant-settings/location-campuses

used in inventory holdings & item records to identify 2nd level in location hierarchy.

  • settings/tenant-settings/location-institutions

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/locationunit.html#location_units_campuses_post

 

settings/tenant-settings/location-institutions

used in inventory holdings & item records to identify top level in location hierarchy.

 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/locationunit.html#location_units_institutions_post

 

settings/tenant-settings/location-libraries

used in inventory holdings & item records to identify 3rd level in location hierarchy.

  • settings/tenant-settings/institutions

  • settings/tenant-settings/campuses

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/locationunit.html#location_units_libraries_post

 

settings/tenant-settings/location-locations

used to identify shelf locations in inventory holdings & item records.

  • settings/tenant-settings/location-institutions

  • settings/tenant-settings/location-campuses

  • settings/tenant-settings/location-libraries

  • settings/tenant-settings/servicePoints

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/location.html#locations_post

Location codes and location names must be unique within a tenant. For example, if the "main" library and "science" 2 each has a reference section, those locations cannot use the location code 'ref' even though the upper levels of the hierarchy identify the location uniquely. One would have to be 'mainref' and the other 'sciref'.

settings/tenant-settings/servicePoints

used primarily in circulation apps to identify points where circulation actions take place. Local tenant may wish to delete default service points provided with FOLIO. As of Q4 2019, the local tenant will apparently have to do the delete using a curl command or other script that deletes from the API endpoint, as there doesn't appear to be a delete function for these in the Settings/Tenant/Service Points UI. Also, although the options to print slips at a specific Service Point appear to be selected by default, they do not become active unless Pickup location is set to 'Yes'.

 

https://s3.amazonaws.com/foliodocs/api/mod-inventory-storage/p/service-point.html#service_points_post

 

settings/users/addresstypes

used in user records to identify type of address. Local tenant may wish to delete default address types provided with FOLIO. 

 

https://s3.amazonaws.com/foliodocs/api/mod-users/p/addressTypes.html#addresstypes_post

 

settings/users/feefinestable

defines manual (as opposed to automatic) charges that can be created in the system. Users can set default amount charged, default charge notice, and default action notice.

  • settings/users/owners

https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/feefines.html#feefines_post

 

settings/users/groups

used in user records to identify patron groups. 

 

https://s3.amazonaws.com/foliodocs/api/mod-users/p/groups.html#groups_post

As of the Honeysuckle release, the system loads reference data for groups, so local tenants will probably want to delete those groups before creating their own.

settings/users/owners

defines how service units are grouped to create a fee/fine "owner". 

 

https://s3.amazonaws.com/foliodocs/api/mod-feesfines/p/owners.html#owners_post

Fee/fine "manual charges", "payment methods", and "transfer accounts" are differentiated by fee/fine owner