Data synchronization | dependencies across apps

This page is to outline issues and expectations on cross-app workflows, the handling of deletions and hanging references.

JIRAs

Define and describe the architecture for how to keep data in sync across multiple apps

FOLIO-1331 - Getting issue details... STATUS

Define and describe the architecture for seamless push and pull of data between apps (interaction)

FOLIO-1273 - Getting issue details... STATUS

Discuss Post

On distributed updates and eventual consistency

Links & documentation

Use cases

#	Use case / Feature	Requirements and/or bug reports	Apps	Source of truth	Goal(s)
	Data sync
1	Courses records store instructor information that can be pulled from the instructor's record in the Users app. (This is optional - instructors can also simply be added as text.) If an instructor is added to a course, and then information on their user record changes, the updated/changed information does not propagate to the Courses app.	UXPROD-2747 - Getting issue details... STATUS (described by Kelly Drake - Courses does not have a PO as of May 2021)	Courses, Users	Users is the source of truth for the instructor name, if the instructor was added from Users.	When an instructor on a course is linked to a FOLIO user object, and the instructor's name changes, Users should update the name in Courses.
2	Course records pull information about an item into the Courses app when an item is added to a course. (Information includes things stored on the item's parent holding and instance, like title, contributors, and electronic access information.) This is meant to support reserves-specific search needs in the Courses app, as well as integrations with outside course management systems like Sakai and Canvas. Once an item has been added as a reserve, if information about that item has been changed, it does not propagate to the Courses app. Note that there have been some choices made in the Courses UI to pull information from Inventory for display to make sure that Courses staff are seeing the most up to date information - for example, an item's location. However, even though those changes display in the UI, they do not propagate to the underlying reserve data that is stored in the app itself.	UXPROD-2594 - Getting issue details... STATUS (described by Kelly Drake - Courses does not have a PO as of May 2021) MODCR-58 - Getting issue details... STATUS the display in the Courses UI display as being updated, when the json shows it actually isn't.	Courses, Inventory, Data Import	Inventory (source of truth for item information.) Courses (able to push updates on temporary location and temporary loan type back to Inventory, and have Inventory accept them.)	Courses should be able to edit the temporary location and temporary loan type, and listen to changes in Inventory
3	When importing users using mod-user-import, externalSystemID, personal.lastname, and username are needed to prevent duplicate user records. In the UI, the following fields are also required to create a user: patron group, personal email, preferred contact type, and the active flag.		Users, mod-user-import
4	User Barcode is required for checkout ~~and checkin~~ (comment from (OLD ACCOUNT) Erin Nettifee- A patron barcode is not required to checkin items.)		Users, ~~Check in,~~ Check out
5	User ID or username, and password, are needed for login.		Users, mod-login
6	User Barcode, PIN, externalSystemID, password, in whatever combination are needed for SIP-2		Users, SIP2
7	When logging in using SAML, the mod-login requires externalSystemID, barcode, username, email, and possibly the Folio record number.		Users, mod-login-saml
8	Maintain relationship between apps when holdings/items are moved Example use cases: a title is ordered as a separate monograph. When it arrives, it is actually a volume of a larger set and needs to be moved to the set record. A title is ordered is a second copy, but is accidentally placed on a record-bearing account, so a separate catalog record is supplied for it. It needs to be moved to a different record so both copies are associated with the same instance.	UXPROD-1647 - Getting issue details... STATUS and associated jiras UIOR-682 - Getting issue details... STATUS Example of a case where a print book order gets attached to an e-book record and needs to be moved to the print record. See this document A volume of a set comes in on approval. Upon receipt, the staff member recognized that the volume belongs on the set record. The item is moved from the holdings on the individual volume record to the holdings where the rest of the volumes of the set record reside. The record for the individual volume is suppressed/deleted (depending on library preference). Expected action: the POL now links to the Instance for the set, and item is connected to that particular PO. Question here: each volume under the holdings record could be associated with a different POL - does this create problems or complications? Single tenant consortium example: several libraries order the same title from different vendors each getting a different instance record from their vendor and attaching their piece records to it. These all need to be combined onto a single instance record at some point. They may be combined before the items arrive or after, or when some but not all of the items have arrived. In all cases, the items still need to find their piece records.
9	Update requester data (names, barcode) on requests if changed in the requesting user record	FOLIO-2473 - Getting issue details... STATUS	users, mod-circulation (Requests)
10	Update item and instance attributes copied to requests when the inventory instance, holdings, or item records are updated (if necessary)	FOLIO-2474 - Getting issue details... STATUS	inventory, mod-circulation (Requests)
11	Fee/fine syncing - when a fee/fine is created, some information on the inventory record is copied to the fee/fine. If information on the inventory record(s) changes after that point, the changes are not updated to the fee/fine record. Note that some attributes on the fee/fine record should NOT be updated if they change in inventory, because the historical data was the basis of the charge. For example, an item's effective location should remain what it was when the fee/fine was charged, even if it is later updated in Inventory.		Users, check in, loans data, fee/fines data
12	Loans data - when a loan is created, information about the item and the user is copied to the loan record. If information on the user changes after the loan is created, it's unclear if information is updated on the loan. For example, if a patron's name changes, it should copy over to the loan. Similarly, if a holding/item is moved, it's unclear what information is updated or not on the loan (and some of that information perhaps should not be updated, or the move of the holding/item should be prevented.)		Users, Inventory, loans data	Users is source of truth for patron identifying information.
13	Show dependencies/Backlinks in user record to all the apps the user is listed in eg as contact in eManagement or Licenses		Users
14	Organization name updated should be synced with cached name in Agreements and Licenses		Users, Agreements, Licenses
15	Request state and details: some modules outside of mod-circulation may need to be aware of changes in the state or details of a request to perform actions such as communicating with a 3rd-party request broker. Other possible use-cases are modules/systems to monitor request and perform real-time printing of page slips or other automated monitoring.	An event queue of actions on requests that can be consumed by modules outside mod-circulation when actions are performed on open requests (create, update)	mod-circulation (Requests), mod-inn-reach (INN-Reach)
16	Loan state and details: some modules outside of mod-circulation may need to be aware of changes in the state or details of a loan in order to perform actions such as communicating with a 3rd-party request broker. Other possible future use-cases include alternative notice systems.	An event queue of actions on loans that can be consumed by modules outside mod-circulation when actions are performed on open loans (create, update)	mod-circulation (Loans, Check in, Check out), mod-inn-reach (INN-Reach)
17	Updates to Tags should be notified in such a way that any cached tag label in Agreements and Licenses can also be updated NB currently tags cannot be updated so this is not an immediate issue		Tags, Agreements, Licenses
	Deletions
18	Delete user record in UI with check for dependencies in other apps and/or inform other apps that user record has been deleted e.g. Agreements and Licenses both store user UUIDs to link an agreement or license to a user. If that user is deleted then agreements/licenses continues to reference and link UUID even though record deleted	In the UI: UXPROD-2388 - Getting issue details... STATUS UIU-1971 - Getting issue details... STATUS Through the API: UXPROD-2728 - Getting issue details... STATUS Do a general dependency check without deleting a user: UXPROD-2904 - Getting issue details... STATUS	Users, other applications
19	Delete instance records in UI with check for dependencies in other apps. Implications for other apps (added by related POs) Data Import Currently, Data Import does not support deletion of Instances or SRS MARC Bibs, so that would need to be implemented. Also need to determine what happens in SRS when an Instance is deleted. Is the SRS Bib unlinked from the related instance that has been deleted and left floating in SRS? Is the SRS MARC Bib deleted? Orders direct links from Orders to Inventory Instance UUID Agreements/Licenses/ERM no direct links from Agreements or Licenses to Inventory right now, BUT do have direct links to POLs and if a linked POL has an Inventory Instance UUID stored with it, then that will be display as a link to the instance in Inventory. quickMARC expansion Check in/Check out Request Courses	In the UI: UXPROD-1624 - Getting issue details... STATUS	Inventory, other applications	Inventory
20	Delete Organization record in UI with check for dependencies in other apps and/or inform other apps that organization record has been deleted e.g. Agreements and Licenses both store organization UUIDs to link an agreement or license to a organization. If that organization is deleted then agreements/licenses continues to reference and link UUID even though record deleted		Organizations, Agreements, Licences
21	On attempt to delete License, do not delete if license is related to an agreement	Already implemented with a hardcoded lookup at the point the user tries to delete a license	Licenses, Agreements
22	Delete Purchase Order Line record in UI with check for dependencies in other apps and/or inform other apps that organization record has been deleted Delete Order line check for dependencies or inform agreements app (as POL UUID stored as link between agreement entitlement and a POL)		Agreements, Orders
23	Check on linked POLs/records when an Inventory Item/Holding/Instance is deleted via online update signal from union catalogue		Inventory, Orders, external system

Next steps as agreed on in Tech Leads meeting on May 12th 2021

Provide a document/page that gathers information that we have to date and lists the problems in various apps
List solutions that have been found for some of the issues as options
Collect expectations how users would like the solution to be

There is a working group formed with App Interaction and Tech Leads members

From AI will take part:

Owen Stephens

Charlotte Whitt

Brooks Travis

?