2022-07-11 - Data Migration Subgroup Agenda and Notes

Owned by Darsi Rueda
Last updated: Jul 11, 2022

Date

Attendees

Darsi Rueda


Meeting Link

Link to Recordings

Discussion items

Time

Item

Who

Notes

10

Optimisitic locking workarounds

Darsi

Stanford tested two scenarios using the same Okapi endpoint that we use in our DAG (/instance-storage/batc/synchronous?upsert=true). In both tests, I added a new _version property with a value of 1.

For the first test, all the records existed in FOLIO and the upsert POST succeed with each record's _version property incremented to 2.

For the second test, the records did not exist in FOLIO and the upsert POST succeed and each record's _version property stayed as 1.

--

Could disable all through testing phase, and always set _version = 1.  Or at least could test this.

10Optimistic locking workaroundsIan

Ian using Data Import app to do a 40K migration.  It deals with the versioning issue of optimistic locking.  Only does for inventory. Uses batch endpoints for holdings and items.  (Bywater instances are hosted by EBSCO.)


“Upsert works” Jira

MODINVSTOR-924  - new Jira asking for a new api that disables optimistic locking while api runs
Lots of comments on this Jira both for and against. Some folks worried about bulk updates happening during work day and having conflicts.  Others supporting need for multiple loads during migration process, and for batch instance updates.

Julian includes workarounds, including disabling optimistic locking:
ALTER TABLE instance DISABLE TRIGGER set_instance_ol_version_trigger;
ALTER TABLE instance ENABLE TRIGGER set_instance_ol_version_trigger;

set_instance_ol_version_trigger could disable optimistic locking at the database level, for instances, holdings, items.  Postgres commands.

Decided we will do some experimenting with disabling optimistic locking and load some records.  Then enable and load some more with version number, what problems will we run into?

We also should continue to fill out our document with the impact of this problem, AND comment on the Jira ticket so the develops see our comments immediately.  So we’ll advocate for ourselves on two fronts.

10

Advocate for choice on the optimistic locking options. With appropriate permissions, should be possible to have an endpoint that disables, loads, reenables.
Could set the _version to 1 if not included, or set “to correct” version.
(or endpoint could add the version for new records and stay the same for existing records)

15Loading loans

Ian discovered he can load loans without including the loan policy, using storage api. Would have to update item status to checked out.  The circ rules do have a default policy.  Used to be you had to put in the UUID for some policy when loading the loan.  Jeff doesn’t remember having to add a policy UUID, he just put in item/user, regular api endpoint, and it would calculate the right loan policy.  Assumes you are checking this out now.  Have to go back and manually adjust those in a 2nd pass.

Ian’s loans might not get notices or age to lost correctly.  Would have to run reports later to see which were still charged out, send notice through api, bill through api.

Action items

  •