2024-3-20 Data Import Subgroup meeting

When creating electronic orders are being created through data import, the electronic resource quantity is not being mapped from marc or from a default value in the mapping profile and the funds are not being encumbered.   Test in Poppy CSP1 in local UChicago environment and in Poppy bugfest. See 

MODDATAIMP-1010 - Getting issue details... STATUS

For Stanford, Acquisition method is not mapping for Purchase. They are also not getting quantity or encumbrances. Also, order type is not mapping for Purchase when it is provided as a default in the import profile for orders. 

Discussion notes:

No one has used the order imports in Orchid. 

Feedback that the quantity should come from the profile/order and not the number of items. Common scenarios where items are not going to be created. 

When processing orders the quantity is always controlled from the order.

Set bib record for deletion is in place in snapshot. 

New set record for deletion action. Need a specific permission to have access to this. 9Separate from inventory All permissions.

Click the action to suppress the record from discovery, staff suppress the record, and set the LDR 05 to d. New SRS property "deleted" set to true. 

Staff suppress will now be defaulted to No, so will not show up in search. 

Q: What happens to attached orders? A: Nothing right now, but Ryan will check on impact on associated records. 

Q: Can you do this batch with a delete file in Data Import? A: No. Just an individual, manual action to take. Step 1 of longer term plans for full deletion options. 

Q: Are other steps going to be continued before this becomes part of a release? A: Will be included as is in Quesnalia. Holdings and items should not be affected as part of this release. Concern: Hacving holdings and items available with the instance being deleted. Cannot imagine a situation in which you want to delete an instance and still have the holdings and item information available with a search. Also, if an item is checked out, it should block you from deleting a bib. Or on hold. Or in course reserves. Not interactive link with inventory or circulation control. 

Follow up Q: Would there be a script available to get this added to Instance records a library has already marked for delete? 

Q: What is the push this out in its partially developed state? A: Someone would use it to delete duplicate instances in the system. These things usually do not have holdings or items.

Q: What would happen if a deleted record was matched and updated as a part of a data import? A: Believe that these records are not discoverable via data import matching and updates, but will check on that.

Q: How do holdings and items show up in search if the instance is not available in the search? A: notetaker is unclear of the answer to this question

Q: Can you reverse this process or undelete things? A: Manually edit the instance to undo the suppression, but cannot edit the marc once it has been set to deletion. Triggered by the status to deletion. Question about whether we need to revisit this. This functionality should be robustly described in the documentation so users fully understand the implications of deleting something. 

Suggestion: The warning toast should be delete rather than just suppressed from discovery and staff suppressed because there are implications like not being able to edit the srs marc

Suggestion to postpone until Ramsons.

Comments that marc modifications was implemented with an expectation that marc modifications should be at the beginning of the job and should act on the incoming record. 

That is true, but past conversations in data import subgroup drew out two use cases: 1 to modify an incoming record before any actions are taken and 2) to modify the final srs record after all of the actions are taken. (Delete 9xx data after it is used to update the holdings and item, for example.)

General experience right now is that marc modifications are working as expected with creates, but is not working or working but with corruption (such as the deletion of protected fields) on updates.

RT: Is part of the problem how we are approaching updates vs modifications? Updates are designed to work with FOLIO records and modifications are designed to work on incoming records.  Should updates have the same potential actions as marc modifications applying the logic to the updated record?

Right now dependencies between srs and instance and the explicit nature of the updates on instance vs marc is problematic. It is difficult to understand what is happening with updates. Process is to put them anywhere to see where they work. Whether we are updating srs, instance or both we should be able to do the same thing. 

RT: You will see different behavior from marc modifications depending on its placement in the profile. Need to a deeper dive into how the behavior changes dependent on placement. 

This would be a good candidate for the functionality / documentation audit.  If development dives into this and the DI lab group dives into this, we could then come together to identify the best way forward

There are 2 issues: experience of unlinking post migration and then experience of unlinking during migration. MODICONV-361 is a P1 with the hope to be released in a CSP #1. The MODICONV-365 is being investigated.

It looks like FOLIO system job profiles are being affected in terms of actions being unlinked. 5C saw that the default ISRI overlay wasn't working correctly. When we checked the default system job profile there were no actions profiles.

Ryan confirmed this issue only affects Action profiles. It is difficult to know how common this is. For 361, the behavior seems consistent. But for 365, this seems to be less common and different tenants have the issue occur on different jobs.

This is the 3rd or 4th time that the issue in MODDICONV-361 has appeared during a flower release. The unlinking/linking issues date back several releases.

To gather more information, it is worth keeping the corrupted jobs and create replacements.

A job with no action profiles or an empty job can be run. There are no error messages when such a job is run. This is something we shouldn't be able to do. Perhaps a warning or an error message is needed.

Overview : Action profiles connected to multiple job profiles are 'unlinked' from job profiles after migration to Poppy.

• It isn't happening for every library after migration or for every re-used profile.
• However, it is happening often enough that libraries should be aware and check.  
• General confusion on how or if the migration to Poppy is causing this issue.  Root cause is not migration, but the migration process does cause the profiles to unlink.
• Script #15 (noted in the topic column) provides a list of profiles that need to be fixed.  It does not fix the links.  That must be done manually by the library.  
• The action profiles actually disappear from the job profile, not just 'unlink'.  They must be re-added.

Comments :

• Lots of work for libraries to recreate job profiles manually.
• Should be a CSP candidate. High priority for correction.
• Could be a blocker for some libraries to migrate to Poppy.
• The "unlinking from one unlinks them all" issue has popped up multiple times.

Sidebar discussion in chat on how job profiles are deleted spurred #42 in the Data Import Issue Tracker.  

Until MODDICONV-361 is fixed, any time a re-used action profile is unlinked in a job profile it will be unlinked in all other job profiles.  Fixing it after migration doesn't stop it from happening again should a re-used action profile be unlinked.  

The development team will be adding new test cases to their workflow to test this type of scenario (re-used profiles) going forward.

Partial matching, e.g. begins with, ends with, is required but it does not function as it should regardless of how it is configured.

• System behaves as though it only looks for exact matches.
• Examples of use include prefixes/suffixes to an 035 added by a vendor or library to designate the source of the record.
• University of Chicago has had same issue.  Corrie submitted MODDICORE-386 on their behalf.
• Question as to whether this is a bug or how the system is intended to function.  Documentation is needed.
• #12 on the Data Import Issue Tracker.  

In lab session on 1/18/2024, we created a wiki page, Data Import Topic Tracker, with guidelines on how to contribute and a spreadsheet to track issues. This is based on the work done in the Acquisitions SIG. An archive area was also created where we could archive outdated pages such as the Archived Data Import Implementers and Feature Discussion Topics.

The idea was to put down issues whether they were linked to a Jira issue or not. Some of the important information that we wanted to track was if there was a linked Jira and in particular when the issue was discussed in the working group and the decision(s) made in regard to that issue.

The spreadsheet is still being developed. Before we add more issues, the group in lab wanted to know:

• Do we adopt this page and spreadsheet? If yes, do we have volunteers to populate it?
• To make sure this page is maintained, the group suggested that the working group look at it once a month to see what is outstanding or new. Is this a practice we want to adopt?

Discussion: 

A link to the new Data Import topic tracker is at the top of the page. Format was worked on at last week's data import session. 

Q: is this only to track Jira tickets? Or will there be other topics added to the agenda. R: In Acq /RM individuals add stories to the topic tracker and the Jira may only be added later to the spreadsheet. (many think this is a good idea.)

Can reference the Acq/Resource Management implementers topic tracker.

Perhaps add widgets that bring in Jiras automatically based on the tag. 

Q: How to add "Click here and expand" text. R: Put the cursor where you want the text block to begin and use Insert Macro function. Type "Expand" to locate the Expand Macro.

Agreed to use the de-duplication discussion to work on building a useful functionality framework.