Archived Data Import Implementers and Feature Discussion Topics

Please add any topics that come to mind, or add notes to an existing topic

Maintained by leeda.adkins@duke.edu and the Data Import Subgroup Please add questions and topics here. When noticed, topics or questions posted in Slack will be added here as well.

Priority	Date added	Topic	Resolved?	From who?	Tags	Notes & resolution (include dates for any discussion & resolution)
HIGH	8/17/22	Updating individual fields in MARC Bibs/related instances	NO	multiple		Planned for Nolana Now that we have updated the field protection handling, figure out what additional work is needed on updating individual fields, e.g. could only happen with Update MARC action, not an Update instance action, right? Any fixes needed for repeatable/non-repeatable fields? See UXPROD-3696 - Getting issue details... STATUS
HIGH	8/17/22	Creating/updating multiple Inventory holdings and items from the same incoming MARC Bib	NO	multiple		Planned for Orchid Most important: how does the data need to be formatted in the incoming MARC Bib to identify the appropriate holdings/items to create/update. What if there are multiple items for the same holdings? What if vendor only knows POLs/VRNs, not Inventory HRIDs/UUIDS? See UXPROD-3729 - Getting issue details... STATUS and UXPROD-2741 - Getting issue details... STATUS
MEDIUM	8/17/22	Deleting old (no longer-used) versions of SRS records from SRS	NO	Ann-Marie Breaux (Deactivated)		Lots of questions, needed analysis described in linked Jira. Also needs more discussion with the Subgroup See MODSOURMAN-857 - Getting issue details... STATUS
TBD	8/17/22	Creating (and maybe updating?) MARC holdings from incoming MARC Bib records	NO	Jacquie Samples		See UXPROD-3782 - Getting issue details... STATUS for sample file and additional questions/details
TBD	4/27/22	Creating custom job profiles for MARC Holdings	NO	leeda.adkins@duke.edu		For libraries storing MARC Holdings in SRS and Inventory holdings is controlled by it, are there cases where libraries will need to create custom job profiles in the UI (currently only a profile to delete, and no UI option)
TBD	11/17/21	Protecting fields in Instances when they are overlaid by MARC	NO	Jenn Colt		It might be that this is too much of an edge case to worry about, but it came up for us so we are just wondering about opinions. In this case it was a FOLIO record that had a link in it that we would have liked preserved when overlaid with marc.
DONE?	11/15/21	Brainstorm log enhancements	NO	multiple		Aim for some updates in Morning Glory
DONE?	11/15/21	Importing updates for MARC Authority records	NO			Spitfire will be working on this in Morning Glory
TBD	11/15/21	Analyzing issues with Inventory Single Record Import	NO			Unclear if the issue are in Inventory, Data Import, or both
TBD	11/15/21	Finalize stories for Creating invoices from MARC Bibs	NO			Should be fairly simple effort since most of the work was done for EDIFACT invoice import
DONE?	11/15/21	Discuss and create stories for Creating orders from MARC Bibs	IN PROCESS			17 August: Working on this in Nolana
MEDIUM	11/15/21	Discuss what should happen with SRS MARC Bibs when their corresponding Instances are deleted from Inventory	NO
TBD	11/15/21	How could we start cleaning up SRS? Work on preview feature? Rollback feature? Endpoint for deleting free-floating SRS records?	NO	Jenn Colt Raegan Wiechert		Break into separate topics?
HIGH	8/18/21	Shared Rancher environment for Import/Export/QM SMEs and POs	NO	Ann-Marie Breaux (Deactivated)
HIGH	6/1/21	Default behaviors (implicit/explicit)	NO	Ann-Marie Breaux (Deactivated)		A-M started a spreadsheet; will this be helpful as a way to make the relationships between the actions clearer? Added rows for quickMARC edit/create actions and Instances created by POLs Update instance - only updates fields that are not part of the MARC record, e.g. instance status Is implicit a temporary situation or not? Does it need to be temporary? Why or why not?
HIGH	6/1/21	Creating FOLIO orders from MARC 9xx data	IN PROCESS	Ann-Marie Breaux (Deactivated)		Lehigh MARC program creates SRS MARC for the new orders as well as Instances. In FOLIO, currently POLs only create Instances, not SRS MARCs Per Michelle Suranofsky this is the project: https://github.com/folio-labs/order-import-poc Chicago has made it more configurable and added the ability to create invoices. Their branch: https://github.com/folio-labs/order-import-poc/tree/uchicago-invoice-import
HIGH	6/1/21	Surfacing errors in log summaries				Split into multiple? Fix column sort Export anything? Only errors? Improve UI order messages
MEDIUM	8/18/21	Deleting import logs	DONE	Ann-Marie Breaux (Deactivated)		Delivered in Morning Glory
DONE?	4/27/2018	Using barcode as a match point		Leeda/Duke	matching	for outsourced cataloging record updates
DONE?	4/27/2018	Loading non-Roman script records		Leeda/Duke	non-Roman
TBD	4/27/2018	Merging best data from multiple versions of the record		Lisa/Lehigh	merging	like MARCedit can do
DONE?	4/27/2018	Loading authority records		Mary/Alabama	authorities
TBD	4/27/2018	Using batch load for deletes		Christie/Chicago	deletes
DONE?	4/27/2018	Export-clean-reload workflow		Christie/Chicago Mary/Alabama	export-reload	our group focusing on loads, not exports; where does exporting fit into the system and dev work?
DUPLICATE?	4/27/2018	Using MARC 9xx data for e-invoicing		Christie/Chicago Leeda/Duke Lynn/NC State	invoicing	do we need to develop MARC 9xx e-invoicing, or is coordinated EDIFACT e-invoicing good enough?
TBD	4/27/2018	Converting spreadsheet into MARC records for batch loading		Christie/Chicago	delimited files	from vendors who cannot supply MARC records; would be helpful if this were part of the FOLIO batch loader, instead of a separate process
TBD	5/2/2018	Batch import of MARC holdings data in same file as MARC bibliographic data		Dec 2017 workshop	holdings
DONE	5/2/2018	Batch import of MARC holdings data in separate file from bibliographic data		Dec 2017 workshop	holdings
DONE	5/2/2018	Ability to match to an existing instance, holdings, or item record and update it.		Dec 2017 workshop	matching
DONE?	5/2/2018	Ability to recognize this instance, holdings, item is not already in FOLIO and create new one		Dec 2017 workshop	creating matching	Different from matching?
TBD	5/2/2018	Logistics around importing batch files (not so much the file itself, but the support processes for dealing with imported MARC files)		Dec 2017 workshop	logistics pickup scheduling staging	* Scheduling/automatic import vs. triggering import upon command * How are files moved around between the source and FOLIO, e.g. FTP, download from website, other * Staging area before files are actually loaded to FOLIO? * Where are they loaded? Does it differ depending on what they are supposed to do? (e.g. maybe load to acq app if they are being used to bring in order details, but load to inventory app if being used to add/update metadata/holdings/item records?)
DONE?	5/2/2018	Matching		Dec 2017 workshop	matching	* Matching (e.g. matchpoints, cascade of matchpoints, variation by type of file or source of file?) * Overlay (e.g. complete overlay, no overlay of existing bib data (only adding new stuff), protecting some existing bib data)
DONE?	5/2/2018	Templates/defaults/scripts		Dec 2017 workshop	templates	For automatic edits when loading data
TBD	5/3/2018	Handling incoming tag data properly		Ann-Marie	tags	If incoming data has tags, the “auto” or “systems” user needs to have permission to add new tags, or there needs to be an error message if only existing tags are allowed, but a new tag is trying to be created. FOLIO also needs to be able to parse tag data in the same MARC field & subfield, separated by a pipe delimiter, into separate tags. And if any incoming tag data is upper-case, needs to be lower-cased. Any space in incoming tag data needs to be closed up.
TBD	5/17/18	Investigate SolrMARC and Catmandu; anytthing we can learn from them?		WOLFcon		SolrMARC project from UVA (https://code.google.com/archive/p/solrmarc/) might be good to pull from; Catmandu at University of Ghent (http://librecat.org/) also another good project to look at - both are open source projects
DONE?	5/23/18	Hierarchies of matchpoints			matching	from 23 May subgroup meeting
DONE?	5/23/18	What should happen when someone is editing (or recently edited) a record in MARCcat, and the same record is being batch loaded?			merging post-load reporting	from 23 May subgroup meeting
TBD	5/23/18	Should a new order create only an inventory instance at point of order, or also a brief MARC record? Maybe only instance, and create/link the MARC record at point of receipt?		Ann-Marie	creating	from 23 May subgroup meeting
DUPLICATE?	6/19/18	Handling tags in incoming MARC 9xx data: multiple tags separated by commas, strip any spaces, what happens if duplicate tag?, tag permissions as relate to data coming in via file loads		Ann-Marie	parsing data reqs
TBD	7/9/18	Need to load MARC-XML as well as MARC-binary		Ann-Marie	formats	based on convo with Tod Olson; XML gets around the length limitations imposed by Binary format
DONE	7/9/18	Load UTF-8 character set only, or MARC-8 too?		Ann-Marie	formats	based on input from migration subgroup
TBD	7/9/18	Include JSON format as input format for loading as well?		Ann-Marie	formats	3 sources on our data source list output in JSON format. Do we need to handle in v1?
TBD	7/9/18	Be able to load item data from MARC Holdings record?		Ann-Marie	parsing	Any library need this? Up til now, we've assuming Item data is coming in MARC bib or flat file.
DONE	7/9/18	Confirm explicit decimal for incoming financial data		Ann-Marie	data reqs
TBD	7/9/18	Document overlaps and differences with migration data loading		Ann-Marie		See Tod Olson's metadata migration doc