Data Import Roadmap: Ramsons, Sunflower, & beyond
- Ryan Taylor
Long Term Goals
Data Import can consistently process large uploads in a single job, without failure, to support a library’s higher volume needs.
Large Data Import uploads can be processed at a dependable speed which does not unreasonably delay or hinder other data processing.
Data Import is intuitive for users to complete both common and unique processes with in-UI validation and detailed error messaging to promote proper usage.
Substantial documentation is available to support user understanding of Data Import’s capabilities.
Current State (September 2024)
In September 2024, a survey was shared with the Data Import Subgroup and the larger Metadata Management SIG as a pulse check to identify strengths and weaknesses in the current iteration of Data Import (all responders were running on Poppy release at the time of inquiry). The results of this survey are distilled into high-level bullets below:
Strengths:
Data Import module has been stabilizing over recent releases
Action profiles generally work as expected
Data Import is reliable for most common data-ingestion processes
Weaknesses:
Documentation is lacking
Large jobs can still be unreliable
Match profiles can do better to support partial matches and complex criteria
Hard to know exactly how a complex job profile will process
Lack of clear, actionable error messages
Project Themes
As we identify our Data Import goals and the work needed to reach them, we want to filter project ideas and concepts through the following themes:
Roadmap Overview
Ramsons In progress :
UXPROD-4348: Data Import: Ability to export Data import profile details via APIIn Review
UXPROD-3556: Data Import: Ability to import Data Import profile details via APIIn Review
UXPROD-4469: Data Import Architecture Refinements (Ramsons)In Review
Sunflower Upcoming :
UXPROD-4944: Set record for deletion - Phase 2 enhancementsOpen
UXPROD-4754: Data Import Architecture Refinements (Sunflower)Open
UXPROD-4080: Review and fix MARC Updates for individual fieldsDraft
UXPROD-4704: Stop processing the job after it was canceled by userOpen
*NOTE: Plans for Sunflower are not yet solidified as of 10/8/2024
Potential projects for Trillium, Umbrellaleaf, etc… future releases :