Data Import Roadmap: Ramsons, Sunflower, & beyond

Owned by Ryan Taylor, created
Last updated: Dec 04, 2024
2 min read

Long Term Goals

  • Data Import can consistently process large uploads in a single job, without failure, to support a library’s higher volume needs.

  • Large Data Import uploads can be processed at a dependable speed which does not unreasonably delay or hinder other data processing.

  • Data Import is intuitive for users to complete both common and unique processes with in-UI validation and detailed error messaging to promote proper usage.

  • Substantial documentation is available to support user understanding of Data Import’s capabilities.

 Current State (September 2024)

In September 2024, a survey was shared with the Data Import Subgroup and the larger Metadata Management SIG as a pulse check to identify strengths and weaknesses in the current iteration of Data Import (all responders were running on Poppy release at the time of inquiry). The results of this survey are distilled into high-level bullets below:

Strengths:

  • Data Import module has been stabilizing over recent releases

  • Action profiles generally work as expected

  • Data Import is reliable for most common data-ingestion processes

 

Weaknesses:

  • Documentation is lacking

  • Large jobs can still be unreliable

  • Match profiles can do better to support partial matches and complex criteria

  • Hard to know exactly how a complex job profile will process

  • Lack of clear, actionable error messages

 Project Themes

As we identify our Data Import goals and the work needed to reach them, we want to filter project ideas and concepts through the following themes:

 Roadmap Overview

Ramsons In progress :

UXPROD-4348: Data Import: Ability to export Data import profile details via APIIn Review

UXPROD-3556: Data Import: Ability to import Data Import profile details via APIIn Review

UXPROD-4081: Refine and standardize handling of 035 OCLC numbers in MARC records (including potential duplicates)In Review

UXPROD-4469: Data Import Architecture Refinements (Ramsons)In Review

Sunflower Upcoming :

UXPROD-4944: Set record for deletion - Phase 2 enhancementsOpen

UXPROD-4754: Data Import Architecture Refinements (Sunflower)Open

Potential projects for Trillium, Umbrellaleaf, etc… future releases :