/
Data Import Roadmap: Ramsons, Sunflower, & beyond

Data Import Roadmap: Ramsons, Sunflower, & beyond

Owned by Ryan Taylor, created
Last updated: Mar 07, 2025
2 min read

Long Term Goals

  • Data Import can consistently process large uploads in a single job, without failure, to support a library’s higher volume needs.

  • Large Data Import uploads can be processed at a dependable speed which does not unreasonably delay or hinder other data processing.

  • Data Import is intuitive for users to complete both common and unique processes with in-UI validation and detailed error messaging to promote proper usage.

  • Substantial documentation is available to support user understanding of Data Import’s capabilities.

 Current State (September 2024)

In September 2024, a survey was shared with the Data Import Subgroup and the larger Metadata Management SIG as a pulse check to identify strengths and weaknesses in the current iteration of Data Import (all responders were running on Poppy release at the time of inquiry). The results of this survey are distilled into high-level bullets below:

Strengths:

  • Data Import module has been stabilizing over recent releases

  • Action profiles generally work as expected

  • Data Import is reliable for most common data-ingestion processes

 

Weaknesses:

  • Documentation is lacking

  • Large jobs can still be unreliable

  • Match profiles can do better to support partial matches and complex criteria

  • Hard to know exactly how a complex job profile will process

  • Lack of clear, actionable error messages

 Project Themes

As we identify our Data Import goals and the work needed to reach them, we want to filter project ideas and concepts through the following themes:

Usability

  • Is the design and user experience of Data Import intuitive?

  • Are features of the app easily understood?

  • Is documentation clear, understandable, and accessible

Functionality

  • Does Data Import do what we expect it to do and does it do it well?

  • Does it "work"?

  • Do we need enhancements to support user needs?

Stability

  • Is Data Import stable in an average environment?

  • Does it crash in certain scenarios?

  • Is it consistent?

Performance

  • How quickly can Data Import jobs process?

  • Is memory/CPU usage well-managed?

  • Does it run well with average tenant setup?

 Roadmap Overview

Ramsons DONE :

UXPROD-4348: Data Import: Ability to export Data import profile details via APIClosed

UXPROD-3556: Data Import: Ability to import Data Import profile details via APIClosed

UXPROD-4081: Refine and standardize handling of 035 OCLC numbers in MARC records (including potential duplicates)Closed

UXPROD-4469: Data Import Architecture Refinements (Ramsons)Closed

Sunflower In progress :

UXPROD-4944: Set record for deletion - Phase 2 enhancementsIn Progress

Trillium Planning :

UXPROD-4754: Data Import Architecture Refinements (Trillium)Open

Potential projects for Trillium, Umbrellaleaf, etc… future releases :

 

Related content