Skip to:

Skip to Jira Navigation
Skip to Side Navigation
Skip to Main Content

NFR: Data Import and Inventory Technical, NFR, & Misc work (Poppy R2 2023)

Description

Feature to hold miscellaneous work for Data Import app in R2 2023 Poppy. These issues are not related to new features, bugfixes, or releases/schema upgrades/RMB-Stripes, which have their own separate features

Low-priority unfinished stories/bugs move to further out NFR features

Still to do

Review/finalize draft stories
A-M to add stories for
- Update MARC-Instance mappings for relator terms to take punctuation into account (see https://folio-org.atlassian.net/wiki/display/MM/2022-12-14+Data+Import+Subgroup+meeting)

Priority

P2

Labels

NFRdata-importepam-folijet

Fix versions

Poppy (R2 2023)

Development Team

Folijet

Assignee

Ryan Taylor

Solution Architect

None

Parent

UXPROD-47 Batch Importer (Bib/Acq)

Parent Field Value

None

Parent Status

None

Linked issues

continues

NFR: Data Import Technical, NFR, & Misc work (Orchid R1 2023)

defines

Batch Importer (Bib/Acq)

is continued by

NFR: Data Import/Inventory Technical, NFR, & Misc work (Quesnelia R1 2024)

is defined by

DI ERROR when updating instances on NLA environment

SPIKE: Define ways to refactor di-core

Remove redundant RequiredFields enum

928 field in dryrun records is parsed incorrectly - fivecolleges

MARC bib - FOLIO instance mapping | Adjust contributor and relator term mapping WRT punctuation

Create migration script for updating customized rules for Poppy relator terms and punctuation

Failing overlay in Data Import

[RRT] Data import finding location on holdings Create error

Remove from the payload extra key after Multiple Optimistic Locking error reveals

Allow Tenant Collection Topics

format numbers in "totalRecords" columns

avoid private paths in stripes-core imports

avoid experimental decorator syntax

Allow to sort logs by "totalRecords" column

Implement fetch method (Poppy)

Tenant Collection Topics

Implement fetch method (Orchid CSP 3 Clone)

Implement fetch method (Nolana CSP Clone)

Consider Global Load Sensor When Threshold Exceeded

When adding items, cursor is not placed anywhere

leverage cookie-based authentication in all API requests

bump zustand to v4

BoundWith incorrectly displayed in inventory UI

leverage jest-config-stripes for all jest and testing-library packages

Statistical Code drop down "contains" type ahead functionality needed (Instance/Holdings/Items)

i18n item status in item list

Change imports for testing-library/react-hooks

SPIKE: Investigate performance improvement of sorting Instance's Holdings item table

use current optionalDependencies

Bump @folio/stripes-acq-components dependency version to 5.0.0

SPIKE: Investigate the possibility of configuring the data fetching interval time

SPIKE: Validate that import profile names do not use invalid characters

Add accessibility testing to automated tests in UIDATIMP.

Provide "required" field for subfields in field mapping

SPIKE: Investigate the posibility of adding accessibiity testing to automated tests in data import

Fix all the failed accessibility tests in ui-data-import

Enable and fix skipped unit tests

avoid private paths in stripes-core imports

Order field mapping: Add info icon to the Acq unit field

Invoice field mapping: Add info icon to the Acq unit field

View all: Create hotlink from job profile name in log to the job profile details

avoid experimental decorator syntax

Fix warnings in unit tests (part 2)

Refactor the ViewJobLog component to be a functional component

Job summary - format numbers in summary table

Clear searches when switching between profile types

Handle errors for settings->JobProfile (Orchid CSP 4 Clone)

Extend schema for profile relations manipulations.

Platform-complete builds are failing (rancher)

Handle errors for settings->JobProfile (Poppy)

Match profile: update options for Instance "Incoming records"

Match profile: update options for Holdings "Incoming records"

Match profile: update options for Item "Incoming records"

Match profile: update options for MARC Bibliographic "Incoming records"

Match profile: update options for MARC Authority "Incoming records"

leverage jest-config-stripes for all jest and testing-library packages

Test and merge PRs on changes for DI wrappers

use current optionalDependencies

Align stripes dependencies

Provide onChange and ariaLabel props to TextArea component

When MARC modification happens at end of job, SRS MARC log shows Discarded, instead of Created

SPIKE: Define changes needed to allow R/W split of the DB (Poppy)

Data import is getting stuck at 0% progress - RRT

Create a standalone script to update mandatory mapping-rules change

Orchid Tenant initialization fails when using Postgresql v12.14

SPIKE: Investigate Timeout exceptions during big imports (Poppy)

Update calling existing RMB "Read" methods to use new RMB Readonly APIs

MODSOURMAN-1005

SPIKE: Define changes needed to allow R/W split of the DB (Orchid CSP 3 Clone)

MODSOURMAN-1006

SPIKE: Define changes needed to allow R/W split of the DB (Nolana CSP Clone)

MODSOURMAN-1007

SPIKE: Investigate Timeout exceptions during big imports (Orchid CSP 3 Clone)

MODSOURMAN-1008

SPIKE: Investigate Timeout exceptions during big imports (Nolana CSP Clone)

MODSOURMAN-1026

Table mod_source_record_manager.journal_records has no new data after update to mod-source-record-manager:3.6.4

MODSOURMAN-1027

"Created" is not a link in Data Import Log record when importing "MARC Authority" file

MODSOURMAN-1029

Introduce Global Backpressure For Kafka Consumption

MODSOURMAN-1037

Lost holdingsId when update items

MODSOURMAN-1038

Job log is not showing after Holdings update

MODSOURMAN-1040

update data import jobs - complete with errors & no job summary

MODSOURMAN-1052

Create script to update mapping rules for the classification field

Prevent redundant holdings record creation during import open P/E mix order

Update calling existing RMB "Read" methods to use new RMB Readonly APIs

Unable to move item within holdings

Data Import errors: the logs make it look like a duplicate of each is being created and then discarded in the process

Instance records are discarded

Allow to overlay source 'MARC' instances without related MARC record

Items with boundwith relationships not displaying on all holdings

Adjust multiple Holdings update to send DI_ERROR only when no Holdings Updated, otherwise complete with partial success

Adjust multiple Items updates to send DI_ERROR only when no Items Updated, otherwise complete with partial success

Adjust Optimistic locking for multiple holdings and multiple items update

The imported file are stuck in Data Import app - Decode exception

Add necessary interface into required interfaces in mod-inventory md

Tech debt: Increase test coverage to 80%

SPIKE: Design approach to remove post-processing step when Instance is created

[Spike] Investigate mod-source-record-manager timeout issue

Test DI performance with changes in SRS

Add steps on a Wiki for updating mapping rules after module upgrade

Edi data import creates 5 duplicate invoices

User get error when running data import job (No such Tenant)

Unique constraint violation errors in some DI jobs

Data import EDIFACT invoice job error

Data import job profile error in orchid environment

Troubleshoot DI issues on PTF envs

Onboarding - Yaroslav Kiriak

Change file extension validation messages which are represented in code format

Job hangs at 99%

[RRT] Jobs completing with errors

Onboarding - Maksat Galymzhan

Unexpected DI results

Update job stucks at 98/99%

Onboarding - Javokhir Abdullaev

Update folio-s3-client to v2.0.4

Update folio-s3-client to v2.0.5

SPIKE: High CPU During Insert & Update of marc_records_lb

SPIKE: Multiple tenant DI testing - import jobs are hanging

Optimize Insert & Update of marc_records_lb (Poppy)

Update Authority job is stuck

Deleting Records in FOLIO leads to Error

Testing to find optimal parameters for pool size and connection pool timeout for Orchid release

Testing 10k Authority update with poolSize=20 for mod-srm mod-srs

Define the delete approach for marc_indexers records associated with old source records (Poppy)

Implement async migration service (Poppy)

Add support for matching for repeatable marc fields

Batch load records failing with a number of errors inquiry

Optimize Insert & Update of marc_records_lb (Orchid CSP 5 Clone)

Optimize Insert & Update of marc_records_lb (Nolana CSP Clone)

Define the delete approach for marc_indexers records associated with old source records (Orchid CSP Clone)

Define the delete approach for marc_indexers records associated with old source records (Nolana CSP Clone)

Implement async migration service (Orchid CSP 5 Clone)

Implement async migration service (Nolana CSP Clone)

Allow DI to function correctly if marc_indexers version value is not set (Orchid CSP 5 Clone)

Ensure only one background job can be triggered to clean up OLD marc indexers (Orchid CSP 6 Clone)

Implement Retries For Database Queries With Specific Exceptions (Orchid CSP 5 Clone)

Allow DI to function correctly if marc_indexers version value is not set (Poppy Clone)

Implement Retries For Database Queries With Specific Exceptions (Poppy Clone)

Test adding default value for version column in marc_indexers (Orchid CSP 6 Clone)

Describe steps how to upgrade SRS on bugfest

Test adding default value for version column in marc_indexers (Poppy)

Ensure only one background job can be triggered to clean up OLD marc indexers (Poppy Clone)

Make sure constraints and indexes are created for clean deploy of the module

SPIKE: investigation the impact and occurrence of extra 999 records

Remove "test.mode" entry from ModuleDescriptor

Disallow deletion of default MARC field protection settings

Add migration script for profiles with mapping for required subfields

Require permissions interface 5.6 for module "replaces" (rename)

Populate "value" with empty string if this field is absent in match profile incoming records indicators

Update calling existing RMB "Read" methods to use new RMB Readonly APIs

Migrate existing profiles to have wrappers for associations

Add wrappers around profiles to build associations

Add wrappers around profiles to build associations

Allow editing of associations after adding wrappers over profiles

Adjust Create/Update endpoints for profiles

Update GET profileSnapshot

Change profile validation messages which are represented in code format (Orchid CSP 4 Clone)

Change profile validation messages which are represented in code format (Poppy)

SPIKE: Check if mirroring associations are present on BF envs and find a way to resolve them

Action Profile can not be deleted

Improve the un-link mechanism for profile wrappers

Additionally test migration of existing profiles on Bugfest data set

Expand module permissions with permissions for sharing marc instances

Single Record Import error: HTTPError: 400: No record found

Setup Jest for unit testing and add initial test

Return profile associations when linking the profile instead of record itself

Add accessibility testing to automated tests in UIPFIMP

Review external FE PRs

Review external FE PRs

Review external FE PRs

relates to

NFR: UI end-to-end current tests for Folijet (Poppy R2 2023)

Checklist

hide

TestRail: Results

Activity

Show:

Ivan KryzhanovskyiJuly 4, 2023 at 2:39 PM

Reviewed estimation adjusted regarding remained scope of work for Poppy (extended) release.

Details
Reporter
Ann-Marie Breaux(Deactivated)
PO Rank
95
Front End Estimate
XL < 15 days
Front End Estimator
Mariia Aloshyna
Front-End Confidence factor
80%
Back End Estimate
XL < 15 days
Back End Estimator
Kateryna Senchenko
Back-End Confidence factor
60%
Release
Poppy (R2 2023)
TestRail: Cases
Open TestRail: Cases
TestRail: Runs
Open TestRail: Runs

Created October 6, 2022 at 3:33 PM

Updated December 28, 2023 at 7:42 PM

TestRail: Cases

TestRail: Runs

Flag notifications

Something's gone wrong

Looks like you've been signed out. Try logging in again.