NFR: R2 2021 Juniper Data Import Stabilization and Reliability work
Description
Priority
Fix versions
Development Team
Assignee

Solution Architect
Parent Field Value
Parent Status
defines
is defined by
relates to
Checklist
hideTestRail: Results
Activity

Ann-Marie Breaux June 17, 2021 at 6:49 AM
Architectural changes moved to Kiwi feature

Ann-Marie Breaux May 26, 2021 at 4:02 PM
After internal conversation, decided to deprioritize some of these architectural changes for Juniper, in favor of bugfixes and automated tests

Ann-Marie Breaux May 18, 2021 at 3:25 PMEdited
Based on number of stories and total points, we likely will need to split this feature, and continue the work in R3

Ann-Marie Breaux April 29, 2021 at 7:50 AM
Hi We'll be putting together details on the wiki in the next week or two. We started with Creates for SRS Bibs, and all 3 Inventory record types. Now we are also working with multiple updates to the same records. We have 5 job profiles that should result in successes for all updates, 1 job profile that will fail for holdings and item updates, and 1 job profile that will fail on item updates. We are not yet making them as varied as you'll encounter in a real use case. Another large part of the testing is related to processing time, so we want predictable results before we start introducing more unpredictability. That whole walk before you run thing.
With a few adjustments to environment setup details, a load of 50K creating or updated 50K SRS, Instances, Holdings, and Items completes in ca. 2.5 hours. We think there are still some steps that we can take to improve that performance, but the devs are still analyzing the various runs, adjusting settings, and trying again. More soon, I promise!

Jacquie Samples April 28, 2021 at 1:21 PMEdited
I am curious to know what kind of testing has happened on the data that was loaded in the 50k import. Has anyone tested updating or overlaying those bib and/or order data? In other words, was the load successful in that those data are now actionable and effective?
I am also interested to know how long it took to load the 50k bib records.
Thanks!
Details
Reporter
Ann-Marie BreauxAnn-Marie Breaux(Deactivated)PO Rank
115Front End Estimate
Very Small (VS) < 1dayFront End Estimator
Ivan KryzhanovskyiIvan KryzhanovskyiFront-End Confidence factor
MediumBack End Estimate
Jumbo: > 45 daysBack End Estimator
Ann-Marie BreauxAnn-Marie Breaux(Deactivated)TestRail: Cases
Open TestRail: CasesTestRail: Runs
Open TestRail: Runs
Details
Details
Reporter

PO Rank
Front End Estimate
Front End Estimator

Front-End Confidence factor
Back End Estimate
Back End Estimator

Architectural and Infrastructure work to make Data Import more stable and performant
Based on feedback from the PTF and architects
As of 14 April 2021, PTF has tested importing files of 50K MARC to create Bibs, Instances, Holdings, Item
Additional testing still needed
Multiple imports triggered by different users at the same time
Imports that update SRS and Inventory records (so have matching portion in the job profile, which creates does not have)
EDIFACT - to get a large renewal invoice from Anne C, so that we can see if that causes any issues