Batch Importer (Bib/Acq)
(UXPROD-47)
|
|
| Status: | Closed |
| Project: | UX Product |
| Components: | None |
| Affects versions: | None |
| Fix versions: | Morning Glory (R2 2022) | Parent: | Batch Importer (Bib/Acq) |
| Type: | New Feature | Priority: | P2 |
| Reporter: | Ann-Marie Breaux (Inactive) | Assignee: | Ann-Marie Breaux (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | cornell-priority | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Release: | Morning Glory (R2 2022) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Epic Link: | Batch Importer (Bib/Acq) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Front End Estimate: | Out of scope | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Front-End Confidence factor: | High | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Back End Estimate: | XL < 15 days | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Back End Estimator: | Kateryna Senchenko | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Development Team: | Folijet | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| PO Rank: | 125 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Implement flow control as a way to ensure that Inventory Single Record Import jobs do not get stuck behind large import jobs. For additional details, see ARCH-14 and https://folio-org.atlassian.net/wiki/display/FOLIJET/ARCH-14%3A+Investigate+opportunity+to+not+take+extended+time+for+OCLC+single+record+import+while+large+file+is+importing. Architectural review in Lotus, but implementation will not be until Morning Glory NOTE Description of the Inventory Single Record Import process drafted in the scope of MODCPCT-38: https://docs.google.com/document/d/1LI3THKLH1C5ovrzvqzt3V6isP0Q1xTxwN0KKTh-hj8k/edit?usp=sharing NOTE Spitfire implementing Delete MARC Authorities via UI in Morning Glory
|
| Comments |
| Comment by Ann-Marie Breaux (Inactive) [ 04/Jan/22 ] |
|
Hi Kateryna Senchenko We need a high level BE estimate for this feature, which we've taken out of Lotus Stability/Reliability general feature and moved to Morning Glory. Could you provide ASAP? |
| Comment by Kateryna Senchenko [ 04/Jan/22 ] |
|
Hi Ann-Marie Breaux, I put XL size for this feature taking into account all the necessary performance testing after implementation |
| Comment by Ann-Marie Breaux (Inactive) [ 04/Jan/22 ] |
|
Sounds good - thanks, Kateryna Senchenko. Do you think we need any other stories or tasks for this feature, or can we consider it fully groomed at this point? |
| Comment by Kateryna Senchenko [ 04/Jan/22 ] |
|
I'd consult with Former user once he is back from vacation, from my point of view we are fine, but we can miss something |
| Comment by Kateryna Senchenko [ 25/Jan/22 ] |
|
Hi Ann-Marie Breaux, discussed this feature with Former user and BE guys and we consider this feature refined. No Karate tests required - existing basic DI functionality is already covered, and this change might only impact the performance (OCLC imports should be faster, while main DI import might slow down). To test the performance we have added
|
| Comment by Ann-Marie Breaux (Inactive) [ 25/Jan/22 ] |
|
Thanks, Kateryna Senchenko That is great to hear. Our first refined MG feature! I talked some more with Hkaplanian and Khalilah Gambrell about this. Would it be possible to add one more task? It might be an ARCH, or a task after the Folijet work is done. We'd like someone to review the MODCPCT and UIIN side of Single Record Import and advise if there are any additional changes there (in the process or in the UX) to improve the speed and reliability of the Single record imports, display of created/updated instances, and Inventory UI toasts. Would it be possible to add that? And let's make sure it's time-boxed to just a couple days, at most. |
| Comment by Jenn Colt [ 01/Feb/22 ] |
|
Since our migration to Juniper we are seeing more instances of this from mod-copycat: io.vertx.core.impl.NoStackTraceThrowable: Add record for job execution failed: POST http://pvt.lb.jcnl.folio-eis.us-east-1:9130/change-manager/jobExecutions/414d7f5e-3c47-41c2-85e3-9671c080a32d/records: Connection was closed
Will the changes made under this story help with this or does it need a separate bug? It seems to occur when we have a large job running and then someone tries to bring in a single record.
|
| Comment by Ann-Marie Breaux (Inactive) [ 01/Feb/22 ] |
|
Hi Jenn Colt Good question. Kateryna Senchenko or Vladimir Shalaev, could you review the above comment and respond? And please let me know if we should perhaps add a bug or task in this feature to check for this mod-copycat error once the dev work is done. Thank you! |
| Comment by Kateryna Senchenko [ 03/Feb/22 ] |
|
Hi Ann-Marie Breaux, added MODSOURMAN-690 to check the mod-copycat and ui-inventory side after flow control is implemented. As for the question from Jenn Colt - it looks like an unrelated issue and should be investigated separately. Thank you! |
| Comment by Ann-Marie Breaux (Inactive) [ 11/Feb/22 ] |
|
Hi Kateryna Senchenko One more question about this feature - will we need any new or adjusted Karate tests? |
| Comment by Kateryna Senchenko [ 11/Feb/22 ] |
|
Hi Ann-Marie Breaux, I don't think we'll need additional Karate tests, checking the main functionality with existing tests should suffice. Thank you! |
| Comment by Ann-Marie Breaux (Inactive) [ 23/Mar/22 ] |
|
Hi Kateryna Senchenko and Vladimir Shalaev Please see Khalilah Gambrell's comments on
|
| Comment by Vladimir Shalaev [ 23/Mar/22 ] |
|
IMO this ticket is solving not the general performance of OCLC import issues, but parallel jobs - big data import and OCLC |
| Comment by Ann-Marie Breaux (Inactive) [ 01/Apr/22 ] |
|
Good to know - thanks, Vladimir Shalaev. Kateryna Senchenko Do we need to add a story to account for MARC Authority deletes? These are individual record deletes, triggered from the MARC Authority app UI, and we need to make sure that they do not get stuck behind long-processing import jobs. These were not taken into account when the feature was first scoped, but may be necessary to provide a decent UX. See
|