Batch Importer (Bib/Acq) (UXPROD-47)

[UXPROD-2114] Define human readable identifiers (HRIDs) for Inventory Created: 11/Oct/19  Updated: 16/Sep/20  Resolved: 26/Dec/19

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: Q4 2019
Parent: Batch Importer (Bib/Acq)

Type: New Feature Priority: P3
Reporter: Charlotte Whitt Assignee: Ann-Marie Breaux (Inactive)
Resolution: Done Votes: 0
Labels: cap-mvp, inventory, mandatory, migration-load, po-mvp, q4-2019-at-risk, sysops_mgt
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: PNG File Skärmavbild 2018-12-20 kl. 17.17.50.png     PNG File Skärmavbild 2018-12-20 kl. 17.19.02.png     PNG File Skärmavbild 2018-12-20 kl. 17.19.42.png     PNG File Skärmavbild 2018-12-20 kl. 17.26.58.png     PNG File Skärmavbild 2019-04-08 kl. 18.25.03.png    
Issue links:
Blocks
blocks UXPROD-2115 Define human readable identifiers (HR... Closed
blocks UXPROD-2116 Define human readable identifiers (HR... Closed
Defines
is defined by MODINV-160 HRID handling in Inventory for Instan... Closed
is defined by MODINV-161 HRID handling in Inventory for Holdin... Closed
is defined by MODINV-162 HRID handling in Inventory for Item r... Closed
is defined by MODINVSTOR-363 Create settings API for hrId Closed
is defined by MODINVSTOR-373 Generate HRID for instance record Closed
is defined by MODINVSTOR-374 Generate HRID for holdings record Closed
is defined by MODINVSTOR-375 Generate HRID for item record Closed
is defined by MODINVSTOR-410 Increase digits for Inventory Instanc... Closed
is defined by MODINVSTOR-411 Increase digits for Inventory Holding... Closed
is defined by MODINVSTOR-412 Increase digits for Inventory Item HRIDs Closed
Relates
relates to UIIN-369 MARC records don't contain the UUID i... Closed
relates to UIIN-752 Unmock the backend once the MODINV/MO... Closed
relates to UIIN-873 Holdings record. When creating new ho... Closed
relates to UIIN-874 Item record. When viewing an item rec... Closed
relates to UIIN-875 Instance, Holding and Item: Get catas... Closed
relates to UIIN-876 Instance record. When viewing an inst... Closed
relates to UXPROD-1595 Connect SRS and MARCcat - Folijet wor... Closed
relates to UXPROD-2078 SRS MARC-Inventory Instance relations... Closed
relates to UXPROD-2115 Define human readable identifiers (HR... Closed
relates to UXPROD-2116 Define human readable identifiers (HR... Closed
relates to UXPROD-2218 Connect SRS and MARCcat - AtCult work... Closed
relates to UIIN-292 HRID - UI refinements Closed
relates to UXPROD-1713 Data Migration. Migrate bibliographic... Closed
relates to UIIN-898 Maximum instance/holding/item HRID is... Closed
Epic Link: Batch Importer (Bib/Acq)
Front End Estimate: Medium < 5 days
Front End Estimator: Jakub Skoczen
Back End Estimate: Large < 10 days
Back End Estimator: Jakub Skoczen
Development Team: Prokopovych
PO Rank: 110
PO Ranking Note: Put it at the top of the Q4 list, as it is the highest ranked not in Q3 2019
Rank: Chalmers (Impl Aut 2019): R5
Rank: Chicago (MVP Sum 2020): R1
Rank: Cornell (Full Sum 2021): R1
Rank: Duke (Full Sum 2021): R1
Rank: 5Colleges (Full Jul 2021): R1
Rank: FLO (MVP Sum 2020): R1
Rank: GBV (MVP Sum 2020): R1
Rank: hbz (TBD): R1
Rank: Lehigh (MVP Summer 2020): R1
Rank: Leipzig (Full TBD): R1
Rank: TAMU (MVP Jan 2021): R1
Rank: U of AL (MVP Oct 2020): R1

 Description   

See details in Umbrella UXPROD-884, plus Slide Deck:

https://docs.google.com/presentation/d/1D4yEPVhFbD8ZXlyKfInRafnJUfQWM8syakLu7v6_c8U/edit#slide=id.p1

Note that the maximum digits for the ID are 11 (slide deck says 8), and for the prefix are 10, so the HRID fields in Inventory Instances, Holdings, and Items must allow for up to 21 characters



 Comments   
Comment by Ann-Marie Breaux (Inactive) [ 29/Oct/19 ]

Cate Boerema Just checking - do you think the Open "is defined by" stories linked to this feature will make it into Core-fxn's next sprint?

Comment by Cate Boerema (Inactive) [ 30/Oct/19 ]

We'll pull in as much as Matt Reno thinks he can handle for the upcoming sprint. We might also be able to bring in Bohdan to help out if Matt Reno thinks it could be helpful. What do you think, Matt?

Comment by Ann-Marie Breaux (Inactive) [ 18/Nov/19 ]

Hi Cate Boerema Any chance for the remaining MODINV stories in this feature to be handled in the current sprint? If not, I'll mark this as "at risk" for Q4. Thank you

Comment by Charlotte Whitt [ 19/Nov/19 ]

Hi Ann-Marie Breaux - do you have a sense of how much work are remaining on this feature?

  1. MODINV-160 Closed HRID handling in Inventory for Instance records – Story Points: 5 Open
  2. MODINV-162 Closed HRID handling in Inventory for Item records – Story Points: 5 Open
  3. MODINVSTOR-363 Closed Create settings API for hrI – Story Points: 5 In Review
  4. MODINVSTOR-373 Closed Placeholder for CF part of MODINV-160 Closed Story Points: 8 In progress
  5. MODINVSTOR-374 Closed Placeholder for CF part of MODINV-161 Closed Story Points: 8 Open
  6. MODINVSTOR-375 Closed Placeholder for CF part of MODINV-162 Closed Story Points: 8 Open

Total: Story Points: 39

Comment by Charlotte Whitt [ 20/Nov/19 ]

Hi Marc Johnson and Ann-Marie Breaux - I'll move the HRIDs in Inventory conversation out of UIIN-875:

Marc Johnson added a comment - Yesterday
Cate Boerema Charlotte Whitt Ann-Marie Breaux

It is clear I do not have the broader context of the HRID work.

I am finding it difficult to follow the reasoning that it would have been better to leave the old behaviour in place, because I understand that wasn’t fit for purpose and did not allow clients to specify their own HRIDs, meaning migration wasn’t viable without it’s removal.

It may well be that I misunderstood the urgency of replacing the existing work.

I shall leave it to the three of you to decide upon the strategic direction for this work.

I would caution that we do not know how much of this work we will or won’t have in place until the Q4 release development has ended, so whilst I understand the HRID feature is at risk for Q4, it is possibly too soon to be deciding that it won’t be sufficiently in place.

Any advice on how to reduce the scope of that work without undermining it would be appreciated.

Ann-Marie Breaux added a comment - Yesterday
Hi Marc Johnson I'm sorry that this seems to have blown up. I vote that we soldier on, with the intention of having something that is not erroring and is predictable by the end of Edelweiss. I don't want to go backwards if at all possible. The sooner the inventory HRID work is complete, the sooner it unblocks parallel HRID work for SRS and MARCcat.

Charlotte Whitt and Cate Boerema interested in your thoughts as well.

Comment by Charlotte Whitt [ 20/Nov/19 ]

The sooner the inventory HRID work is complete, the sooner it unblocks parallel HRID work for SRS and MARCcat.

My thinking is: If HRIDs in Inventory will not be solved for Q4 2019 - then this work is a must for Q1 2020. We have early implementers who have absolutely no work arounds for this, if the HRIDs in Inventory is not enabled. The institution can not migrate their data if there is no HRIDs in instance, holdings, item records, and they must be able to continue their existing number sequence, when adding new records.

Comment by Cate Boerema (Inactive) [ 21/Nov/19 ]

Thanks for your comment, Charlotte Whitt. We understand the urgency on this and are pushing forward as quickly as possible. What doesn't get completed in Q4 will definitely be scheduled for Q1.

Comment by Ann-Marie Breaux (Inactive) [ 21/Nov/19 ]

Thanks Cate Boerema I can see that Matt Reno is working on stuff, and it's clear that things are moving forward as fast as possible. I'm not totally clear on how the parallel MODINVSTOR and MODINV stories relate to each other, and whether they really add up to 13 points worth of work for each of the 3 Inventory record types. My hope is that after the first record type, it'll get quicker/easier for the next two.

Charlotte Whitt once the Inventory work is done, we'll definitely make the corresponding SRS work high priority for Folijet. I can't speak to the MACRcat work because I don't have any understanding yet of when AtCult might be able to start on it, and how it fits with their other priorities. Eventually (and I would say sooner rather than later) we need to have all 3 apps accessing the same sequential number generator and referring to the same HRID settings.

Comment by Marc Johnson [ 21/Nov/19 ]

Eventually (and I would say sooner rather than later) we need to have all 3 apps accessing the same sequential number generator and referring to the same HRID settings

By accessing the same sequence, do you mean that data import and MARCCat will take the HRID allocated to an instance and use that in their own records.

That is what I understood, which is why the sequence is currently private to inventory.

Comment by Marc Johnson [ 21/Nov/19 ]

Ann-Marie Breaux

My hope is that after the first record type, it'll get quicker/easier for the next two.

My understanding is that the unknowns will have reduced after the first record type, and the work will be better understood. This might make it slightly faster, there is still quite a lot of changes to make.

Instances are likely the most involved because of the different ways that instances can be created, there are two different batch APIs as well as the single record way

Comment by Ann-Marie Breaux (Inactive) [ 21/Nov/19 ]

Hi Marc Johnson By accessing the same sequence, I mean that if a record is starting life in MARCcat or SRS, then the next HRID will be grabbed from Inventory (and will represent the Instance HRID), so that we can display that ID in the SRS and MARCcat versions of the record as well.

If a record is starting life via SRS, the sequences would be:

If a new HRID is to be assigned:
1. New MARC record created in SRS during import
2. There is no 001 field, so we know a new HRID needs to be assigned
3. FOLIO creates the corresponding instance and assigns the next sequential HRID
4. Inventory returns the Instance UUID to SRS to be stored in the MARC record 999 field
5. Inventory returns the Instance HRID to SRS to be stored in the MARC record 001 field

If already an existing HRID (no new HRID to be created)
1. New MARC record created in SRS during import
2. There is an 001 field, so that is retained as the Inventory Instance HRID
3. FOLIO creates the corresponding Instance and assigns the 001 as the HRID
4. Inventory returns the Instance UUID to SRS to be stored in the MARC record 999 field
5. SRS already has the Instance HRID in the 001 field

Comment by Marc Johnson [ 21/Nov/19 ]

Thanks Ann-Marie Breaux

I think the key statements here are:

3. FOLIO creates the corresponding instance and assigns the next sequential HRID
...
5. Inventory returns the Instance HRID to SRS to be stored in the MARC record 001 field

Which means that an HRID is allocated to an instance, and then used by SRS.

Rather than SRS being allocated a new HRID and that being used when creating the HRID.

These are subtly different, in the first case the sequence itself is hidden from clients, in the latter it must (well allocated from it at least) be exposed.

Does that make sense?

Comment by Ann-Marie Breaux (Inactive) [ 21/Nov/19 ]

Yes, I get it - and it'll be similar with MARCcat. If a new record is being created, the HRID needs to come from the Inventory sequence. We've just got a "then a miracle occurs" step since we're still not quite sure yet about the MARCcat connection/synchronization with SRS/Inventory. That's why sorting out the MARCcat/HRID interaction is last

Comment by Marc Johnson [ 21/Nov/19 ]

Ann-Marie Breaux

Yes, I get it

I hope my explanation / question didn't come across as condescending. If it did, I apologise. I'm aware the differences in this area are subtle and I like to periodically check if my explanations make sense, so I don't drift too far

Comment by Ann-Marie Breaux (Inactive) [ 21/Nov/19 ]

No, not at all - just answering and in meetings at the same time, so keeping it short!

Comment by Ann-Marie Breaux (Inactive) [ 27/Nov/19 ]

HI Marc Johnson What do you think are the chances of finishing this feature in Q4? I see several in progress, but a couple still open. Please let me know if I'll need to split it and spill some of it over into Q1. Thank you.

cc: Cate Boerema Charlotte Whitt

Comment by Marc Johnson [ 27/Nov/19 ]

Ann-Marie Breaux I think you are best off asking Matt Reno as he is primarily working on this feature

Comment by Ann-Marie Breaux (Inactive) [ 27/Nov/19 ]

No problem! Matt Reno What do you think are the chances of finishing this feature in Q4? I see several in progress, but a couple still open. Please let me know if I'll need to split it and spill some of it over into Q1. Thank you.

Comment by Matt Reno [ 27/Nov/19 ]

Ann-Marie Breaux I expect the storage stories for HRID to be completed by and included in Q4. There is some minor work that will need to be done in the BL module, but I don't think that significantly blocks other stories. At this point, the HRID settings UI could be hooked up and we should be seeing generated HRIDs for instances and holdings. I just checked folio-snapshot and can confirm instance HRIDs and holdings HRIDs, but no item HRIDs in inventory. Item HRID work should be completed today.

Comment by Ann-Marie Breaux (Inactive) [ 02/Dec/19 ]

Matt Reno Bohdan Suprun We have mocked backend data for the Inventory HRID settings page, until the backend work was completed. That means any changes to the start number or prefix are not being saved. Can we unmock that now, or should we wait until the HRID backend work is completed in Data Import and MARCcat as well? We have a small story UIIN-752 Closed that we'll schedule if you confirm it's OK. Or we'll wait until after the corresponding Data Import/MARCcat HRID work

cc: Marc Johnson Charlotte Whitt Taras Tkachenko Mariia Aloshyna

Comment by Marc Johnson [ 02/Dec/19 ]

Ann-Marie Breaux That UI depends upon the work in MODINVSTOR-363 Closed which has been completed, the mocked data can be replaced.

I don't think that should wait until the data import and MARCCat work is done, even if those areas don't use this information at that point.

Comment by Ann-Marie Breaux (Inactive) [ 02/Dec/19 ]

Perfect - thanks, Marc Johnson

Comment by Ann-Marie Breaux (Inactive) [ 10/Dec/19 ]

Hi Marc Johnson Bohdan Suprun Matt Reno

I've hunted but I can't find the comment or Slack message where the issue was raised last week of whether 8 digits for the HRID was long enough.

Right now, it's looking like we'll need to expand from 8 digits to 10. I'll write the UI update story later tonight, and Folijet will take care of it. For the backend work, do you expect that will be much effort, or is it basically a simple update? I'm happy to draft the stories, if you confirm whether they should be MODINV or MODINVSTOR. And if it's a simple update, do you think it's something that Core-fxn can pick up in next sprint and can be included in a Q4 bugfix release?

Thank you for your help!

cc: Charlotte Whitt Cate Boerema

Comment by Ann-Marie Breaux (Inactive) [ 10/Dec/19 ]

Changing this feature back to In progress to accommodate the increase in number of digits

Comment by Bohdan Suprun (Inactive) [ 11/Dec/19 ]

Hi Ann-Marie Breaux,

I think it is about 3 story points. We mostly have to update our DB sequence: set the new max value and change the sequence return type to bigint.

The story should be for MODINVSTOR. We don't have to change anything in MODINV.

But maybe Matt Reno can better clarify this, he was mostly working on this feature.

Comment by Ann-Marie Breaux (Inactive) [ 26/Dec/19 ]

Re-closed this feature

Generated at Fri Feb 09 00:21:15 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.