2022-8-23 Bulk Edit Working Group Meeting Notes

Development updates

• Development status
• Development team scrum board

• Buk edit  - Morning Glory Bugfest tickets:

• Bulk Edit Morning Glory Known issues (as of August 18, 2022):
  • UIBULKED-122 - Implement logic for record counts when items bulk edit is triggered by holdings id
  • MODEXPW-203 - "Fail to upload file" error with large amount of Users barcodes
  • MODEXPW-202 -Incorrect number of records changed on the confirmation banner  
• Performance tests results:
  • Item records: Bulk Edit Items App report [Morning Glory]
  • User records: Bulk Edit report for Users App [Morning Glory]
  •  Combined tests: Combined test for data export worker (Morning Glory)

Let's dive into the Morning Glory bugfest tickets. As you see, the list is significantly shorter than last time; we have three remaining stories. Two of them are in code review. One is in progress. All of them are related to making MOD data export horizontal scaling. And the issue is not only for Bulk Edit but also for other modules that use it for exporting reports, e.g., EDIFACT exports, eHoldings exports, Circulation log, and Bursar information. All of them are using this module. So the work we put in here will benefit others as well. The other good news is that we were able to resolve the issue where once the Bulk Edit completes, the confirmation banner needed to be manually refreshed to provide the number of records affected or updated. This is done. We are not able to recreate it anymore. So it's good news. The links to the performance tests are here, they have not changed since the last meeting, and we will definitely be doing more tests once those three stories regarding horizontal scaling are addressed, and we are sure how the system behaves when we have more than one module available. And the questions?

Erin  08:16
So the other two things that are listed as known issues, it looks like we're going to talk about one of them, the other is going to be worked in no Nolana?

Magda  08:25
So the first, 122, will be worked on in Nolana. 

• UIBULKED-122 - Implement logic for record counts when items bulk edit is triggered by holdings id

The second one, 203, is in progress now. The developer is looking into it. If ,we find a solution for this and we can squeeze it into Morning Glory then we will include it there. The issue occurs when you try to upload more than 3000 user records, which if this is the case and we we are not able to fix it in Morning Glory, I still feel it's okay if we move it to Nolana. I will know more about that at our next meeting.

• MODEXPW-203 - "Fail to upload file" error with large amount of Users barcodes

In terms of the development status, if we look at the SCRAM board, in addition to the bugs that we are addressing from bugfest, the team is also working on the backend functionality for Nolana so that UI work can start in the sprint that started yesterday. So we will be addressing issues in Nolana. The links are here if anyone wants to review them. if you have questions please let me know.

Expected behavior for: 

• UIBULKED-122 - Implement logic for record counts when items bulk edit is triggered by holdings id

Existing behavior: 

Magda  10:30
So I would like to move to the expected behavior for the bulk edits-122. I have slides here and I can also demonstrate the behavior if this is not clear. So what is happening right now, is that if we upload a file with 25 holding records, and out of those 25 Holdings, only 20 holdings find a match, there are 5 errors with no matches. So does this result screen make sense?

Erin  13:03

So what can we control here? Can we add a blue eye and show information? What are we able to do to try and provide clarity?

Magda  13:22
Just tell me what you think would make sense. And then, I can discuss this with the developers.

Erin  13:33
Lita chimed in the chat, and I definitely think more people should talk. But you know, that's valid, right? Y

Lita  13:43
This is a case we are used to seeing when you're working with holding IDs. And in this case, you want to edit all the items associated with those holding IDs. Some of those holdings are going to have more than one item.

Magda  14:02
Yes. And we will have the same behavior if we start, for example, editing holdings by providing instance IDs. There are many other use cases for this. So I just want us to devise a solution or approach that would make sense so that it will not confuse the user. Bob has his hand raised.

Bob 14:34
Yeah, I just want to be sure I understand the confusion. Is it because the user knows they loaded 20 and see the number 24?

Magda  14:45 - Magda clarified the issue with the displayed results again.

Erin  16:27
So, my first suggestion is, rather than using the word records, can we use the name of the identifier? Or say the name of the thing we're talking about, e.g., 24 item records, 24 or 24 item records matched, and five holdings UU IDs errored, or errors.

Lita  17:01
I don't really have a problem with this top screen, because I can see here in the column that says barcode if I have a barcode, I'm on an item. I would start to worry is if I had fewer numbers than I submitted.

Magda  17:25
This may happen as well, because if out of those 20 holdings, there were some holdings that did not have associated items, then you may end up say, 18 items, for example.

Erin  17:50
So I think adding the name of the record type in front of "record," and the type of the identifier in front of "error" would help.

Magda  18:01
Okay, so the top level would be 24 items records match. And on the errors, we will have 29 entries, and I need to investigate how we got to the number 29, to be honest because 29 is nowhere to be found here. But this is the bug on the technical side.

Erin 18:55
24 item records matched. And then you can say five UID errors. Or, yeah, I don't know the best way to handle plural versus singular there. I'm sure there's guidance somewhere about it.

Magda 19:15
Would it be okay as holdings identifiers errors?

Rein  19:21
I thought it would be easy just to use the name of the identifier, but I don't have a strong opinion about it. If anybody else does, please speak up.

Bob  19:34
On the left, you're using the term holdings UU IDs.

Magda  19:38
So you would like to preserve this naming? Correct?

Erin 19:44
I just think it would make sense in context. It looks like Christie has her hand up.

Christie  19:56
My question would be, are the errors always going to be related to the holdings UU IDs, because I can imagine that there is a holding with an item, and there was an error with the item?

Magda  20:13
You will see all the errors that occurred while matching the provided IDs.

Christie  20:28
So it will only be the holding UUIDs there?

Magda  20:35
In this case, the errors that are here will be IDS you have provided that don't match.

Chrisie  20:44
So my other question is, what happens if you have a holdings ID and it's got 10 items, but only nine items show up because one of them had an error? Would you even know about that?

Magda  21:09
This is a very good question. And why do you think would not? Why do you think that one of those would not show up?

Christie  21:25
That's a good question.

Erin  21:27
All that's happening here is you're giving a set of holdings record IDs, and then it should just return all of the item records that are on these holding UUIDs.

Magda 21:40
But this is a good question. So, for example, if, let's say, one of the item records, has some invalid data that prevents rendering it in the UI. Christy, is this the scenario you have in mind?

Christie 22:02
I didn't really have a specific scenario in mind. I was thinking about this from a quality control standpoint. How will I confirm that I'm changing everything I expected to change? And, looking at the errors, I would be really surprised if we'd never have that happen. And this scenario you gave is actually a really good one because we're finding that some of the data we migrated with is just not standing the test of time as we migrate from one version to another. So I think it's perfectly reasonable that we have items out there with data that will go stale and no longer be valid in the system or is corrupted for some reason. And you know, what happens in that scenario? And I would expect it to show in errors that, you know, there was one record that we couldn't pull for whatever reason.

Magda  23:16
So, in this case, let's say we have one item that did not show up. So we would add here a barcode and the comment, data corrupted. And then, you will need to know which is the value for holdings and which is the value.

Unknown Speaker  23:44
In Christie's scenario, you might not even get a barcode back.

Magda  23:49
But you will have the UUID.

Erin  23:56
My question is, how does Bulk Edit know how many items it expects to get?

Magda  24:09
It doesn't know. It just gives you this is what it found for what it was given. We can track the errors that occur in the process. It will be kind of difficult to recreate this issue with the data. But let's come back to this at some point.

Bob  24:47
I just have a quick question. So what's happening down below under the errors is a simple match on the UID in the spreadsheet to the UUIDs in the system. So no items are really being checked. It's just a simple comparison. So, it doesn't come into play regarding this error message down here, right?

Unknown Speaker  25:19
Yes, but

Unknown Speaker  25:22
it may come into play up above when we're trying to display the items. But it won't even do that, because it's it didn't, I'm sorry, yeah, so it makes the match. And there are five items attached to the holdings. And he tries to display it in the upper section here; you might get an error, right? If it's corrupted or whatever way at all.

Magda  25:48
When the user saves all matches locally, if there is a problem with one of those records, that may cause a problem down the road. I would like to table this and come back to this conversation. Let me think about this. For now, obviously, we will not handle this case; the record will be excluded unless the problem is such that will prevent users from saving the matches found. I will come back to this. But this was a good point, Christie.

Bob  27:23
Can I just add one thing I thought of in terms of the messages about the record numbers in the result? It just occurred to me that a single box with multiple lines describing the results in one place might read better. Reporting the results of every type of record that returned and the errors, all in one place so you're not scanning in different places to make sense of the numbers. Just one idea.

Magda  28:36
Okay, we can talk about this too. My understanding is that if everything goes fine, if there are no errors, and you see this top 24, this is all you really need. And when there are errors, you have them in one place, followed by what went wrong. But I put this also on the list of things to discuss how to make it a little bit more prominent or the reporting more prominent. Are we any comments in the chat that I need to discuss before we move to the next part?

Lita in Chat: When we download a CSV of the preview file, do we get only the columns chosen or all the columns?

Magda 29:36
We get everything in CSV format, whatever is in the item record.