Page Comparison

Development updates

• Development status
• Development team scrum board

• Buk edit  - Morning Glory Bugfest tickets:

• Bulk Edit Morning Glory Known issues (as of August 18, 2022):
  • UIBULKED-122 - Implement logic for record counts when items bulk edit is triggered by holdings id
  • MODEXPW-203 - "Fail to upload file" error with large amount of Users barcodes
  • MODEXPW-202 -Incorrect number of records changed on the confirmation banner  
• Performance tests results:
  • Item records: Bulk Edit Items App report [Morning Glory]
  • User records: Bulk Edit report for Users App [Morning Glory]
  •  Combined tests: Combined test for data export worker (Morning Glory)

Let's dive into the Morning Glory bugfest tickets. As you see, the list is significantly shorter than last time; we have three remaining stories. Two of them are in code review. One is in progress. All of them are related to making MOD data export horizontal scaling. And the issue is not only for Bulk Edit but also for other modules that use it for exporting reports, e.g., EDIFACT exports, eHoldings exports, Circulation log, and Bursar information. All of them are using this module. So the work we put in here will benefit others as well. The other good news is that we were able to resolve the issue where once the Bulk Edit completes, the confirmation banner needed to be manually refreshed to provide the number of records affected or updated. This is done. We are not able to recreate it anymore. So it's good news. The links to the performance tests are here, they have not changed since the last meeting, and we will definitely be doing more tests once those three stories regarding horizontal scaling are addressed, and we are sure how the system behaves when we have more than one module available. And the questions?

Erin  08:16
So the other two things that are listed as known issues, it looks like we're going to talk about one of them, the other is going to be worked in no Nolana?

Magda  08:25
So the first, 122, will be worked on in Nolana. 

• UIBULKED-122 - Implement logic for record counts when items bulk edit is triggered by holdings id

The second one, 203, is in progress now. The developer is looking into it. If ,we find a solution for this and we can squeeze it into Morning Glory then we will include it there. The issue occurs when you try to upload more than 3000 user records, which if this is the case and we we are not able to fix it in Morning Glory, I still feel it's okay if we move it to Nolana. I will know more about that at our next meeting.

• MODEXPW-203 - "Fail to upload file" error with large amount of Users barcodes

In terms of the development status, if we look at the SCRAM board, in addition to the bugs that we are addressing from bugfest, the team is also working on the backend functionality for Nolana so that UI work can start in the sprint that started yesterday. So we will be addressing issues in Nolana. The links are here if anyone wants to review them. if you have questions please let me know.

Expected behavior for: 

• UIBULKED-122 - Implement logic for record counts when items bulk edit is triggered by holdings id

Existing behavior: 

Magda  10:30
So I would like to move to the expected behavior for the bulk edits-122. I have slides here and I can also demonstrate the behavior if this is not clear. So what is happening right now, is that if we upload a file with 25 holding records, and out of those 25 Holdings, only 20 holdings find a match, there are 5 errors with no matches. So does this result screen make sense?

Erin  13:03

So what can we control here? Can we add a blue eye and show information? What are we able to do to try and provide clarity?

Magda  13:22
Just tell me what you think would make sense. And then, I can discuss this with the developers.

Erin  13:33
Lita chimed in the chat, and I definitely think more people should talk. But you know, that's valid, right? Y

Lita  13:43
This is a case we are used to seeing when you're working with holding IDs. And in this case, you want to edit all the items associated with those holding IDs. Some of those holdings are going to have more than one item.

Magda  14:02
Yes. And we will have the same behavior if we start, for example, editing holdings by providing instance IDs. There are many other use cases for this. So I just want us to devise a solution or approach that would make sense so that it will not confuse the user. Bob has his hand raised.

Bob 14:34
Yeah, I just want to be sure I understand the confusion. Is it because the user knows they loaded 20 and see the number 24?

Magda  14:45 - Magda clarified the issue with the displayed results again.

Erin  16:27
So, my first suggestion is, rather than using the word records, can we use the name of the identifier? Or say the name of the thing we're talking about, e.g., 24 item records, 24 of 24 item records matched, and five holdings UU IDs have errors.

Lita  17:01
I don't really have a problem with this top screen because I can see here in the column that says barcode if I have a barcode, I'm on an item. I would start to worry if I had fewer numbers than I submitted.

Magda  17:25
This may also happen because if out of those 20 holdings, there were some holdings that did not have associated items, then you may end up with 18 items, for example.

Erin  17:50
So I think adding the name of the record type in front of "record," and the type of the identifier in front of "error" would help.

Magda  18:01
Okay, so the top level would be 24 items records match. And on the errors, we will have 29 entries, and I need to investigate how we got to the number 29, to be honest, because 29 is nowhere to be found here. But this is the bug on the technical side.

Erin 18:55
24 item records matched. And then you can say five UID errors. Or, yeah, I don't know the best way to handle plural versus singular there. I'm sure there's guidance somewhere about it.

Magda 19:15
Would it be okay as holdings identifiers errors?

Erin  19:21
I thought it would be easy just to use the name of the identifier, but I don't have a strong opinion about it. If anybody else does, please speak up.

Bob  19:34
On the left, you're using the term holdings UU IDs.

Magda  19:38
So would you like to preserve this naming? Correct?

Erin 19:44
I just think it would make sense in context. It looks like Christie has her hand up.

Christie  19:56
My question would be, are the errors always going to be related to the holdings UU IDs because I can imagine that there is a holding with an item, and there was an error with the item?

Magda  20:13
You will see all the errors that occurred while matching the provided IDs.

Christie  20:28
So it will only be the holding UUIDs there?

Magda  20:35
In this case, the errors that are here will be IDS you have provided that don't match.

Chrisie  20:44
So my other question is, what happens if you have a holdings ID and it's got 10 items, but only nine items show up because one of them had an error? Would you even know about that?

Magda  21:09
This is a very good question. And why do you think would not? Why do you think that one of those would not show up?

Christie  21:25
That's a good question.

Erin  21:27
All that's happening here is you're giving a set of holdings record IDs, and then it should just return all of the item records that are on these holding UUIDs.

Magda 21:40
But this is a good question. So, for example, if, let's say, one of the item records has some invalid data that prevents rendering it in the UI. Christy, is this the scenario you have in mind?

Christie 22:02
I didn't really have a specific scenario in mind. I was thinking about this from a quality control standpoint. How will I confirm that I'm changing everything I expected to change? And, looking at the errors, I would be really surprised if we'd never have that happen. And this scenario you gave is actually a really good one because we're finding that some of the data we migrated with is just not standing the test of time as we migrate from one version to another. So I think it's perfectly reasonable that we have items out there with data that will go stale and no longer be valid in the system or is corrupted for some reason. And you know, what happens in that scenario? And I would expect it to show in errors that, you know, there was one record that we couldn't pull for whatever reason.

Magda  23:16
So, in this case, let's say we have one item that did not show up. So we would add here a barcode and the comment, data corrupted. And then, you will need to know which is the value for holdings and which is the value.

Erin  23:44
In Christie's scenario, you might not even get a barcode back.

Magda  23:49
But you will have the UUID.

Erin  23:56
My question is, how does Bulk Edit know how many items it expects to get?

Magda  24:09
It doesn't know. It just gives you this is what it found for what it was given. We can track the errors that occur in the process. It will be kind of difficult to recreate this issue with the data. But let's come back to this at some point.

Bob  24:47
I just have a quick question. So what's happening down below under the errors is a simple match on the UID in the spreadsheet to the UUIDs in the system. So no items are really being checked. It's just a simple comparison. So, it doesn't come into play regarding this error message down here, right?

Christie 25:22
It may come into play above when we're trying to display the items if there are five items attached to the holdings and you might get an error because one record is corrupted or whatever.

Magda  25:48
When the user saves all matches locally, if there is a problem with one of those records, that may cause a problem down the road. I would like to table this and come back to this conversation. Let me think about this. For now, obviously, we will not handle this case; the record will be excluded unless the problem is such that will prevent users from saving the matches found. I will come back to this. But this was a good point, Christie.

Bob  27:23
Can I just add one thing I thought of in terms of the messages about the record numbers in the result? It just occurred to me that a single box with multiple lines describing the results in one place might read better. Reporting the results of every type of record that returned and the errors, all in one place, so you're not scanning in different places to make sense of the numbers. Just one idea.

Magda  28:36
Okay, we can talk about this too. My understanding is that if everything goes fine, if there are no errors, and you see this top 24, this is all you really need. And when there are errors, you have them in one place, followed by what went wrong. But I also put this on the list of things to discuss how to make it a little bit more prominent or the reporting more prominent. Are we any comments in the chat that I need to discuss before we move to the next part?

Lita in Chat: When we download a CSV of the preview file, do we only get the chosen columns or all the columns?

Magda 29:36
We get everything in CSV format, whatever is in the item record.

Magda
I would like to start with a few questions that were reoccurring. Several responses to the question about call number display in the preview and confirmation screen indicated a preference to see each call number prefixes and suffixes elements in separate columns. And I would like to understand what the use cases are. Why do you think it should be separate? I just want to make sure I understand.

Jackie
Well, one thing is that that's how you can see if, for example, somebody put the prefix in the call number field instead of a prefix field. And if you have them all together, you can't actually see

Magda
Yeah, it makes sense.

Erin
That's an excellent example.

Magda
This is a good point. If you combine everything, you will never know what is the part of the call number and what is the part of the prefix.

Sara
I think the examples are exactly right. If I need to do a cleanup job, then the call number prefix, etc., need to be separated.

Bob
So are we talking about just this preview screen of 10 records? Or are we talking about the download file?

Magda 
The download file will have everything and will be structured like the record. So you will have everything separately. The prefix call number and suffix will be done separately. But we are talking about preview, we are talking about the are you sure form, and we are talking about their confirmation and all those places when those records are being displayed.

Magda
The other question about the preview, are you sure form and the configuration screen is someone mentioned (it was Duke) the need to see the holding statements and holdings notes.

Leeda
So I checked in with my leader, who helped me fill out this survey. And she said both types of data would be useful for analyzing the scope, ensuring accuracy, and troubleshooting when needed. But again, since we're just talking about a 10 to 20-item preview screen. We're fine if we can get to it in the CSV file.

Magda
My concern is that if we add holding statements and notes, I don't know how lengthy those notes can be.

Leeda
It was alright for concatenating the summary statement, and then for the holding notes, I think just the presence or absence of a note would be fine. Again, it's just a case of when we get ready to run a bulk edit, inevitably, there can be records in there that we didn't really mean to hit.

Magda
Leeda, do you want those to be columns in the preview?

Leeda 
The feedback I got was yes, but I'm not sure certain that the that they understand that you can capture that in the CSV. I will get back to you on that one. If I had to make an executive decision, I would say it's not absolutely necessary. But I do like the idea if we can have a presence or absence. Kind of like a yes, no, and then download the whole thing to find out what's there.

Magda 
What about electronic access? This was also mentioned. Electronic access has several fields and is long. It would make the display pretty difficult. So how do we concatenate, or do we leave them out?

Leeda 
I could see again that it would be more for ensuring that you're touching the right records. Because I know at Duke, we have a lot of titles where we hold the print and the electronic. And we don't want to touch the wrong title in that case.

Sara
I would not rely on the Preview or 10 records. I would have to download them to ensure that I wasn't changing something I did not intend to change. If I understand you correctly, everything will be the download file. Do I understand that correctly?

Magda  14:12  
That's correct. All the data from the Holdings Record will be in the CSV file.

Sara
So, in this case of only being able to see 10, it's unclear how this will help. Maybe then it's not worth putting so much time and effort into getting all these fields exactly right or including too many or too few.

Magda
And my thinking runs along the same lines as well as Sarah's. To make you feel a little bit better, we plan to change the preview to allow pagination of the results, etc. But this is not happening in the Nolana. It will be coming later once we start building the query tool. Because when you submit the list of identifiers, we can assume that those identifiers were based on a query or you have the items in front of you and you scanned the barcode. So, you know what dataset you are acting on. Once we get to running queries, then the top 10 is not going to do anything for you because you will need to see the query results. This is when we will be making changes to the preview. But this is not in Nolana. We may start talking about this in Orchid and then follow up in Poppy. So it is something in our pipeline. It is just not available here. So going back, do I understand what you're saying is that since the preview is only an abbreviated preview, you will need to go to CSV anyway. So we can be a little bit less verbose in defining what fields need the need in the preview screen. Is this correct?

Leeda
Yeah, I think so.

Sara
I just wanted to follow up briefly on something that I think it's worth noting. Just because I have a stack of 20 books and can scan their barcodes into a file does not necessarily mean I know what is in their records. I think this is worth noting because this happens to me all the time. For example, I get a list of barcodes that need to go on display. I'm changing the temporary location. A pop-up note for the circulation people is supposed to be added. But sometimes it turns out there's already information there. And so then, if I do a bulk edit without checking first whether or not that field already has information, which I cannot tell from just having the physical book in my hand, I will override that information. So we should not assume that I know what is in the records just because we have a stack of physical items. efforts.

Magda
That's a very good point because I didn't think about that.

Magda
So, I would like to look at the specific questions I have from the survey responses. So again, we are talking about identifiers here. My question is about the OCLC number. Is there an OCLC number on the holdings or just on the instance?

Sara
No, it is only the instance level from the MARC source.

Magda
Okay. So I think someone who suggested OCLC understood that when we submit the list of OCLC numbers to identify instances and then from instances go to holdings, this is something that we will be addressing through the query. So I will table this for now, as well.

Magda
The next question was about the columns. I see that we need to put a call number with all the elements in separate columns, including their call number type. So we already talked about the holding statement and electronic access. My concern is if we add electronic access to the preview, the rendering will be difficult at this point. And then the following questions are similar. One of the respondents suggested letting the user manipulate the order. This is the long-term goal, but it will not happen for Nolana. So, for now, we need to have a predefined list of columns.

Magda
And the last question was about the columns in the action menu that are not selected.

Unknown Speaker  23:18  
How should statistical codes be separated because this is a multiple-entry field? Should we use a comma-separated list or a new line?

Jennifer
It looks like, for notes, they're using pipes. So that's probably fine for that two.

Magda
And call numbers we actually covered. We also have a response suggesting staff suppress. There is no staff suppression on the holdings. So I believe the respondent was referring to the instance level, which will again be drilling down to the holdings record that will be covered by the query. Electronic access, as I mentioned, worries in terms of rendering it. Is the URL the only valid field, or do you need the type of electronic access as well?

Christie
I think that in our use case, we would want to see everything but the URL because the URL is unique. And maybe the CSV is enough for this. We frequently put our platform names as the link text in the URL. And we're frequently normalizing the material specified statements. So for us, it would be materials specified statements, link text, notes, and the type.

Magda  25:53  
And I understand that you may want to change that. In Nolana, we will only support changes to temporary and permanent holdings locations. The other fields will come later. But do you want to see this information rendered in the column on the preview screen?

Christie
I don't remember whether we asked for this in the survey. I don't know that this is a high priority for us. I know we have a use case for changing that, which is why I spoke up.

Magda
And I understand that these fields will be supported in the in-app approach for Holdings Record edits, but I'm not sure we need to display this data in the columns.

Sara
If this is just about the confirmation screen, I think it was Leeda who earlier said that it would be worthwhile having just kind of something indicating if it is present or not present just to have that added knowledge that you didn't get the ebook rather than the print book. Otherwise, I don't think the confirmation screen needs all this detail. In the CSV, it does need to be there, including the URL string, which I have had to do bulk changes too frequently. There are definitely use cases for needing to change the URL in batch, but that can be in the CSV, obviously.

Magda
There was one mention of including associated order and purchase order lines. Is there any feedback on that?

Magda
That would be something more at the query level. I think the usefulness of the preview screen would be more to preview the columns that I was actually taking an action on. I'm starting to think about it more in that respect.

Magda
In chat, I see Christie asking about the ability to change part of a string. Yes, in Nolana, this is available to cover common use cases, for example, changes in the email domain. Find and replace functionality will be added. It will be available for other fields down the road. Some fields, like location and date, will not be supported in Find/Replace, as it is not applicable. Is that a limitation?

Christie
That is a limitation, but there is a process.

Magda

We are at the top of the hour. See you all next time.