| Nolana features review | UXPROD-3230 - Bulk delete user records UXPROD-3705 - Bulk Edit - User data - in app approach UXPROD-3704 - Bulk Edit - in app approach - holdings locations UXPROD-3706 - Bulk delete inventory item records UXPROD-3707 - Bulk edit - inventory items - csv approach - 10:14:32 Magda: The next items that I would like to cover are the features that I would like us to work on in Nolana. However, I have some questions for this group and also to provide you with additional information.
- 10:15:02 Magda: Let's start with deleting user records. This is something we discussed, I think partially last meeting. I followed up with the User SIF and also with some subject matter experts that were not present during the meeting. And this is what we came up with,
UXPROD-3230
-
Getting issue details...
STATUS
.
- 10:16:20 Magda: So the first thing that I will start is what is outside the scope. Soft delete is outside the scope. The reason for this is when we do the bulk edit functionality, we use the existing API's. and soft delete is not implemented on user records. It's not even implemented anywhere in yet in FOLIO. It would actually require a platform-wide conversation. So we would implement the bulk edit delete the way it is for single records in user. This means references to circulation transactions will be handled because this is what is happening on the single record delete. However, all the shortcomings of the implementation will be present as well.
- 10:17:26 Magda: So, for example, if we delete a user that is referenced somewhere in the metadata, let's say in inventory the metadata says this record was updated by Magda Zacharska. Then, let's say we deleted Magda's record from Users. Then, when you open the inventory record, you will see that the record was updated by an "unknown" user.
- 10:18:02 Erin: That was just fixed by Dennis Bridges team, I think. I'll send you the Jira Magda.
- 10:18:25 Magda: I am aware of this but do send me the Jira. I doubt this is how it was implemented.
- 10:18:27 Erin: There was a gap in how it was done.
- 10:18:27 Magda: The other problem with the existing implementation of the delete users is that it deletes the user record, but it does not delete references from the permissions table. So you will have abandoned records. It does not cause any harm besides growing data, you have the entries in the permission tables that refer to non-existing users.
- 10:19:20 Erin: So what remains in the permissions table is the UUID for the user, but you don't have the name anymore. If we deleted your account, whatever UUID you had would remain in the permissions users area. But you wouldn't be able to look up the UUID to find the user's name.
- 10:20:33 Sara: Okay, but would I be able to use that UUID without a name and see what changes did this UUID make in inventory, for example?
- 10:20:49 Erin: I don't know.
- Sara: Because that could be of interest if I have a user who leaves and later we discover they made lots of errors, and we're trying to track down all the errors to fix them. Then it could be irrespective of the name irrelevant but the UUID is still everywhere.
- 10:21:16 Magda: I think I think this is a good point. So does it mean that this is a desirable behavior?
- 10:21:29 Sara: I've definitely had to use the username of somebody who's done lots of work, and I needed to find all their work to verify that it was done correctly after they left.
- 10:22:02 Erin: But what you should do in that scenario is deactivate that user, but keep their record until you have validated everything and then delete the user. The answer isn't to trace the UUID.
- 10:22:15 Sara: Well, maybe not. But maybe i'm not the one who deleted the user. And then later, we figure it out. I'm just wondering. And I'm wondering also in connection with acquisitions and this just came up. If my name was associated with an order because I requested that thing to be ordered, what happens there? Does it also turn into an unknown user?
- 10:22:48 Magda: I do believe that the orders are being covered.No, actually...
- 10:22:59: Erin: Requesting is one of the dependency checks that happens with the user deletes.
- 10:22:59 Sara: Not request in the sense of requests, but there's a field called or selector or requestor in the order where you can mark that somebody asks for something to be ordered.
- 10:23:27 Erin: That's just a straight-up text field that has no connection to a user record. So deleting a user wouldn't do anything to this field.
- 10:23:29: Sara: Okay, great thanks.
- 10:23:38 Magda: So I would like to move to another feature as well. And just a quick note. I'm meeting with the User SIG tomorrow again to discuss this implementation, and get their final blessing for the approach or not. In any case, for now, it is planned for Nolana. If rejected by the SIG, we will will discuss this probably later.
- 10:23:39 Magda: The next user feature is about the in-app approach for user records. As you recall, at the end of Morning Glory, user records can only be deleted through the CSV approach, meaning the user will download the CSV file with the matching records and make the change locally and upload the CSV. As you also may recall, there was a user testing done in the early months of this year, and it was stated by the community that the preferred way of doing bulk edit is in-app. To meet this expectation we will be adding this functionality for user records as well.
- 10:25:20 Magda: So at the end of the Nolana, the user records in the in-app approach will support changes to the patron group, expiration date, and the email address. Those fileds were not selected randomly. I reviewed all of the user records fields with the User Management SIG and we come up with priorities for the fields. All those that are marked as 0 "must-have," will be addressed right now, with the exception of permissions which are a separate record group and will be addressed in later releases.
- 10:26:46 Magda: And I do believe that in my out of scope, I specified that this is planted for the Orchid release, which is the next release after Nolana.
- 10:27:14 Magda: The next one on the list is the holding's location. And I think it's clear we need this and we did similar work for the item's location and is straightforward. This definitely would be implemented in the scope of Nolana.
- 10:27:37 Magda: The next one is deleting item records which makes me a little bit nervous. And I would like to hear from this group your take on this.
- 10:27:54 Erin: Deleting records just makes you nervous?
- 10:27:54 Magda: Yes.
- 10:28:04 Erin: I understand that. I definitely think it's needed. I think we would want to spend time talking to Metadata management and Charlotte.
- 10:28:14 Magda: So again, the bulk Delete uses the existing APIs. So whatever is done on the inventory side will be brought in...
- 10:28:30 Erin: Jennifer is commenting in the chat that right now Metadata Management is doing a lot of work on an instance deletion and gathering those use cases. Because we've implemented item delete already, then maybe there's not a ton of conversation or maybe the people here from Metadata Management can go back to that working group and say, hey, this is coming, is there anything you want us to make sure that we talk about or that you're concerned about? And so maybe that's enough. I guess I just feel like having that conversation might ameliorate any worries that we might be implementing functionality that isn't desired.
- 10:29:13 Magda: I know it is desired functionality because I have another spreadsheet that is based on use cases provided by the community and I will share this shortly as well. But my concern about the deletion of items right now is that the functionality and dependencies are handled in the UI, and I see John stated that this is true, meaning we have this on the Ui. That means if we call APIs we are deleting everything, causing a lot of damage, which we obviously don't want to do. If we need to implement this on the back end, then it significantly increases the scope of the work.
- 10:30:29 Erin: I see Sarah's hand up. I'm curious to hear what you think.
- 10:30:29 Sara: It's interesting because this morning this came up as a topic to me. We have rows and rows and rows of stack shelves that we want to weed, and we don't have the time right now physically take that on. On the other hand, if I my one instance record has 1,668 items attached to it, and that's just one of the things that we want to weed, if I wait till FOLIO that will be an individual item by item by item, deletion, right right now, Erin, right? And there is no way in inventory for me to just say, check all of these off and delete all of them, because I've just weed them. So I am debating for 10 to 20 such cases whether to do this in my old system where I can just do a total delete with one push of the button and just leave them on the shelves, and then just make myself a huge note, saying, don't forget to remove them from the shelves, and put them in the garbage at some point in the future. Do you see my dilemma like if I wait? What am I gonna do?
- 10:32:05 Erin: it's not a great answer. But it is pretty trivial to script it.
- 10:32:05 Magda: In fact, yes. But if you script this Erin this would be exactly the same case we are facing right now that everything will be deleted.
- 10:32:25 Erin: You have to check the dependencies manually. But then if you were absolutely ready to get rid of these 1,500 records that's pretty easy to do with, like a python script.
- 10:32:34 Sara: Yeah, through the APIs. So again, though, this all relies on the access to that, the ability to do that, and so forth.
- 10:32:49 Erin: Sure that's why I said it's not a great answer, but there is a way to do it.
- 10:32:53 Sara: But eventually I don't think this is how we want to have to do this.
- 10:33:03 Erin: And that's exactly what Magda is outlining in this Jira some of the challenges that we'll be involved in getting there. It looks like Jackie has her hand up.
- 10:33:17 Jackie: I do. I would say I'm on the other end of what Sarah is describing, and Jen can speak to this too. We were required to do a mass withdrawal project after implementation, and I don't know how difficult it was for the batch processing office to take care of the records that were so huge that we couldn't manually delete them. But we had staff sitting here deleting hundreds of records manually, and it was awful. If we can have a solution to that I would be very happy. I don't know if that's a constructive comment.
- 10:33:55 Magda: It is. And all the comments are constructive. I just see we need to implement the dependencies that are currently in the UI on the back end. I also believe that not all the dependencies that are implemented on the UI cover all use cases for deletion. So, this one requires additional research.
- 10:34:49 Bob: For us, deletion across the system, of various record types, is a very high priority. I have a hard time understanding why this isn't a very high priority, as in most current systems you can delete records. I mean this is just gonna be this is a big problem for us. We're always deleting things out of our current system.
- 10:35:10 Magda: This is really good. If you can add ranking to this ticket, that would be helpful, and I will follow up with the Metadata Management SIG to talk about the dependencies and what we need to implement to make it work. I see that there is a huge need for this.
- 10:35:47 Erin: So Nolana so far is bulk deleting inventory items, users in-app approach, and then is that it, or is there other stuff in Nolana?
- 10:36:06 Magda: This is Nolana.
- 10:36:13 Magda: The last one, bulk edit inventory items using the CSV approach. This is the most controversial from my point of view, and I would like to hear your feedback on this.
- 10:36:21 Magda: This is in draft, and the behavior will be exactly the same as we do with the user records right now. So we are identifying records for based on submitted identifiers, then saving records to the CSV file, the user makes the changes locally and then uploads the file, and the system updates every field specified in the CSV. The reason I think it is controversial is because of the dependencies. What we are going to do. We are going to get the CSV file and put the records as they are specified in the data. Obviously, there will be all these steps that are in the user records update when we are asking you to "are you sure" you want to do this? You can also revert your changes by resending the original file.
- 10:38:29 Erin: I have a question. Do item records have optimistic locking at this point?
- 10:38:43 Magda: They do. But I don't think optimistic looking will prevent someone to make a change in loan types, for example, that will affect the behavior, or will change the status for the record that is in order, for example, or something like that. The CSV approach relies heavily on the knowledge of people who do the bulk editing. We can limit access to this functionality through permission so only selected users will have access to it. But it's an extremely powerful tool. You can do whatever you want on the record. But you need to know the consequences of your actions.
- 10:39:31 Erin: Optimistic locking should stop it if like let's say, for example, I downloaded a 100 items and I was working on the CSV file, and during the time I was working on the CSV file 5 of those items were checked out. That change in the item status should increment the record version. So I should get an error when I post that record back.
- 10:39:56 Magda: Yes, so optimistic locking will work only if the change occurred in the meantime. But let's say the status is paged. You downloaded the file this morning, and in the afternoon the status is still page, and you are changing the status to available. Optimistic locking is not going to prevent you from doing it.
- 10:40:41 Erin: It Should. You are making another change.
- Magda: There was no change. This is another update. Optimistic locking is not looking at the values. Optimistic locking is only checking the version.
- Erin: I guess I would assume that the version that you have is like version 20 and when you downloaded it, the versions matched But then when the status changed in inventory, became version 21 and then at that point the versions don't match, and so I shouldn't be able to post that record back to inventory.
- 10:41:29 Magda: Why would they not match?
- Erin: Because of the change of the status in inventory while I was working on the CSV.
- Magda: I created a file this morning, and let's say the version of the record was 30. There were no changes in the record. The record was paged all the time. I make the changes locally, and a couple of hours later I'm: uploading my file. So, this is a new version, and overwriting the page with available. The version status changes to 31. Optimistic locking would work only if I had version 30, someone in the meantime, in the system changed the status to awaiting pickup, and so then that would change the version in optimistic locking, and my override would not work because I had the older version. But if the record has not changed in the meantime, nothing will prevent me from changing the record that was page to be available.
- 10:42:50 Erin: I see what you mean.
- Magda: I see a lot of conversation in chat, can someone speak up and say what they are typing.
- 10:43:07 Erin: There's only one real question in here it's from Sara who says that the in-app approach for item records is locations and statuses, and she's asking about whether in-app is supporting loan type for item records.
- 10:43:18 Magda: No. This is actually on my list of things to do because we were talking about loan types at some point and decided this is dangerous territory, and we don't want to venture there yet. But if we go with the CSV approach, the user will be able to change the loan type anyway. And when I look at my spreadsheet of the most frequently requested fields...what I did was go through all use cases that we had for bulk edit noting what changes were requested for each record type. When I look at the item record, the areas that were mentioned are: statistical codes, tags, discovery suppress, URLs, delete, status (item), call number types/prefix/copy number, and loan types.
10:44:49 Magda: I'm wondering if it would make sense, instead of investing time into the CSV approach, to add those missing fields to the Inventory in-app approach. - 10:45:50 Erin: So what you are saying is that doing the CSV approach next would address the things that are listed here.
10:45:59 Magda: Yes, but in addition to others, and I'm just concerned that we are opening a can of worms. 10:46:11 Erin: I mean I think you'll find that any library can come up with a use case for editing a field on the item record and need to do it in bulk. My concern would be that there are things that are inherited on the item record that you wouldn't want to necessarily edit because you don't wanna mess with business logic. Things like item effective location. You don't want to set that but if it's a field that is directly on the item. Why not be able to edit it. Magda: Do we have anyone from the MM SIG to chime in? Let me check the chat. Jen, I see your comments. Jen: My most recent chat comment was just agreeing that for reserves it is a whole process. We need to change the temporary location, and the temporary loan type, they can only do half of that. They're still gonna have to come to us and get it finished.
10:47:25 Magda: So basically you're saying the location without loan types doesn't really work. Jen: I mean the reserves are just such a common thing, it's you know it's like 2 or 3 times a year for multiple libraries we have to do it. - 10:47:47 Erin: There are absolutely use cases for just changing locations.
- Jen: Yes, they are just less...So, like 3 times per year, I'm gonna have to do this for 4 libraries, right? I can predict it.
- Magda: Jen, what are the implications of changing the loan type It's just a temporary one, So it's very low. Also, as long as we are not deleting, my feeling about a lot of these is you've given it a list of IDs, and you have told it to make the changes, if you change your mind, you can give it the list of IDs again and change it back. So for me, it feels low risk.
- 10:48:25 Erin: All that a loan type is gonna do is potentially influence a circ rule. If I change a loan type, It's not messing with data integrity in that sense you're not you know you're not risking losing information necessarily.
- Magda: So you are saying we should concentrate on temporary loan types?
Erin: You would want to be able to do both permanent and temporary. 10:49:06 Jen: The temporary just happens more often. I mean at least in my experience loan type doesn't change. Sometimes something goes into or out of a special collection and that'll be a field that changes. But that's a lot less often. Erin: But we have used cases at Duke, for example, making things non-circulating and circulating again, and we would use loan type for that. And you know if you're doing that for a year you're gonna change the permanent loan type. 10:49:39 Magda: How could change on the loan type change the item status? - 10:49:46 Erin: It doesn't But would you? If something is checked out, for example?
- Erin: You can change the loan type on a checked-out Item.
- Magda: And this is a design behavior?
- Erin: It could be. It doesn't influence the active loan. What would happen is, that if the person chose to renew the item, they might get a different circ rule. So you know, if I have an item where my loan type is a standard loan, and I need that thing to be a course reserve, and I change the loan type from standard loan to course reserve then that may mean that the patron cannot renew that item and that's desired right because I want that thing to come back. So I mean there's something people need to understand the implication. But it's not hurting things there and you don't need to build in a particular guard rail In my opinion. Unfortunately, I have to drop off for my next meeting.
- 10:50:49 Magda: And I see in chat that Sara is in agreement with Erin's suggestion.
- 10:51:04 Sara: Sometimes check-in/out notes go hand in hand with these types of things. So, for example, we do a lot of displays, and so we have the temporary location that gets changed to "on display" and that may or may so require us to change the loan type if we're making that display non-circ for a period of time and then we usually have a check-in checkout note to alert staff to place the item back on display if it is dropped off at the counter, and because somebody's looked at it and then left it lying somewhere right and did not put it back on display. 10:52:08 It is really helpful, and that would have no dependencies. Just going back to what I think Erin was saying earlier, you know these things are just really items and why shouldn't we be able to change them.
- 10:52:38 Magda: I will follow up. We have two options. One. Add to the list of properties that are supported in the in-app approach or spend time on the CSV approach for items. How does this group feel would be more beneficial? Additional fields in the in-app approach, or all fields in CSV approach?
- Jen: My vote is for CSV.
- Sara: I can see the benefit of the CSV really, truly, Yes, and obviously there are a lot of things there that have to be done that way because the dependencies will just make it easier. But I think that there are also a lot of things that are irrelevant for dependencies like the check-in checkout, for example, then users who do not necessarily have the permissions to do the CSV version can take care of in-app, and I think there's a benefit to letting people do their job. And making everything more permission-heavy is a barrier.
- 10:54:39 Magda: So, Sarah, are you saying that you want to have a more in-app approach?
- 10:54:45 So I want everything right. I realize you have to prioritize. If certain things can easily like for example editing the check-in check-out note in-app then it would be really nice to have it sooner rather than later. But if everything has to go towards the CSV because that's what gets it done, then yes, of course, that's what...
- Magda: I have to admit that I am surprised that I get so many thumbs up for CSV because I was expecting I would get a push back from this group. So thank you very much for this feedback. In the long run, obviously, we would like to do the in-app approach to make it easier for the users. But we are facing the question of giving you something or giving you nothing and the CSV approach can be done faster but it puts a lot of responsibility on the users. However, as I said before, and I think Jenn repeated it, the changes that are being made with CSV can be reversed by submitting the file again. But I will follow up with the MM SIG one more time, and to make sure I was talking to them about the deletion of items and will mention that this group strongly supports the CSV approach.
- 10:57:06 Magda: We have 3 min left, and I would like to use the 3 mins to follow up on the discussion that we had in slack regarding the suppression. And this is what Jennifer mentioned, and I do believe that this should be implemented first on the inventory for one record. So once you suppress the record on the instance level it should automatically suppress the holdings and items. This is currently not implemented on the inventory. And my basic question to this group is, do you think this is just bulk edit specific functionality? Or is this functionality that you would like to have in inventory as well.
- 10:58:19 Jennifer: I think I'd like to see it on inventory. I mean if you are doing it at the instance level. it's kind of the top-level whether or not you have an underlying MARC SRS. I mean in that case you're suppressing everything from discovery. So it makes sense to suppress everything.
- Magda: I'll bring it to MM SI as well. And if this is implemente down so to the and i'm sick as well to to Get that.
10:58:56 Feedback, and if this is implemented on the inventory, then we can reuse it very easily for the bulk edit as well. 10:59:04 I do have one use case where this is not a good idea, though. 10:59:11 I am in the process of suppressing a bunch of records in a collection that we're moving, and 10:59:28 In some case it gives, some of the records will be unsuppressed in the future, because some were keeping in some or not. 10:59:36 , I'm. not going to know which records were which holdings and items records were suppressed previous to my having 6. 10:59:51 The instance record. So if I suppress the instance records in it, suppresses everything underneath it. 10:59:59 But then, when I unsuppress, what needs to get unsuppressed apps, the item and holdings level 11:00:12 This can get complicated. And how are you using those suppression? 11:00:18 Is this for or is this for something else? We have a collection that we are in the process of withdrawing a large percentage of the collection, but not all of it. 11:00:32 2 facilitate that right now we are simply suppressing everything in the collection. 11:00:41 And then we will go back and unsuppress. 11:00:44 The problem is I mean it's the problem i'm running into now is that some of the titles also have holdings outside of this collection. 11:00:58 So. In that case actually I can't suppress the instance record. 11:01:05 I can only suppress the holdings record, so maybe this is I mean, maybe this is a moot point, but I can see instances where you're going to suppress an instance record, and then you're going to turn it back on later. 11:01:18 On. but now everything underneath it's suppressed and you don't know what was previously suppressed, and what was suppressed as part of suppressing the instance and Yes, and I think there is also a question of 11:01:33 what will happen if you to the suppress suppress instance, you add a new holding holding soup. 11:01:40 Would you assume that it will be suppressed automatically as well or not? 11:01:48 And the same for items in so i'm doing this of course, by script right now. 11:01:54 , But no, I am not suppressing the holdings, because eventually most of those holdings will be withdrawn, and they will get suppressed as part of that process. 11:02:06 But when I go back and unsuppress the records that we're going to keep, I don't want to have to unsuppress the holdings records, so in my case I only want to suppress 11:02:18 let's see the instance records I'm. 11:02:23 Sorry I have to run to another meeting. we will believe we will continue this conversation. 11:02:30 The suppression conversation during that, and then sick meeting. If I get on their schedule this week or next week, we still have the permissions. 11:02:40 For the expert manager. We will talk about this, and and next time, and the Jennifer. 11:02:48 I saw your comment about the bug edits and looked at the item statuses and inventory. 11:02:58 I need to. There is this: There are a couple of statuses that are currently not supported by bucketed. 11:03:05 I will need to take a closer look at the implication of it and notification if needed. 11:03:10 We will add this in a Nolana as a supported, additionally supported status. 11:03:23 But this requires some work on my side first I also have another meeting. I'm. 11:03:25 All of the late for it have to run i'll see you in 2 weeks 30 min late, and then normally I will post this on the channel. 11:03:35 Thanks again for your feedback. have a good rest of your day, Thanksgiving. 11:03:40 Bye, |
