| Expected behavior for clearing values from permanent and temporary location. Should this be allowed? | Magda: Let's start with the standard development status. This is the FOLIO snapshot. We did some progress on the in-app approach. I am logged as a user with both permissions for the CSV approach and for the in-app approach. The user behavior has not changed. So let's concentrate on the in-app approach and the changes we have made so far. Magda: The user selects in-app (by selecting the "Inventory - Items" radio button). Magda: The dropdown menu is populated with the items-related values. So let's select the item barcodes. Magda: Choose the file to upload. And I will take item barcodes from my list of files. - Magda: It populated the screen with the items that meet the search criteria.
- Magda: We can add or remove columns. And we can start to bulk edit.
- Magda: As I mentioned, because I have two permissions, both CSV and in-app, I see these two options.
- Magda: So I am starting bulk edit in-app.
- Magda: Here are on the screen where we choose the options., one option, is permanent location, we have two action options, replace with and clear field.
- Magda: If we select replace with, we are picking the location using the location lookup and then save and close.
- Magda: We can also add another option, which will be a temporary location, and we can clear the field. So whatever is in the temporary location will be cleared.
- Magda: We, do you have any questions so far comments?
- Erin: It would be nice if the location could also have the inventory type of ahead modal in addition to the drop-down modal. People may find that that is easier, especially if they have just a few locations, so they don't need to like go hunting and, or maybe they know exactly what the location is called and they can just type it.
- Sara: The only problem is, fo us at the five colleges that is not, you know, if we have a dropdown, then that dropped down becomes ...
- Erin: I'm not saying remove it. I'm saying have both options.
- Sara: Okay, good. That would be very helpful.
- Magda: I will write a story for this. I'm not sure we can deliver it for Morning Glory. I will need to talk to developers about how much work it would take.
- Magda: Um, I have another question related to this, but I want to go through the whole process and then we will return to this screen.
- Magda: Okay. So we are confirming the changes and we get the response that 2 records will be changed if you saved them. Before we do the save, we do the download preview.
- Erin: So the previous says, there are no items. Is that a bug?
- Magda: This is a bug. You will see a list here. Actually this story, the form "are you sure?" is still in progress, so it is not committed. So it's a work in progress.
- Magda: So when we look at the preview, this is the preview of the changes. This will show you how the records will look like after we make the changes, the changes are not committed yet.
So as you see here, effective location and temporary location, this is before the changes are committed in the database. If you click cancel at this moment, you don't want to proceed, no changes will happen. But if you continue, you see that all the records will have main library and they will have no temporary location. And it will obviously affect the effective location. If I click keep editing, it will take me back to the screen and no changes were commuted.
- It shows that 2 records have been successfully changed. The error and reason for are displayed. I will need to investigate why we got this error, but at least we know we were not able to update this record. And also on the action menu, you can download the errors.
- Sara: I was just wondering would something not being able to be changed constitute an error? And how will that be counted? Because you have this interesting count thing going on. Let's say I have three records, in the meantime, somebody went in and manually already changed one to not temporary. And, so I'm asking all three to become temporary, to have everything be changed to their permanent location. But it can't actually affect the change on the one that was changed in the interim.
- Magda: It will overwrite the change. So let's say you have your list of identifiers, your scan the records and you are doing the bulk edit and you got distracted and do not proceed. In the meantime, someone goes and changes the records. Then you come back and you continue with your bulk edit. You're overwriting the change. Please note that this is the in-app approach. So it is not that you can upload the CSV file and you can start over. Those are the changes you are specifying in the UI.
- Sara: So I guess it still seemed though that I had this file that had three things.
- Magda: This is, this is just for your preview. In the in-app approach, you don't upload the file to trigger the update. The preview is only so you can see how the records will look before you commit the changes.
- Okay. Is it clear, Sara? If it's not clear, please ask more questions.
- Erin: What you will get from an error is you will get like what an API would return, which would be like what you would get if a pop-up showed up or something like that, as opposed to something where you might have over overwritten something or something like that, you're going to get an API message.
- Lloyd: Sarah's question made me wonder how is this is interacting with optimistic locking?
- Magda: Inventory obviously uses the optimistic locking. But what you have to keep in mind is you will be updating the latest version of the record. The bulk editor at this point, in the in-app application, does not have a snapshot of the data that was taken when you started the bulk edit process. So for this, I would like you to forget about the CSV approach. There is no CSV approach at this point for item records.
- Erin: So you pull the records, show me the preview, but you are not in any sort of edit function. It's just giving you the fields. And then when you do the actual bulk edit, it's gonna Repull the record and apply the changes?
- Thomas: In both app versions, couldn't you store the version number of the record that is supposed to be edited and check that before you do the bulk update? There's probably a locking magnetism with a version number. So the version of the number incremented by one, it would mean that there was a change in between.
- Magda: Let me walk again through the process and maybe this will clarify (At this point Magda had to restart FOLIO and then when she ran the file no records were updated due to errors. The main point here was that when there are errors reported from the app, no changes will happen to the records in FOLIO when the you save and close/commit).
- Magda: The question you had about storing versions and updating the version at this point in the in-app approach, you don't have a version of the, of the records. You don't know what you are changing. What we see this is the given moment. When you run the update. Why would you like to know what is version of the record you are updating? What would the use case for this.
- Thomas: I was thinking of a safety precaution. I understand that no changes have been staged yet. And until you hit that final screen where you have the option of downloading the Excel file, at that point, you have staged changes. They're not committed yet. They're staged. And the window between that, and when they're actually committed is very small. But there is still a chance someone else could go in and change a file between those steps, even though it was a very small time period. And my suggestion is that when you do start those that when you do that staging process if you store the version number of the current state and then when you commit the record, you could match them up to ensure that there were no changes in between those two steps. It is more like a safety net. I think it would be a good approach for both this, as well as the CVS approach, to catch changes to inventory between when you make that staging file and that commit file.
- Erin: And in all fairness, too, we have not uploaded 10,000 records, 15,000 records. So I do think. Until we know more about performance.
- Leesa: I was about to say the same thing, Erin, that window might not be small. You may decide what you're going to change. And like you said, it's 10,000 records. And depending upon how how fast that runs and what it might affect running and circulation or cataloging or somewhere you might choose to wait and run that overnight or later in the day.
- Magda: So I think when we are talking about items, and thousands of items, probably this will not be going into hours, but we'll see, we don't have the performance test for items conducted just yet only for users. But, tens of thousand shouldn't be that long, but if you go into hundreds of thousands, then we are adding time. I made a note for the version control. That definitely will be important once we start working on the CSV approach for item records. But, the CSV approach for item records will not be a part of Morning Glory, most likely. So I will come back with more questions.
- Erin: Jen is pointing out in the chat that data import doesn't have a safety mechanism right now. So it's not like bulk edit is not doing something that data import does.
- Magda: The difference is we are a new applications. So we we can be more agile and we can adjust accordingly.
- Sara: So again, so I get the in-app part. In my current system, I can do a query. It creates a file. It sits on my computer. I haven't downloaded anything. If it's an especially large one, I might not run it right at that moment, but on my schedule, I can prep it. Now I'm going to run it at 7:00 PM when I know things are slower or something. So, there's a window there of time where things can change. And so I guess my question before was more about going back to how it was nicely saying I've updated two records, there's an error. And so the third one wasn't updated. I was just trying to get a handle on what constitutes an error. What is and is not counted, so that the output can then be evaluated by me. So I was asking if it finds that in the interim, the change that I was trying to do on an item had already been handled, would it count that as an error? How does that counted? Because it's not affecting a change on that item because it's not finding anything to change.
- Magda: Sarah to make sure I understand your question. You run the bulk edit and then you encounter the error recorded in the errors part in the results. Then you correct the record so that the error will not occur anymore. And you are running again.
- Sara: No.
- Magda: I'm sorry. Let's say I do a query for everything that has a permanent location poetry room, and I do this at 11:00 AM. And so I get three things, let's say, for example. So I want to change it so that they're no longer a temporary location of poetry room, they should just go back to what they were. But some time lapses for whatever reason. And so I don't run it. And in the meantime, somebody has come in and and changed one item manually taking the temporary location off the record, which is fine. And so now two can be changed by the bulk edit. And the third one can't be changed because there's nothing to change.
- Bob: You can. It will just wipe out what's there, which is nothing.
- Sara: It will wipe out nothing with nothing?
- Madga/Bob: Yes.
- Sara: Okay.
- Sara: So I was just trying to ask whether nothing with nothing is an error?
- Magda: No.
- Bob: It doesn't have any way of knowing what the state was before.
- Erin: It would only ever be an error if a field was required and you tried to erase it. That would return an API error and you would see that message.
- Bob: I think if we had what Thomas was suggesting, then it would know that there had been a change and there would be an error. But, since it is not recording the state that it was in, it's just at the moment you click run, making the changes that you have asked for, regardless of what state was before or not. It's not keeping track, right. Magna?
- Magda: Yes, that's correct.
- Magda: So did I answer all questions? I have a couple of questions regarding locations because as you saw on the bulk edit screen, I can set the action for permanent location and temporary location to clear the field. Nothing prevents me from doing it. Is this desired behavior, or would you like to have some limitations on this?
- Thomas: This is only changing the locations on the item record. So the holding records would still have a permanent and temporary location?
- Magda: if it was populated before. But if you, for example, remove all the locations being stored on the item level item, and you don't have anything in the holdings...
- Erin: Holding will always have a permanent location that is a required field.
- Magda: Okay, then. Perfect. Thank you.
- Thomas: Then, if that is the case, I don't have an issue with clearing the permanent and temporary location off the item field since the holding record will always have a permanent location, there will always be an effective location.
- Erin: I think there are definitely use cases for being able to clear both.
- Magda: Awesome. Okay, great. Thank you. Because this was my concern that we may be a little bit too aggressive in this.
- Bob: Sorry to interrupt your flow. I just have a quick question about the UI behavior. If you choose to change the permanent location, then you went down to the next line. When you went to choose temporary location, you could still see permanent location. Is that something that could be updated when you choose something from the dropdown the second line would not present that as an option.
- Erin: I think the assumption Bob is making, which I think is probably true, but people should jump in if it's not, is that when you're doing this kind of work, you would only make one change on each field.
- Bob: I don't think I can think of a reason why you would have it twice. Right?
- Erin: I'm specifically talking about location. There may be because location is a pretty simplistic field. You know we're not talking about something that's storing an array of values or things like that.
- Bob: I mean, it's not a huge thing, but it just jumped out to me. And I saw that again, you know, we've already chosen it.
- Magda: You are absolutely right. Uh, I'll add to the story for this as well.
- Thomas: Would it make more sense or make it easier instead of having a dropdown box where you choose permanent item location, and temporary type item location, they're just static. And then the action could be selected action. And if nothing is selected, then nothing has done.
- Magda: Can you repeat? Because I think I missed it.
- Thomas: Can you, instead of having the options as dropdowns static options, have permanent locate item, location, temporary location, but then have the action dropdowns have a default to select action. And if it stays on select action, it just doesn't do anything. So you don't have to worry about disabling an option, whichever one they choose. It's just static cases across.
- Erin: I think that might be a little bit more confusing. I don't know, other people should chime in, but I think that what I like about this is that the UI looks like other UI options in FOLIO. Think about for example an electronic URL on an inventory record. Like, it looks like this, like you choose the value, you choose the thing, right? You just go drop-down by drop-down.
- Magda: I do believe this is the reason that we went this route to make it consistent. The list will obviously grow as we are. So by the end of the Morning Gory release, we will have permanent location, temporary location, and status, those three fields. And the list will grow as the application matures.
|
| - Magda: The next part of our agenda is performance tests. We conducted the performance tests for the users only.
- Magda: The feature for bulk edit performance will include established performance for Item locations and item statuses. However, this is not implemented yet. That's why it's blocked. We will be working on this slightly later, but I would like to walk you quickly through the results of the performance test and how we approach that. It's a lot of technical information and I added the link to the agenda If you want to spend some of your time looking at this. But, it's rather a self-exploratory what we have done. So we run basically three tests. One tested the results of updating 10 records of users. So one user updates 10 user records sent at a rate of one request per second and repeated 100 times.
- Magda: Times that are recorded in milliseconds. I was surprised to see that the logging took as much time almost as getting the link to download the matched records.
- Erin: Yeah, it has a bunch of business logic there.
- Magda: Keep in mind, this is for the CSV approach and for the user records The next type of tests where we again had one user updating 100 records of user data. And then it was done for one second and repeat 100 times. And obviously, the time increased when we add more. But I think the performance is still pretty good.
- Magda: And the last one was that we tested the same, one user updating a thousand records at once. And does it one per second and does it 100 times. For this amount of data, we did not encounter any problems when we were submitting the list of identifiers. The problems were once we ran the query. We got the error due to the performance of the query. For now, we will not do anything. We will be addressing it sometime later. If you scroll down, I think there should be a link to the query used. I will post in chate the query and And the user identifiers that we use in our tests. Do you have any questions?
- Jenn: Will they also provide some data on the overall system impact. That's what we see with data import. Like it may run quickly, it uses a great deal of energy from the system.
- Magda: So I think with the users, the impact will not be that bad, but for inventory, the impact of the bulk edit will definitely be affecting other areas. I will keep it in mind once we start working on the performance for inventory. But I will ask the developers to double check the impact of the users on the system.
- Thomas: Yeah. I would also check check-in check-out because that requires the user data quite a bit. So I could see updating a lot of records impacting that.
- Sara: In our current system when bulk editing is done in say inventory, that is my primary experience. It could be that in our system data import and bulk editing kind of happen within the same module. And that might be different in folio. But, if not, records are being impacted simultaneously potentially. So I can set up a big task with many, many, many records, and then there are other jobs that have higher priority and they can interrupt. You end up with a queue. But they're allowed to interrupt to do their thing. And then once that is done, mine can resume. And so this again could happen and there's a delay in the starting of something and finishing, but also overall system impact. And that's what you're talking about here. Will this be a possibility in FOLIO too? And do we have to somehow allow already for that or account for that?
- Magda: So you are asking if we can queue the jobs. So for example, if you are importing the data at a given moment, and then someone else does the bulk edit and both of them are taking a lot of resources. So you would like the import to complete first before we start doing the bulk edit, right?
- Sara: This is for example, yes. Or it could be other types of things. Yes.
- Magda: What other types of things do you have in mind?
- Sara: Well, because it's also impacting inventory, for example, you have to pull slips that certainly run to go and fetch things off the shelves, they are also querying basically the same underlying data that they are, etc. So those, I think come into, I think they can run sooner. And then because of the five colleges, I might start something, UMass might start something, and we're basically all utilizing the same resources.
- Magda: And this is also an example of something that we see with OMPMH when you are harvesting data. You're getting the data from inventory, from SRS, and it takes a lot of resources. So it impacts all the system. And especially if you do the full harvest, you should be doing it outside the regular hours because of its impact. I see what you are getting at here Sara. I will need to run it by the developers and by the architects to see how this is being handled because obviously, we don't want to clog the system. Let me talk to them and I will get back to you at our next meeting about how we are going to handle that. This is definitely a part of the performance investigating and addressing performance issues. This was a very good point.
- Sara: And also just going back to locations. Our discover layer uses RTAC to display where something is located. And that also depending can be like a slight delay there. So I'm also not quite sure what kind of resources are utilized there or the timing.
- Mada: RTAC also a module on the FOLIO side. It is more PAC I don't remember.
And I also am not fully familiar what this acronym stands for. I only know that the R stands for real time. So it means that it is actually queering the database for the location. So it, your, uh, discover layer since requests to, um, sense some request to the, uh, Inventory for your inventory and gets the information, uh, from inventory about the location of the item. So once you do update the inventory, the, the location item, location, artwork, we'll get the updated information in real time. So if you request before the update happens, you will get the old location. And then after it will get the, the new one, we are running out of time. And I was hoping I will be able to do walk through of the, uh, um, user acceptance testing. We most likely won't have time for this. I will put this in the slack channel. Um, this relates to the, um, Um, to the usability and the user or user acceptance testing for CSV approach, the environment was totally reset from what we had discussed on the beginning. And unfortunately, the setting, the users that we set up in the fall, uh, with the, the passwords are probably not there anymore. I have created a user for, uh, for us to test and they will also provide a sample of the different IDs. So you don't need to, uh, look for the files. You are more than welcome to do them, but in case you don't want to spend time on selecting IDs, uh, you can use, uh, you will be able to use the files. Uh, as I mentioned, I will pass this in that slack channel. And if possible, if you can do the, um, uh, eh, user acceptance testing for user records for CSV approach, uh, this week, that would be awesome. And if something in my, uh, instruction will be unclear, please feel free to reach out to me for clarification. Thank you. Thank you all for your feedback and great conversation. And I'll see you in two weeks. Bye bye. |
