2022-11-1 Bulk Edit Working Group Meeting Notes

Attendees (please add your name):

Magda Zacharska (OLD ACCOUNT) Erin Nettifee Jennifer Eustis Amanda Ros Timothy Dannay Thomas Trutt Jackie Magagnosc Monica Arnold Christine Tobias Erin Weller Scott Perry Kimie Kester Donald Depoorter

Note Taker:

Robert Scheier

Meeting Recording:

Recording and chat: https://recordings.openlibraryfoundation.org/folio/bulk-edit-working-group

Discussion:

Topic	Details	Notes
Nolana UAT Scenarios	Decide which environment to use
UXPROD-3842 Bulk edit - architectural improvements	The changes are needed because the current implementation is not scalable enough to support larger data. Results of current performance tests conducted by the team Proposed redesign
Changes to the Bulk edit roadmap	TBD
Housekeeping Attendees - please add your name to the list of attendees Meeting host - please turn on Transcript option for the meeting - do we still need that? Updates on bulk edit rancher (aka bulk-edit-perf environment) We still need someone to take on upkeep / edits of the Bulk Edit app documentation ((OLD ACCOUNT) Erin Nettifee ) - can you ask at your institutions / SIGs if you are not able to do so?	[Magda Zacharska] 10:03:13 Big. Thank you, Bob, for being our note taker for more than a year Now we've been meeting for more than a year in October. October was the one-year anniversary we really appreciate your detail. I really appreciate your detailed notes. They are very helpful if I need to go back to our prior meetings. So thank you very much for your work, and if you get to the point that you feel you need someone else to take it over, please speak up. [Magda Zacharska] 10:03:51 I added a column on our wiki space notes pages table to allow a place to add any notes because I started to add a little bit more details for each topic that we discuss to provide some information to help you plan your week if you will be attending the meeting or not. The update on Bulk Edit Rancher--we talk about that during our last meeting. It used to be the Bulk Edit performance environment that we shared for user acceptance testing and performance testing that the team was doing. Then we run into the problems with updating the environment. I promised to share the link. I did not share the link, and this was not by omission. This was on purpose. We are competing for the same resources as much as the UAT is important, performance testing is also important. In Morning Glory, the UAT took press precedence, and we spent a significant amount of the last weeks before the release improving performance. I did not want. This happened in Nolana as well. So I suggest we either do UAT using the bugfest or snapshot environment. Other teams have done it. We will get to this conversation later. So you don't need to speak up right now. But this is what I will propose. In the future, and by that, I mean the Orchid release, we will have the option to spin up an environment only for Uat purposes. But this is not in place just yet. so that's why I suggest we use either of the existing environments. Any comments on that concerns? [Erin Nettifee] 10:06:30 No, I mean, I think snapshot would be challenging to do just because there aren't enough records. But I imagine we'll get into that later. on. [Magda Zacharska] 10:06:35 Okay, that makes sense as well. [Erin Nettifee] 10:06:42 We are still looking for somebody to take over the upkeep and edits of the Bulk Edit app documentation. And my request at this point, because no one has said they would do it, is, can you please ask around your institutions or SIGS if you don't have the capacity to do this? I think it's a good opportunity to learn the app and to get involved more deeply with FOLIO, and there is definitely onboarding that happens with that process. You're not just thrown into the deep end, but you would be taught how to maintain documentation and how that process works and stuff like that. So we really still need to find somebody who can do that. So I'm just gonna reiterate that request. [Magda Zacharska] 10:07:43 How much time do you think it takes? How much time would that require? [Erin Nettifee] 10:07:56 It is hard to forecast because we are adding functionality. So that's a lot of updates and creating new content instead of editing or fixing existing content. I would say, I think it's probably gonna be maybe at least 5 hours a release right now, and maybe more, and regular meetings. So it's not a trivial amount of time, but it's also not like asking somebody to be a product owner. It's just making a commitment and being willing to take on that ownership. [Magda Zacharska] 10:08:34 Great. Thank you, Erin, for bringing it up. We will be announcing this until someone finds time to take this on.
Development updates Development status Development team scrum board Recordings of implemented Nolana functionality can be found here	[Magda Zacharska] 10:08:47 Regarding the development updates. The links to the Snapshot environments, and these scrum boards are here, as always. [Magda Zacharska] 10:09:29 You can check them any time you want. but I would like to bring to your attention that we have a folder with recordings for functionality added for the release. [Magda Zacharska] 10:09:40 So here is the folder. There is a folder called Bulk Edit app recordings, and for Nolana, I created a folder with the name Snapshot. [Magda Zacharska] 10:10:03 The folder has recordings for each functionality that we have delivered in Nolana. And this covers user records in-app approach, item records in-app approach, and holdings records in-app approach. [Magda Zacharska] 10:10:22 I also attached the CSV files that I am using in my demo to trigger Bulk Edits in case you want to play on your own, you can use those files. [Magda Zacharska] 10:10:39 The recordings don't have a sound it's only the recording you can stop and rewind if something is unclear. Any questions? [Magda Zacharska] 10:10:53 I do believe this will be helpful. when you start looking into functionality that was in the scope of Nolana. [Erin Nettifee 10:14:53 I think it's especially helpful to have the files and the videos in the same location.
Review of the functionality delivered in Nolana Modifications to the UI: Selected columns persists through out the bulk update screens Adjustments to the left pane so that the record types are listed on top before identifier selection User records - in app approach: Patron group can be updated Expiration date can be changed Part of the email address can be updated (host name changes) Holdings records: Permanent location can be updated Temporary location can be removed or updated Item records: Permanent loan type can be updated Temporary loan type can be removed and updated	Magda Zacharska] 10:11:10 The next topic is the review of functionality. [Magda Zacharska] 10:11:20 I have a Powerpoint presentation that I would like to walk through and then do a quick demo of the functionality we have done. [Magda Zacharska] 10:11:37 Please feel free to interrupt me while I go through the slides. [Magda Zacharska] 10:11:52 So in the Nolana there are 3 main functional features, one to support user data updates using in app approach. For item records we implemented locations and statuses. We also added loan types. [Magda Zacharska] 10:12:18 Records we added to already a implemented holdings, locations, and statuses. [Magda Zacharska] 10:12:25 And for holdings, we started with the locations as well. [Magda Zacharska] 10:12:34 Those 3 functional areas are completed. We are still testing. Possibly some bugs will be discovered. But the main development has been completed. [Magda Zacharska] 10:12:47 In addition to that, we have a series of NFS testing which includes automated tests for the backend. [Magda Zacharska] 10:12:57 This is Karate for front and releasing the modules and updating that technology, especially for the spring application we needed. [Magda Zacharska] 10:13:18 So what have we done while working on those features and some changes to the Ui? [Magda Zacharska] 10:13:31 Some of them were mentioned during user acceptance testing at the end of Morning Glory. [Magda Zacharska] 10:13:39 One of them was to change the order of components in the left pane, where the user selects the record type first, and then selects the identifiers. [Magda Zacharska] 10:14:01 The next suggestion from user acceptance testing was preserving the selection of the columns. [Magda Zacharska] 10:14:10 Now the user can select the columns that are displayed on the landing page when the matching records are displayed, and the selection of those columns is preserved on the are you sure form where we preview the changes that will be made and also on the confirmation screen once the changes are committed. [Magda Zacharska] 10:14:39 And the last Ui improvement is on the export manager. Magda Zacharska] 10:14:50 We can now filter out Bulk Edit jobs and also by statuses: successful, in progress, or failed. [Erin Nettifee] 10:15:05 Bulk Edit does not use scheduled right? [Magda Zacharska] 10:15:28 Correct. In this case it means that something went wrong with the uploading the identifiers and the job never progress further. [Magda Zacharska] 10:15:38 So this is something that is in a not expected behavior. When you click on the export manager on the row, it will tell you the details of what went wrong. But you are correct. The scheduling of the Bulk Edite Jobs is not implemented yet. It is planned for the later releases. [Magda Zacharska] 10:16:01 And the other questions, the comments. [Magda Zacharska] 10:16:08 The nice thing I will move to that user data in the in-app approach. We will be supporting, in addition to the CSV approach that was delivered in the scope of Morning Gloria, on the request of community, we also added support for the in-app approach, starting with the most common changes: change in email hosting, expiration date update, and patron group. [Magda Zacharska] 10:16:55 I will demo this later, and this is just a short preview of of the of the changes we made. [Magda Zacharska] 10:17:12 On the item records Bulk Edit, we can also do the changes to temporary loan type and permanent loan type. [Magda Zacharska] 10:17:17 In addition to already implemented support for a temporary item location, permanent item location, and item status. [Magda Zacharska] 10:17:35 For the holdings records. we added support of temporary and permanent holdings locations. [Erin Nettifee] 10:17:50 And this is just the inventory records this is that's that is correct? Not SRS? [Magda Zacharska] 10:18:02 That is correct. This is only for records that have a source folio. [Magda Zacharska] 10:18:04 Okay, So let's do a quick demo (see recording for demo. Notes below are just regarding questions and comments that arose). [Erin Nettifee] 10:19:34 Is the the fact that the actions are great out here mean that is the only choice? Or could you choose a different action? So if I wanted to, for example, erase remove the expiration date from a set of user accounts, I would have to use the CSV approach? [Magda Zacharska] 10:20:19 We don't support setting it to empty Also the saem with Patron Group. Patron group which is a required field, and it needs to be populated. [Erin Nettifee] 10:20:30 Expiration data is not required on the user record. [Magda Zacharska] 10:20:34 So for now we can only support replace with something. [Erin Nettifee] 10:20:45 We will need to add this functionality separately. [Magda Zacharska] 10:21:01 So let's make it again [Magda Zacharska] 10:21:07 Let let's confirm the changes so we see here first of all, the the columns that we picked on the on the preview is preserved. [Magda Zacharska] 10:21:23 The we see that the date will be updated as specified here. [Magda Zacharska] 10:21:33 The patron group will be The faculty and the emails will be example to talk the functionality of downloading the preview is still supported. [Magda Zacharska] 10:21:47 So you can still download the file and see all the records [Magda Zacharska] 10:21:58 And you see the records in its the entire record, not only the columns you have selected [Magda Zacharska] 10:22:13 And we are committing the changes, and here is the confirmation of the of the changes we have a committed It's also the the option of downloading the the change records. [Magda Zacharska] 10:22:33 Any questions on on this functionality [Magda Zacharska] 10:22:41 Great. So let's move to the next one and it will be item records again, selecting item records [Magda Zacharska] 10:23:03 And we have 3 that are available, all of them are let's see the permanent location and temporary location. [Magda Zacharska] 10:23:12 We need the long time we don't need the material type [Magda Zacharska] 10:23:21 So here what we, he here, and that the I can temporal location, this main library and permanent location for some is empty. [Magda Zacharska] 10:23:34 One has annex and the permanent hello, typist can circulate and temporary load. [Magda Zacharska] 10:23:43 Low type is a curse reserve so let's make changes and assume that those 3 items will move to the reading room [Magda Zacharska] 10:23:56 We will make temporary location as replace with [Magda Zacharska] 10:24:07 Popular reading room. We will not touch the permanent allocation at all. [Magda Zacharska] 10:24:15 We will change the item status to make it restricted. [Magda Zacharska] 10:24:21 You can use it only in the reading room, and the temporary long time will change to [Magda Zacharska] 10:24:29 The replace with reading room computer changes again. Let me see, the preview that the status will be restricted. [Magda Zacharska] 10:24:46 The permanent long time has not changed, but the temporary, long time change to the reading room. [Magda Zacharska] 10:24:53 Permanent location has not changed either. The item, a temporary location change to the popular reading room. [Magda Zacharska] 10:25:01 We are committing the changes, and here we go those are the changes i'll commit that the database any question on that [Erin Nettifee] 10:25:22 No, I don't think so they're not from me i'm not monitoring chat. [Magda Zacharska] 10:25:34 If someone has any comments, please speak up if I don't address them, and the last one is the inventory holdings. [Magda Zacharska] 10:25:44 Let's do again holdings hrid's [Magda Zacharska] 10:26:01 We are going to start Bug Edit and here's again. [Magda Zacharska] 10:26:06 The case that if we have a permanent location that is required, you can only replace it. [Magda Zacharska] 10:26:13 You cannot specify You cannot seen it clear it. [Magda Zacharska] 10:26:22 And for the temporary location. This is what we are going to do will clear. [Magda Zacharska] 10:26:28 The field confirm the changes Oh, no I didn't have to expect this. [Magda Zacharska] 10:26:39 Well, there is a recording of this working so I will like I don't want to try it again. [Magda Zacharska] 10:26:50 Let's yeah let's leave it to where it is for now we will be addressing the the holdings and issue in the back backfast. [Magda Zacharska] 10:27:08 Release. So any questions on this [Erin Nettifee] 10:27:15 If you are doing a book edit and you get that error message, Do you see? [Magda Zacharska] 10:27:22 Is there any way to access information about what happens like? Would you have to go into developer tools? [Magda Zacharska] 10:27:29 Or let me see so let's see what is happening here. [Magda Zacharska] 10:27:33 We are selecting bug edit. This is probably our last job, but this is the last one that happened. [Erin Nettifee] 10:27:42 Send, I do believe, 2 min ago. Yeah, this is the issue. [Magda Zacharska] 10:27:48 I believe, happened on the Ui that the Ui did not handle this correctly. [Erin Nettifee] 10:27:54 Okay, let me go to easy if the snapshot is a bill. [Magda Zacharska] 10:28:05 I'm available. I wanted to show it on the on the snapshot within the the Us. [Magda Zacharska] 10:28:10 Code that may have handled the issue. I do believe there were changes on the back end, and were not handled by the by the Ui. [Magda Zacharska] 10:28:18 That's Why, we see we see the problem and we don't see anything in the logs here. [Magda Zacharska] 10:28:24 So once we meet next time I will. definitely sure. [Magda Zacharska] 10:28:34 Sure. The update on the issue on the holdings. Yeah. [Magda Zacharska] 10:28:38 Location and any questions. months from a new one [Erin Nettifee] 10:28:58 No I don't have any and i'm i'm not seeing any in the chat. [Magda Zacharska] 10:29:02 Sorry. I assume folks are are feeling okay. So in I would like now to move to the Eu user acceptance, testing and the scenarios. [Magda Zacharska] 10:29:19 I change a little bit. approach this time, instead of selling the survey and to awaiting your responses, I am. [Magda Zacharska] 10:29:34 I would like to use the excel spreadsheet approach. [Magda Zacharska] 10:29:38 This is with a spreadsheet that contains that's, steps for each record type user records in app inventory items and inventory holdings, and each step here shows what needs to happen upload to the [Magda Zacharska] 10:30:03 files with users, review the matching records, download the matching records to your machine, and then click! [Magda Zacharska] 10:30:12 Start, and all the steps that we that we have that I showed you today. [Magda Zacharska] 10:30:19 I added column for each institution that is on the [Magda Zacharska] 10:30:25 By Gadget Working group listed. If your institution has more than one representative, if you want to add your comments, please clone the column and and add your comments here in this empty fields [Magda Zacharska] 10:30:48 type Pass not pass, or any other comments. [Magda Zacharska] 10:30:48 You, you find apple applicable Also, if you represent the organization that is not listed here, please feel free to add another as well. [Magda Zacharska] 10:30:59 Since we agreed that we are going to use the back edit environment and the bug not packet. [Magda Zacharska] 10:31:08 It's breakfast environment and the backfast starts next week I will repost the link to this document in our slack channel with the logging information to the backfast environment and we will have 2 [Magda Zacharska] 10:31:27 weeks to complete the the user accent acceptance type, and they could [Erin Nettifee] 10:31:50 The next part is import. 1 one question which might be getting into the environment question, which is for the user acceptance testing. [Magda Zacharska] 10:31:59 We are all doing the same editing the same records it's it's up to you because if we I can provide you with the list of records identifiers, then you will be using the same [Magda Zacharska] 10:32:16 files, and you will be most likely stepping on each other toes, if you do those at the same time. [Erin Nettifee] 10:32:25 Sure, and you can use your own for identifiers or I can provide you with the multiple list of identifiers, and then you will be marking which one you use. [Erin Nettifee] 10:32:47 You that like what records was, or did I use so I don't know how that would be done. [Erin Nettifee] 10:32:51 Most effectively I can. You know you could just put a note in the user testing. [Magda Zacharska] 10:32:56 You could, but that just seems like something that would matter. [Magda Zacharska] 10:33:01 Some of a test failed. So in bulk edit, Google drive for your Google drive we have a Do we? [Magda Zacharska] 10:33:10 Have a folder for you at Nolana right now. [Magda Zacharska] 10:33:21 It contains only a user acceptance testing scenarios. [Magda Zacharska] 10:33:24 This is the area where you will be able to drop the files. [Magda Zacharska] 10:33:31 If you are using your own set of identifiers, and you want to. [Magda Zacharska] 10:33:36 Refer to those in your comments. Please save those here in this folder. [Robert rscheier@nelib.org] 10:33:46 So Mac, that we use provide the files, and then I could take a copy of those files and use those same files. [Magda Zacharska] 10:33:56 So I will provide a list of files The the challenge is that if we all use the same identifiers and can change to one value. [Magda Zacharska] 10:34:09 And this somebody else does it at this very moment, as you do. [Robert rscheier@nelib.org] 10:34:14 It will affect your results. I see. Okay, I got it [Jennifer Eustis (she/her)] 10:34:26 Oh, my God, I I just have a quick question the uat testing is, in addition to the test. We're doing in test rails. [Magda Zacharska] 10:34:34 Right. That's correct, alright. thank you this is only for our bunk, added App. [Magda Zacharska] 10:34:43 We do have a lot of about added tests in in test, right? [Magda Zacharska] 10:34:49 But this is a slightly different point of view. You can use your [Magda Zacharska] 10:34:57 X. If you, if you are volunteering for running bug edit tests in testile, you can also use your experience while doing it to update the user acceptance testing scenarios. [Magda Zacharska] 10:35:11 I specifically, leave those columns blank so that you can feel free to add any information you feel is, [Magda Zacharska] 10:35:24 Relevant to to the testing. If you feel like some changes will need to happen. [Magda Zacharska] 10:35:32 Please add them as you know that whatever was mentioned in morning Gloria user acceptance testing, we we got very scarce response. [Magda Zacharska] 10:35:44 But for those that were mentioned. we tried our best to. [Magda Zacharska] 10:35:49 Accommodate that. So tickets when you, when you, when you will do the user acceptance testing, please keep in mind that we are listening to you And in trying to implement yeah feedback, And I would like to move to [Magda Zacharska] 10:36:13 the to the feature that we will be working in Atlanta. [Magda Zacharska] 10:36:18 Those is related to architectural improvements in by edit, and we have less than 30 min left. [Magda Zacharska] 10:36:28 So. I would like to spend this time on talking about that and getting your feedback. [Magda Zacharska] 10:36:37 This is a a big endeavor to change. to make architectural changes, we need to start them early. [Magda Zacharska] 10:36:46 To make sure we have enough time to test it properly. [Magda Zacharska] 10:36:50 And this will definitely affect the buck edit road map I don't think we will be able to talk about this today. [Magda Zacharska] 10:36:59 I'm also in the process of updating a bucket road map, so we will return to to this next time we meet [Magda Zacharska] 10:37:10 So why do we do? why do we decided to make architectural improvements in Morning glory when we did that performance testing, and the goal was to support editing at least 100,000 records? [Magda Zacharska] 10:37:31 We realize that the architecture does not support that, and the performance is going even worse. [Magda Zacharska] 10:37:40 If there are other application using export manager and those other applications are exporting edifact records. [Magda Zacharska] 10:37:53 Bars are export circulation, log exports, all of those. [Magda Zacharska] 10:37:59 When they are happening they have an impact on bulk edit. [Magda Zacharska] 10:38:03 So, after further analysis of the code and into implementation, our Solution Architect got a special call, proposed, redesign it to separate and bug edit from export manager. [Magda Zacharska] 10:38:22 So that it is not affected by all other things that are happening. [Robert rscheier@nelib.org] 10:38:29 And in while other export. Oh, well, other exports export the data a a added link to the performance results from a morning glory, and you can take a look at them when you have a chance. [Magda Zacharska] 10:38:50 I added the link to the proposed proposed design, and I would like to spend a couple of minutes talking about it. [Magda Zacharska] 10:39:01 I will not be going into the details technical details, but there is one part I would like to spend a couple of minutes. [Magda Zacharska] 10:39:08 It's a bulk operation states because this describes how the data is being processed. [Magda Zacharska] 10:39:17 How will be processed? They will be several States. [Magda Zacharska] 10:39:23 Not all of them will be implemented in scope of rocket. [Magda Zacharska] 10:39:27 Release, but those that will include new. This is the the moment when the user uploads the identifiers or run the the query. [Magda Zacharska] 10:39:40 This is the other feature that we will start working in. [Magda Zacharska] 10:39:44 Okay, the next step is when the user retreats the records. [Magda Zacharska] 10:39:51 So we submitted the the way we want to identify the records, either identifiers or the query, and the records are being retrieved from the storage storage modules. [Magda Zacharska] 10:40:06 This is this step. State A is the second state, the state. [Magda Zacharska] 10:40:13 The third State is saving records. This is the snapshot of the records that you get when you run the query. [Magda Zacharska] 10:40:23 Those files. This is written Csv. file, but this is basically a file that is created in export manager. [Magda Zacharska] 10:40:32 But we will move this file to the bug edit. [Magda Zacharska] 10:40:36 Good storage. This will be not model internal storage. [Magda Zacharska] 10:40:40 This will be a storage in S. 3 Armenia for those who are not using Amazon web services. [Magda Zacharska] 10:40:48 This is the file where they that it will be stored. [Magda Zacharska] 10:40:52 This is not the Csv file. you are downloading to your local machine. [Magda Zacharska] 10:40:57 This is something that is still happening in there, the in the system. [Magda Zacharska] 10:41:01 The next state is data money, a modification. This is when user specifies what changes will happen either in in up approach or in the Cse approach for user records. [Magda Zacharska] 10:41:18 Next state is reviewing changes. This is where the records are being displayed on. [Magda Zacharska] 10:41:24 Are you sure? form the the last one? The apply changes is the last one that we are mostly familiar with. [Magda Zacharska] 10:41:37 This is when the changes are being committed to the to the records stored in the stored in the storage modules. [Magda Zacharska] 10:41:46 So this is so saving the data. the suspense status is one of the statuses that we will not support. [Magda Zacharska] 10:41:55 Yet in our this will be implemented later, when the user will be able to suspend. [Magda Zacharska] 10:42:05 The bulk edit. This will obviously require further discussion. [Magda Zacharska] 10:42:10 How we want to implement that completed. This is self self-explanatory, explanatory. [Magda Zacharska] 10:42:19 And this is the job that was completed, and with no errors. [Magda Zacharska] 10:42:23 The next one. This will errors when you will get the notification, and that it completed. [Magda Zacharska] 10:42:30 But they were errors, and you will still see the errors in the error accordion, and the the next one is cancelled. [Magda Zacharska] 10:42:40 We are not going to support canceling jobs. in orchid something to discuss later that's also the case for scheduled also results for the future, and failed. [Magda Zacharska] 10:42:55 This is when something went wrong and did not, and and was not [Magda Zacharska] 10:43:04 And and the operation did not complete [Erin Nettifee] 10:43:08 Can I ask some questions, Magna: Sure. So this list of statuses is this something that is just part of the new design? [Magda Zacharska] 10:43:19 Is this something that would end up being exposed in an that the bulkheaded app both? [Magda Zacharska] 10:43:25 Yes, so th. This is a part of the new design, and it will be something that will be. exp. [Magda Zacharska] 10:43:32 Then exposed in Bug edit, and the way we are going to do this. [Erin Nettifee] 10:43:37 It's by adding additional tab to back edit. Please let me know if you can see this is probably a little bit so small. [Erin Nettifee] 10:43:49 But yeah, no, this is better. now. yeah, that's a little bit better. [Magda Zacharska] 10:43:53 Okay, so you'll see this in other logs. tab. So in addition to identifier query and logs, this would be again driven by permissions. [Magda Zacharska] 10:44:10 But if you have permission, you can see the log and the standard for your filtering and the functionality by status by record type. [Magda Zacharska] 10:44:24 What recorded type was export and modified. [Magda Zacharska] 10:44:28 And then type of bucketed operation it was if it was bucketed or bug delete. [Erin Nettifee] 10:44:35 When the job started when the job ended. Okay, So this is kind of like deciding to build our own version of the export manager stuff just for bulk edit, so that it can be customized to bulk at [Magda Zacharska] 10:44:53 its , [Erin Nettifee] 10:44:59 It has a more information. You get more data here. So what you have here you have okay, barg, edit operation, type, record type status. [Magda Zacharska] 10:45:12 Those are the status that we mentioned. this is who ran it when it started running when it ended running the number of records. [Erin Nettifee] 10:45:24 That was okay affected. How many records like how many records were requested? [Magda Zacharska] 10:45:32 How many records were were processed. Was it the editing in app or Manual Manual? is the [Magda Zacharska] 10:45:41 Csv. approach, name a replacement, and here you have dogs, information or empty. [Magda Zacharska] 10:45:48 If the actions are empty, it means no files were created, and it's at this point no files are available. [Magda Zacharska] 10:45:57 When you click on, Doc, this will be the next screen shot up. [Magda Zacharska] 10:46:02 This is this: again. so clicking on the action menu will give you the option to download the file, and you can download each of the files that are being created in the process of of bug editing so [Magda Zacharska] 10:46:25 First it starts with the with the file. you can retrieve the file that you use to trigger the bug edit. [Magda Zacharska] 10:46:33 Then the matching records errors that were accounted during the matching the the preview of the changes. [Magda Zacharska] 10:46:42 This is what is the equivalent on our issue? Form, then the list of it. [Magda Zacharska] 10:46:50 Updated records and the file with er errors that were encountered by when the changes were committed. [Erin Nettifee] 10:47:02 And so Thomas is asking about in the chat about whether the status is, and information that we're seeing here would be useful in the export and the export manager. [Erin Nettifee] 10:47:15 App. But I think what i'm guessing is that when this is implemented, there will no longer be bulk at it. [Magda Zacharska] 10:47:21 Stuff in export manager will just be all encompassed in its own interface. [Magda Zacharska] 10:47:31 So when we go to the statuses there's one part that happens still in in the export manager. [Magda Zacharska] 10:47:39 So when when the records are being retrieved from the back end. [Erin Nettifee] 10:47:44 Okay, So this is what is happening in the export manager. [Magda Zacharska] 10:47:50 But the file that is being created is being moved to debug edit short storage, so the file will not be accessible. [Magda Zacharska] 10:47:59 We will see that the records are being retrieved. [Magda Zacharska] 10:48:02 But we will not have those files at all in export management. [Erin Nettifee] 10:48:11 Okay, that feels a little disjointed to me but I don't know if others have thoughts or on it. [Erin Nettifee] 10:48:18 I think I would want to be able to see all the stuff just in one app. [Magda Zacharska] 10:48:25 So you will see all the stuff in bug edit so you don't, You would see all the files in in bulk edit. [Magda Zacharska] 10:48:39 You don't need to go to export. manager we can hide whatever is in export. [Erin Nettifee] 10:48:46 Manager. So it's. not confusing for anyone okay, I mean this is all kind of abstract, so you i'm not advocating for a particular decision. [Thomas Trutt] 10:48:56 I don't think but yeah go ahead thomas sounded like you were about to say something [Thomas Trutt] 10:49:05 Popping up in the export manager. Then other things popping up hidden here are duplicated over here. [Thomas Trutt] 10:49:12 My big worry about this is I like the different statuses. [Thomas Trutt] 10:49:18 I had, like all the information as being shown here. But are we solving the issue? [Magda Zacharska] 10:49:24 Are we just moving it to another app? So they were. Yeah, we we first of all, we are not competing for resources in with another applications. [Magda Zacharska] 10:49:37 That was one of the issues we are facing. when [Thomas Trutt] 10:49:44 When you look at this, the this is the status. This is just a top of what the skipping on the on the flow here is what is happening right now before Bug Bulk added. [Magda Zacharska] 10:50:01 All the updates, everything that the most resource, greedy part of bug edit is actually saving the changes to the database and handling those changes occur accurately. [Thomas Trutt] 10:50:14 This was until and Nolana. This has been done by. [Magda Zacharska] 10:50:20 But export manager, which is not the place where those things should happen. [Magda Zacharska] 10:50:28 This should be a bulk-added responsibility [Thomas Trutt] 10:50:36 Does it answer your question a little bit? it it it's I still have a concern. [Thomas Trutt] 10:50:43 The only reason why? I said is Yes, it's it's It's getting more access to the bulk at it back end process. [Thomas Trutt] 10:50:48 Because now it's his own app and it's able to do all this processing on its own my worries is that it's still going to be computing with the export manager and other processes, because it's [Thomas Trutt] 10:50:58 still going to be hitting the same Api endpoints on the other internal apps. [Thomas Trutt] 10:51:03 So it might add, even compound it, because you now have 2 large apps hitting the same api's at the same time. [Thomas Trutt] 10:51:12 I guess I could see where this might make a little bit more sense, because at the X data export, App was doing the updates moving that component out of it. [Magda Zacharska] 10:51:23 And that does make sense. Because, like, why would you have updating data in the actually export? [Magda Zacharska] 10:51:28 App and I see your point that we are hitting the the same like in case of hi, Thomason, holding. [Magda Zacharska] 10:51:42 So we are. We are hitting the same environment with different apis [Magda Zacharska] 10:51:51 But this will happen if somebody also exports, or does the oapm H. [Magda Zacharska] 10:51:58 Or make any other inventory changes. this is not what we can prevent. [Magda Zacharska] 10:52:06 What we know, however, that if something goes wrong we can identify the module, and we can handle this within a module. [Magda Zacharska] 10:52:16 One module. and this is a good point. We probably will be coming to this once. [Magda Zacharska] 10:52:22 We have a a performance test for the in place for the new for the new design. [Magda Zacharska] 10:52:31 I was told this will resolve our problem with the limit of 10,000 records. [Magda Zacharska] 10:52:42 At this time we should be able to go up in the number of records we can update through the bulk operation, but this definitely will not be a superb blood for inventory or circulation modules performance. [Magda Zacharska] 10:53:06 I would like we have some of the minutes left. There is one more mockup that I would like to show. [Magda Zacharska] 10:53:17 Is those [Magda Zacharska] 10:53:22 Hi Tim, that have expired because we will be putting the record. [Magda Zacharska] 10:53:28 The the files, a large number of files in the external in the external storage. [Magda Zacharska] 10:53:37 We will keep them for a month. So for that bug edits that happen in the previous month, you will be able to access them. [Magda Zacharska] 10:53:47 But then we will remove them, and the files that were removed will be marked with this information, as you see on the screen, plus unavailable for download, because 30 days have elapsed since the job was run [Magda Zacharska] 10:54:04 in export manager currently you are getting the Xml error that tells you that the exploration talking elapsed, which is not very user friendly. [Magda Zacharska] 10:54:15 That's why we decided to go this route to to notify the user that defaults and no longer, and [Erin Nettifee] 10:54:31 So the 30 days then is hard coded. we can make it. [Magda Zacharska] 10:54:41 We can make it configurable through the api because if it turns out it's a preferred approach. [Magda Zacharska] 10:54:52 We will start with hardcoded value of of 30 records 30 days. [Magda Zacharska] 10:54:58 This was the the value that was at some point proposed for the files that are being generated by data export as well. [Erin Nettifee] 10:55:06 Does anyone have any comments? Sure, I mean that that and 30 days makes as much sense as anything to me? [Erin Nettifee] 10:55:12 It might be worth a question to Sisops or to, you know, ebsco hosting or index data, just to get a sense. [Erin Nettifee] 10:55:21 This, too, what they think of that number and configuration and stuff like that. [Erin Nettifee] 10:55:27 But here I mean the the stuff that we would have to retain would be things like financial records, and we're just not. [Erin Nettifee] 10:55:34 That's not what's happening here. so I I think 30 days, I think would would be okay. [Erin Nettifee] 10:55:40 But it I it's probably worth just asking around [Magda Zacharska] 10:55:50 So I run this obviously by obscure hosting team. [Magda Zacharska] 10:55:55 And I do also believe that that may depends on size of the institution. [Erin Nettifee] 10:56:04 Sure, the large institution that have larger about edits, and they have a larger file size same way. [Magda Zacharska] 10:56:13 Also want to cop on calls of storing those files because they will be eating much more space than for smaller institution when they have a smaller and smaller file size, they will not be eating that much of this storage space and [Thomas Trutt] 10:56:35 30 days may be not even required for them. They could go longer without running out of space or incurring costs, for it sounds like this is almost a from what you even just described. [Thomas Trutt] 10:56:56 Now as this might move around based on the institution, and that would make more sense to have. This is a tenant level setting, or have it as an api endpoint that could be hit. [Magda Zacharska] 10:57:05 You could say delete all files after the State that could be set as a cron job or something. [Magda Zacharska] 10:57:11 I, Rather let the user the tenant to specify their grace period and yeah, that would be. [Thomas Trutt] 10:57:23 That would be my preference, as well but if that's not possible. [Magda Zacharska] 10:57:27 The second one would be having something that a host hit an Api and say, Remove these files up after this date sounds good. [Magda Zacharska] 10:57:36 I will bring it up to the to the development team [Magda Zacharska] 10:57:45 So this is it what I had for today? [Magda Zacharska] 10:57:49 Do you have any comments, questions. [Erin Nettifee] 10:57:59 I know how many i've [Magda Zacharska] 10:58:12 Thank you all, and i'll see you in in 2 weeks and We will start with the roadmap updates. [Erin Nettifee] 10:58:20 Thank you. Hey, Erin, Can you save that file? [Erin Nettifee] 10:58:25 Yes, I will save it for you. Thank you. have a good one.

Chat Discussion	00:03:36 Jackie Magagnosc: Boy log in in the middle and wonder what is going on 00:04:01 Jackie Magagnosc: LOL 00:05:36 Robert rscheier@nelib.org: I am having issues. Can you turn on transcription 00:06:29 Robert rscheier@nelib.org: No worries. If you still want me. LOL 00:06:54 Robert rscheier@nelib.org: will do 00:07:08 Jennifer Eustis (she/her): Thanks Bob for being notetaker :) 00:13:55 Jennifer Eustis (she/her): This is great Magda. Thank you for these videos and lists of ids 00:14:08 Christine L Tobias: Thank you, Magda! 00:48:36 Thomas Trutt: But wouldn't the same statuses and information be useful in the export tool? 00:49:25 Erin Nettifee: i think the necessary backend changes to support bulk edit probably make building another interface an easier path... 00:56:13 Ros, Amanda L: I think it has potential. In the long run I will like it better as it's own app 00:56:30 Ros, Amanda L: (I hope) 00:58:38 Thomas Trutt: Agreed, id run the 30-days past a few other groups.