Skip to end of banner
Go to start of banner

2021-11-30 Bulk Edit Working Group Meeting Notes

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 21 Next »

Attendees (please add your name):

Magda Zacharska Robert Scheier Amanda Ros (OLD ACCOUNT) Erin Nettifee Monica Arnold Jennifer Eustis Christine Tobias Sara Colglazier Erin Weller Jenn Colt Jackie Magagnosc Lloyd Chittenden 

Note taker:

Robert Scheier

Meeting Recording:

https://drive.google.com/file/d/1xkLm-P4H2KkEjuVbqe8jh9hff5x3jzCs/view?usp=sharing

Discussion:

TimeTopicNotes 

Housekeeping:

  • Please add your name to the attendees list
  • Make sure Transcript option is turned on during the meeting
  • Magda: The first question I have is related to the transcript. If I turn on this transcript, it will crash my computer and then I will not be able to last until the end of the meeting.
  • Bob: I will record with my phone if everyone is OK with that.
  • Magda: Please add your name to the list of attendees on the wiki page.

Development updates:

  • Magda: Developments - Unfortunately not that many changes are visible. There is progress on the backend happening that will unblock the UI work. Hopefully next time there will be more to discuss. However, the team has created a rather detailed technical design document. The link is in the agenda. Please feel free to review the document. The document is fairly technical and is inventory-specific. However, the development team is aware that we are building an application that will support bulk edit in other areas.
    • Erin: Question: This document specifically mentioned S3. Will this be similar to other approaches where S3 will be required to use the app? I am thinking about Texas A&M and other places?
    • Magda: Actually, we started using S3 in data export but in the Kiwi release we made the hosting gnostic, meaning the libraries don't need to use S3. And this will be also a part of the bulk edit. They will not be required to run bulk edit on AWS. It's a very good question though. Feel free to add this to the comments on the page. I discussed this with the developer. He did not respond to that in writing, but I will follow up and confirm.



Bulk Edit Pilot - expectations - continue discussion 

So far we discussed and I believe we agreed upon :

  1. Small but reliable functionality that is limited to the user records as defined in in the scope of UXPROD-3225 
  2. Evaluated by Bulk Edit Working Group, SIGs, Product Council and  Technical Council: 
  3. Available for acceptance testing in a separate environment that will contain larger dataset and will not be refreshed nightly (RANCHER-63)

We still need to discuss:

  • Should it be included in Lotus release?
  • Next steps
  • Magda: I would like to continue with the bulk edit expectations. In a very short summary, I think we agreed that:
    • The scope should be small, but reliable
    • functionality is limited to user records as defined in the scope of UXPROD-3225
      • In scope will be:
        • Limited bulk edit for user records
        • Search by user group, list of barcodes
        • Edit: remove permissions, update expiration date, update patron group, update email
        • All elements related to bulk edit workflow (even if only limited form):
          • Identify records for editing
          • Edit record(s): add data, remove data, update data, i.e., change the values of the fields within the records. It is not adding new or deleting recorders.
        • Question from Erin: I think that removing permissions which aren't in the core user record shouldn't be in scope for the pilot.
          • Magda: I would agree with you that permissions will be a difficult thing to implement, especially since this is a separate record. It is not, as I like to refer to it a linked record, similar to fees and fines. But I would like to keep it for now. And then maybe towards the end of the release, we will evaluate it.
        • Question from Jennifer: When you say remove the permissions, is that like both individual and group permissions?
          • Magda: I was thinking in terms of groups especially the use case where you have student workers and at the end of the semester you need to remove permissions for this group of users.
          • Erin: Permissions are different because you have the concept of a user record and then you have the concept of a permission user record. Permissions aren't directly stored on the user record, they're just linked. It's not like you would go into the user record and just say, remove this attribute and this attribute and this attribute, you have to break the link with the permission user. So that's where this would be really complicated. But if you were to remove permissions, you are removing the link to that person's permissions, whether it was individual permissions, permission sets, visible, invisible, whatever. You're just removing them all.
          • Magda: It's not only breaking the link; it is also removing the record from the table that contains the data. We don't want to have abundant data left in the tables.
          • Sara: In our current system I can have an expiration date for the permission. So, I can assign a student worker, certain permissions and it expires on such and such date, but the record remains because I find this useful to be back, we'll shoot and see that that person did work here with this username and find if they did anything that I want to fix.
            • Erin: Unfortunately, FOLIO does not have that functionality to expire permissions. There's an expiration date on an account, but there is no expiring on permission.
            • Magda: I really liked the comment, Sarah. This is important feedback you need to go back to the data, and if we remove it, it's gone. I will make a note of this. Also, if you talk with Erin about this, please include me as well.
            • Sara: At the Five Colleges permission groups will be set up by the FOLIO Administrator. Will this be a permissions problem for me in bulk edit if the permissions are controlled by someone else?
            • Erin: So, my guess is that it shouldn't if you have one person who creates the permission sets, but then you farm out assigning those permission sets to people.
            • Magda: If you have a right to add the permission, you should be able to bulk edit those as well.
            • Amanda: That's the same structure that we're using at Texas a and M. And right now, I don't have any problems taking permissions away from someone. So, I wouldn't think that there would be a problem with that unless something gets broken when they're fixing something else, which unfortunately we've seen in a couple of instances.
            • Sara: Questions: Will the permission to do bulk edits be separate from the permission to do individual edits?
            • Magda: Permissions will be at the app level. So yes, they will be separate permissions.
  • Magda: Getting back to scope - I just want to go quickly over the list of the covered elements.
    • So we will identify the records for editing, then edit by adding, removing, updating data within the record.
    • Review changes, commit changes, log results
    • Handle exceptions, including notifying the users.
    • And we discussed who should give the green light to the approach. Not only this group but also Technical Council, Product Council. In fact, the Product Council reached out to me and asked me to present the planned functionality at the next PC meeting this Thursday. The reason for this is that a bulk edit will be a new module. And those new modules will need to be approved to be included in the Lotus release. And the deadline for this is December 3rd and we are coming to close to the deadline.
    • Magda: And we also discussed that we would like to have a separate environment for the usability testing for the environment where the group can play around with the functionality that was delivered. I created a story for FOLIO DevOps to create an environment. And as I mentioned, I'm coming from vacation and I'm not sure where we are on this. The status is still open. I will follow up with them to see what is the estimated delivery for this Rancher. I would like this environment to be available for SME's usability and acceptance testing. It would have the latest FOLIO snapshot version, and the environment will not be updated automatically every night. It will be a refresh after we request that. The data there will also be larger than that in the snapshot testing environments. Ideally, it will have several thousand records and have a way easy way to have the developers populate with more data as needed.
    • Magda: The environment needs to be stable. SMEs should be able to use the data between when the environment is refreshed. And the environment, as I mentioned, is refreshed on-demand, not automatically.
    • Magda: The environment will not be for performance testing.
      • Only a couple of thousand user records should be needed.
      • We will be doing performance testing in parallel. But I do not want this environment to be used for performance testing.
      • Once we move to inventory we will probably go up to 1 million records.
      • Question from Erin: So how will performance testing happen?
        • Magda: I think the performance task force team has their own environments. And this is how the performance testing happens for the data export and OAP MH. Once development has progressed, we just create a ticket with PTF and they spin the environment and we specify how many concurrent users we want to have.
        • Question from Erin: Is the testing that they're doing about the responsiveness of the UI?
        • Magda: The performance task force mostly works on the backend. So, they will be checking the behavior of the backend. The team that works on the bulk edit, Firebird, is one of the few teams that has a UI tester, automated tester, and engineer who will be writing the UI tests. I will follow up with the PTF and ask them what they want to do about UI.
        • Magda: But from my experience, when the system is slow, it's slow because the backend is slow. The UI will not slow things that much.
        • Question from Sara: I was not just thinking about performance, but also about connectivity and interaction. So I would suggest that we think about having users that have things on loan, users that have other issues maybe owe some money, etc., so that we can see if thins break, or if things no longer show up, or we've just erased things that we didn't intend to erase, or broke the link, or it just chokes and does something weird because it can no longer pull that data from the other side. I'm just basing this on experience in our production side, with for example agreements where just weird things suddenly pop up if you put a tag somewhere and it breaks everything. I would hate for us to sign off on the pilot only to discover that there are weird hiccups.
          • Magda: So, Sarah, should we create some kind of document that would track all the things that we should keep in mind when setting up the data?
          • Sara: Is there a way we can can also think about quality in the sense of complex data. So not just simple data where you have users at different levels or different groups. But more complexity with things like loans and fines.
        • Magda: I probably will need to reach out to a user management group to provide some real-life examples. And then we will need to do this. Once we move to the other areas, we will need to do this for other areas as well.
        • Sara: Is there a way, without invading privacy, to get some kind of dump from a site that is life?
        • Erin: There is a dump of that kind of data on bug Fest. The concern about bug Fest is that the data has been permutated over and over and over again, and has a lot of cruft in it. But that is something that has been done before. I just don't think that we have done it for several releases.
        • Magda: I will reach out to FSC?
          • Erin: Maybe Anton directly.
          • Erin: I believe that the user data that's on buck Fest came from the University of Chicago.
          • Magda: I agree with you that the best way would be to have real-live data instead of artificially setting up the data because there may be use cases we did not think of. And we would need to anonymize, the data for privacy.
          • Magda: Let me follow up on this. And I will get back to you via slack.

Mitigation of Bulk Edit UX issues 

Slide 9:

  • Query-based, itemized, combination (for example: change location of the following holdings that are not currently checked out) 

Slide 16:

  • Long term -  we will support both in app and local editing
  • Short term  - local editing for the pilot program

Slide 17:

  • Provide a more transparent way of understanding what FOLIO did when batch processing
  • Better understanding of what a failure means -- did it revert the record, delete the record, invalidate the record?
  • Itemize failures so it’s easy to identify affected records, e.g. “record #XXXXXX failed,” not “7 of 10,000 changed failed”
  • Pay particular attention to global failure where feedback is especially spare
  • Give some indication, if possible, of how failure can be avoided
  • Provide some indication on how to re-try or follow-up from a failure

Slide 19

  • Optimize .csv files for understandability
  • Make it very clear which fields can or should be edited and which should be avoided  (in app or local edits?)
  • Provide some degree of instruction on how to make changes to commonly edited fields like status and location (in app?)
  • Give users the power to rollback changes upon upload if files are corrupted or mistakes were made (pre-update records stored in S3)
  • Anticipate potentials error and provide warnings, e.g. “Is this correct?,” “Are you sure?”

Mockup- file uploadMockup - are you sure | Mockup - delete modal | Mockup - delete errors




Survey review:




Future discussion topic (time permitting):

  • Bulk Edit of the linked records
  • Scheduling edits



  • No labels