Page Comparison

Attendees (please add your name):

...

Topic

Housekeeping

Attendees - please add your name to the list of attendees
Meeting host - please turn on Transcript option for the meeting

Magda 8:27
Thank you to those who already added your names to the attendee list. We have 15 participants at this point in the meeting. And I don't think I see 15 attendees on the on the page. So please, please add your name to the attendees list.

Bulk edit documentation for Morning Glory

Thank you (OLD ACCOUNT) Erin Nettifee and @Bob Scheier, @Emma Gollner, @Christine Tobias and @Amanda Ros!

Magda 9:01
Thank you so much to everyone who was involved in working on bug edit documentation for Morning Glory. It is so nice to have a new release with updated and very thorough documentation. Thank you so much.

Erin 9:23
We still need somebody who can own this app going forward with the documentation working group. So if that's something you think you might be able to do, please message me or Christine and we can talk further about it. Both Christina and I already have big areas we are responsible for so it would be really hard for us to continue to own the bulk edit app documentation.

Development updates

Recordings of implemented Nolana functionality can be found here

Magda 9:56
Development updates--there are links to the scrum board and to the snapshot environment. I highly encourage everyone to play with Bulk Edit in the snapshot environment. The majority of the functionality that was assigned to us or that we committed to do in Nolona is already in place. So you should be able to update user records.

Erin 10:41
Snapshot, not the performance environment?

Magda 10:47
The performance Bulk Edit environment URL has changed because we added support for multi-tenant environments. I will post the new URL in the Slack channel.

11:07
[Magda reviewed the Bulk Edit functionality in the Snatshot environment at this point in the meeting. ]

Unknown Speaker 11:51 51
packet did I choose the file

Magda 14:13
Please spend some time in snapshots, I do agree there is hardly any data. Now, I will start planning out user acceptance testing in the performance testing environment so we have a larger data set. Any questions?

Bob 15:33
I'm just a little confused about why we can't do it in the performance environment.

Magda 15:50
I will I don't remember exactly the new URL which has changed. And also the performance environment is not automatically updated with the latest code. So whatever you see in the snapshot environments may still not be deployed to the performance environment.

Morning Glory release notes

Bulk edit:

required configuration
known limitations
known issues

UXPROD-3468 Removing and adding permissions - continued from the last meeting

What data should be presented in the matching records table:
* Last name, first name, middle name, patron group?
* Permission name or permission display name?
Should the user be able to bulk edit mutable, hidden and deprecated permissions?

Magda 16:18
So let's go to our agenda again. And let's move to the next part, which is the Morning Glory release notes. As you know, morning, Glory was released on September 16th, and I would like to walk you through some points on the release notes. So first, the new app. There is only one a new app which is Bulk Edit in Morning Glory. And a quick overview what we support. In the in app approach, inventory items temporary and permanent locations and item statuses. In the CSV approach, user records. There are two sets of permissions that are required for this application, in app edit and view, and CSV edit and view. Depending on those permissions a user can access those applications. There are known there are Known limitations. As our performance tests discovered, in Morning Glory, you can only do a Bulk Edit of 2500 user records at the time. And for items, the limit is 10,000. This is a lot less than we were hoping for. Unfortunately, this is what it is for Morning Glory. We are looking at architectural changes to support a larger data set in the future.

Magda 18:54
The last part here, module configuration, is for those who are hosting themselves. For everybody else, this is being set by the hoster. Those are the memory requirements for the container that runs the module data export worker, which is being used in Bulk Edit

Erin 19:01
The performance test the performance work that you were talking about, is tracked in a particular Jira?

Magda 19:09
JIRA and reports. I will link the list. There is a new performance testing Jira that we started in the Morning Glory. Some of the work will spill over into Nolana. Starting next week, we will be working on bug fixes for the functionality that was already delivered and testing, performance testing, adding API tests, and adding automated tests for the UI. So, most of the tests can be run automatically.

Regarding the known issues, there are two Jiras that are linked to Bulk Edit. One which we discussed a couple of meetings ago, has to do with Bulk Edit results triggered by holdings identifiers. When the mapping between holdings and items is not one-to-one, it may happen that the list of identifiers is shorter than the list of matched items. This will need to be addressed in Nolana. We will be working on this functionality.

Then the next issue is a little bit more serious. When you update datasets larger or larger than 10,000 records, there is a significant delay of up to several minutes before the display of the preview screen, i.e., the "are you sure" form. We are working on improvements.

Erin 22:59
Did you ever see it timeout or just it would always come?

Magda 23:05
We saw this, but it happened rarely. So I cannot tell it never happened. Yes, we saw that happened. But if this is the case, you just need to submit the Bulk Edit again. The data is not committed at this point. Any additional questions?

Erin 23:55
Jennifer has a question in the chat. "Does it affect any other operations happening at that time?"

Magda 24:34
So the problems we saw with slowness are is caused by the mod data export worker that is being used. So separate from the inventory module or a data import module there should be no interference. However, if somebody exports EDIFACT or circulation logs are being exported, or bursar is being exported at the same time, it will have an effect. However, in our tests, when we ran all of those three exports at the same time, EDIFACT, bursar, and circulation with a Bulk Edit of more than 10,000 records, that is when we say it affects other exports happening at the same time. Any other questions?

Holdings source - Inventory vs. bulk edit (MODEXPW-247)

In inventory app, if the holdings record is not populated, it assumes that holdings source is the same as the instance record. Is that expected behavior for bulk edit as well?

Magda 26:55
Then I will move to the next part of our meeting. I would like to ask you about the behavior of the holdings source. There is a different behavior in inventory than in Bulk Edit. In Inventory, If their holdings source is not populated, the UI assumes the source of the instance that the holdings is associated with. In Bulk Edit, we don't do this because I am afraid this is a dangerous process. But I may be wrong. So I would like to hear your opinion. This is this JIRA, the screenshots will demonstrate the issue. So the question for this group is are you comfortable with the source being left blank here because it's blank in the database? Or would you rather we follow the inventory behavior?

Erin 28:33
So are you 're saying that if the Holdings Record doesn't have a source, it assumes it's the same as the instance record? And is that implying that it just displays it in the UI, but it does have value in the underlying record?

Magda 28:50
That's correct. So this is what is happening here in the inventory UI. The UI populates this field with the data that is actually coming from the instance.

Erin 29:05
That seems weird to me, but I don't know how the metadata folks feel about that. This is not the only place in FOLIO where the UI shows you something that doesn't isn't actually in the underlying data.

Jennifer 029:45
So I'm wondering because the source folio or source marc, that actually determines what you can open up the record in order to edit it. So for example, if it's a marc holding, you'll edit it in Quickmarc, whereas for folio source records you just use the actions to edit it. So, I find it strange that this is null in the back end and populated in the UI only. We have had problems with this, where if it wasn't folio we couldn't open the file, because it thought it was null. So I think for us, this shouldn't be null.

Magda 130:34
I agree with you. And this is also the case for Bulk Edit because, in Bulk Edit, we will not allow making the location changes for the records that have source, marc. That's why this is so important for us. I was surprised by the fact that this value is not populated. I admit, I did not look at the record if this is a required field or not. It may not be required, this may be a bug on the UI on the inventory side.

Erin 231:11
I wonder if this is an issue with the data that's on Snapshot.

Magda 232:16
I will double-check in bugfest environment. Okay. And if someone can check in their production environments as well.

Erin 232:32
Whether it's actually being populated?

Magda
yes.

Erin 232:39
Okay, because I don't know if anybody has Morning Glory yet?

Magda 232:40
Well, but the source was not added in Morning Glory. In earlier releases, I don't remember when it was.

Erin
But they could only verify by looking at the inventory data. So you're not asking people to look at bulk data yet?

Magda 232:55
No, no, no, no, no, just if you if you are comfortable looking at the records and checking the developer tools, which is by clicking F 12. And that you will be able to see if you don't if you are not able to do this, I will just please let me know. And I will find a way to get this data. I mean, to get the data, is this is populated in the production or not. If it's not, then we definitely will need to create a JIRA in the inventory. That may be migration script issue, that that migration script did not updated this value. But in general, you agree with me that this logic of assuming that the instance source is a base for holdings source is inaccurate? Correct?

Leeda 434:04
Yes. Well, I'm not sure if anybody's in production yet with marc holdings, but Duke for one will be will have both types.

Erin 434:15
Yeah. And it looks like the source ID is not required. Yeah, on the holding record.

Magda 434:24
So I'll check the bugfest environment after after this meeting to see what's going on. My only scenario was that you can import marc records using data import and then you can manually add holdings, and then that would make the holding marc holding which is not always the case. But thank you very much for clarifying that.

And we have less than 30 minutes so we won't be able to get to permissions.

UXPROD-3785 Advanced query for single record type

Building search query
Previewing results

Magda

4

34:27
But I would like to start a conversation about the query tool that is needed. We started actually the last meeting. I have some high-level Balsamiq mockups that I would like to walk through and get your feedback on. Please feel free to interrupt me. So this is the balsamic implementation of rendering of our bulk edit. This is something that we talked about--switching the order of the identifiers, put them on top, and then having the query or the identifiers below. So this is what is represented here

represented

.

Those

The two links at the bottom, save the query as

are up an existing list. It's not in

, and open the existing list, these are out of the scope of our conversation, for now

, this

. This is something we will come back to later.

But for, for

Image Added

For the purpose of this conversation, we

have, we

are on the Query tab

,

in bulk edit

, there

. There is a new button called Build query. When the user clicks the button, it opens a new pop-up that looks like this. It's a long data entry textbox with a test button disabled.

Unknown Speaker 7

Erin 37:06
So what you

're

are proposing

here

is a redesign from what's currently implemented

. And notUnknown Speaker 7

?

Magda 37:11
Not really redesigning,

I think we

adding. We will get to this

, we will get to this. So I noticed this should be this big square textbox. Here below, it's missing on my mock up. But basically, I'm adding the new button that will allow us to build the queryUnknown Speaker 7:39
this what you see here is plan would be a plan for puppy the other one. This is plan for Arcade. The difference is that in Arcade we would like to support just one record type. So you would be able to build the advanced query using only fields from the one record in Arcade it No, I'm sorry, I'm missing the releases in Arcade, there will be just one support for one record type in Poppy, you will be able to select to build the query based on several several record types. So in order to know what you have, there will be additional labels stating that you selected users because users are select that you can still edit only one record type at a time. But to build the query you have selected users. And then because you selected users user, users

. Basically, I'm adding the new button that will allow us to build a query.

Magda 37:39
And then in the Poppy release we will add additional record types. In Orchid we will support just one record type. So in order to know what you have, there will be additional labels showing that you have access to users inventory items, circulation loads, and then circulation requests etc

. This is for the demonstration purposes. And the questions on that Commons

Unknown Speaker 9:11
No, I think I'm with you so far.

Unknown Speaker 9:14
Okay, so we are back to arcade release user clicks on the on the textbox SEC

. to build your query. You can still only edit one record type using Bulk Edit. But in Poppy you will be able to access additional record types to build your query. Any questions or comments?

Orchid release

Image Added

Poppy release

Image Added

Magda 9:14
The user clicks on the search textbox and the list of available properties is

being

displayed.

Unknown Speaker 9:40
So

Unknown Speaker 9

Image Added

Magda 39:42
The user selects

,

patron group

this was

, one of the

property and then

available properties. Then the list of

identifiers and not an operator is available and the list of operators will obviously depends on the on the type of the, of the

available operators is displayed. The list of operators will depend on the the selected field. We will start

obviously

with equal, in, not equal, and not in.

Unknown Speaker 10:22
So this is the same rendering by preserving the, the first option. So a

The properties list is preserved so the user can go back and change

, ask a questionUnknown Speaker 10:34
Is it? Do we want to switch between words and punctuation? If if, like I understand what exclamation mark equals not, like not equals, oh, we should say not equal. Instead of, I don't know if other people have opinions on it, but just

it.

Image Added

Erin 40:34
Should we use words instead of symbols for consistency in that drop

down. Just if we're gonna use word stick to words. Yeah,Unknown Speaker 11

-down?

Thomas 41:05

I want I mean,

I think

there's

these are standard search operators, but still, I think I would just do equals, not equals, in, not in.

Unknown Speaker 11

Magda 41:17
Okay

, thank you. So, we have the operators. Now, we selected the the

, thank you. We selected the operator and now we

are getting

get the list the pre-populated list of available options.

Image Added

Magda 41:19
So that would obviously be

a

in place only for the controlled vocabulary. So if in some case user selects

the

a title, then obviously

,

we will not pre-populate the drop-down with all 10

Millions of

million titles that the collection may have.

That

Instead we would

be

present a

test

text entry

text where the user will be entering the data

box. But for

for

this example, we would pre

populate the values with the pre

-populate the combo

with the

with the existing values

for

. For options that will have hundreds of options like locations, for example

. The

, the drop

down will be a filtering drop down the same that

-down like what already exists in inventory

, where you select and your selection,

where what you type

and you're typing

narrows down the list.

Unknown Speaker 12:51
And the question No,

Unknown Speaker 12:57
that all makes sense. So

Unknown Speaker 12:58
next step is a user added the value. And the

Any question?

Magda 42:58
The next option is the Boolean operator

that

whbch will automatically be displayed. And user can choose to continue to add to the query or just not select anything.

A

Please note

, please note

that once the first option is entered, the

text

test button is enabled. It was disabled until now.

Image Added

And here's the example when the user continued to build

the query by adding the Boolean operator so we are back to I think this is up here we go.

the query by adding a boolean operator.

Image Added

Here's the example

that

where the user

submitted the query, click that

wrote the query, click the

Test this gray area it's a

test. The gray areais the progress bar that shows that

there is a progress bar happening when

the records are being counted. And in this example, we have more than 100,000 records returned and the notification please narrow

, narrow

down your search. The question here is what should

be

the number be, and how we should handle

the

that number. I do believe we talked about this at some point and was proposed that this

that

would be

configurable but

configurable

, a value that the library can set up, set it up

. However, we need to keep in mind that having a big number of

the

records

that you will Like to retrieve to query, it

will definitely impact the performance of

bulk edit

Bulk Edit.

Image Added

Unknown

Speaker 15

Speaker 45:06
Is this a number across the board for queries? Or is it like record by record type or

Unknown

Speaker 15

Speaker 45:11
that would be across the queries? Okay. So this is something we definitely will talk later. But I would like to you to keep it in mind. I see something in the chat. Let me take consistencies good.

Unknown

Speaker 15

Speaker 45:32
Green. That's from the conversation about the

Unknown

Speaker 15

Speaker 45:36
Okay. Okay, thank you. So no comments so far on this? Does anyone has like a quick feedback on the word? Oh, if this should be a limit of the bulk edit up.

Unknown

Speaker 15

Speaker 45:57
I mean, I would think that it probably should, if we're talking about if we're talking about our current example, where we have 25,010 1000, I would want 10,000 As this number.

Unknown

Speaker 16

Speaker 46:12
It's correction every net 25,000. But 2500 I'm sorry. So yes, this is just a proposal, as I said, we'll definitely come back to this lighter. So here what is happening right now, we run the query. And the query returns 1000 Records right now. And we see the preview. This is the hated preview that does not show all the records, but just some of them. And here you have the option of selecting columns. So this is easy to select. To see what you want to see and check and check the columns that you have. Once the preview is populated, there is a new button run the query. So the run the query will actually when the user click Run the query, what we'll do it will go back to the landing page, it closes the the pop up the login. And what is happening here the query that was created in the in the pop up is displayed here at this point is read on read only. And the list of the identify the list of matching records is here. So you see we can paginate to the to the results. Similarly, how this is done in inventory. By 100 records. There is also the option of the checkboxes at that user can narrow down their search results by selecting checkboxes. So in our example, we have 1000 records that that were found to match the query. But out of those 100 Only 10 was selected. And again here is the option of changing the columns. In this example, the user selected 100 records and on this 100 records, there will be a standard behavior of the bulk edit meaning you can save those 100 records into the CSV file. And then when you start and also when you start bulk edit the bulk edit will affect only those 100 Selected Records

Unknown

Speaker 19

Speaker 49:24
comments make that does the checkboxes. When you get your search results, they automatically come pre populated like all checked and then you you select

Unknown

Speaker 19

Speaker 49:40
I would say I will say yes that it will be populated. But this can be discussed in the details will not get to check boxes in Arcade that's for sure. But I would say yes let's populate and then buy sell Selecting that top checkbox, it will unselect the whole 100 that is displayed.

Unknown

Speaker 20

Speaker 50:09
Okay. Yeah, I think that's a good idea too. And the other comments?

Unknown

Speaker 20

Speaker 50:25
So the question I have is, if we get because this a rendering of the of the records, we can implement this also for the identifiers. Once you submit the list identifiers, instead of those top 10, you would get this behavior as well have the list that you can paginate. But the question I have for this group is, what makes what is reasonable to

Unknown

Speaker 21

Speaker 51:03
paginate?

Unknown

Speaker 21

Speaker 51:04
Through if you have 10,000 records that match your search criteria? Will you be paginating? By through those 10,000? By hundreds? No,

Unknown

Speaker 21

Speaker 51:18
I would I I'm only paginating. To verify that. I'm, I didn't do something dominant I might page through the first part of it. But I'm not going to page the entire set. Anybody who would records?

Unknown

Speaker 21

Speaker 51:35
I would think our filtering and things like that would be done in outside the system in a spreadsheet.

Unknown

Speaker 21

Speaker 51:42
Right? You mean you might want to you might want to download the entire set? Yeah, which you'll be able to do, but you're not going to paginate through that in that UI.

Unknown

Speaker 21

Speaker 51:52
Yeah, totally agree with that.

Unknown

Speaker 21

Speaker 51:56
So how many records? Should we actually what will be the cutoff number of records that will be rendered to mean?

Unknown

Speaker 22

Speaker 52:10
I? I guess I'm just thinking like, to me, that screen looks normal at paginating through 100. Right? And then it pulled 100 records or pulled 1000 records. And I would probably expect that if I clicked 10 times I would paginate through all the records. But I wouldn't probably do that very often. I'm sorry that I know, that's kind of like a wishy washy answer. No,

Unknown

Speaker 22

Speaker 52:41
I was asking the same question at some point in inventory. Display. Because yet, go ahead,

Unknown

Speaker 22

Speaker 52:50
I'm more likely to paginate through them, when I'm first getting used to how the bulk edit looks and works. You know, maybe the first 500 may be the first 1000. But if I'm getting consistent results that I expect to get, you know, the more that I use Bulk Edit, the less that I'm going to be scrolling through if I'm you know, if the in the beginning, it's doing what I want it to do.

Unknown

Speaker 23

Speaker 53:28
Sure, that makes sense.

Unknown

Speaker 23

Speaker 53:31
The thing is, I might I might want to see the end of the file results. Because just to make sure if I'm looking for any anomalies.

Unknown

Speaker 23

Speaker 53:43
Yeah, that also makes sense. You know, if I'm expecting to get results all the way through to ZTE or

Unknown

Speaker 23

Speaker 53:49
office, I don't know how these are. I don't know how they're coming back, you know, I'm just seeing the top of the file. So maybe there's no problems with the top of the file. But you know, halfway down the file, there's a bunch of errors for some reason, where I'm getting things back that I shouldn't that I didn't expect.

Unknown

Speaker 24

Speaker 54:05
Yes. So the other option would be to sort it the the other way around. However, this is expensive operation, the sorting of 10,000 records that we need to be mindful. So it may make more sense maybe to add another link that will take you to the last one.

Unknown

Speaker 24

Speaker 54:29
Can we can we export this to look at it in a spreadsheet?

Unknown

Speaker 24

Speaker 54:32
Yes. So this is the Yeah, the existing behavior of the of the spreadsheet that we have is preserved. So you still can send the list of matching records. Save the list of matching record, as you do right now in bulk edit.

Unknown

Speaker 24

Speaker 54:51
I would probably do that. You know, because I would want to see a whole bunch just look for any problems. So read

Unknown

Speaker 25

Speaker 55:00
So do you see the value in this, of adding that preview? Or not? Really? Because you would still go to the

Unknown

Speaker 25

Speaker 55:09
I think people were were expressing an interest in Yes, still seeing more than 10. You know, so I think this is valuable, but it doesn't have to go to the, you know, 200,000 Records. In my opinion,

Unknown

Speaker 25

Speaker 55:25
unless you take away our exploitability, which it doesn't sound like you're going to do it,

Unknown

Speaker 25

Speaker 55:32
can you Amanda, can you repeat,

Unknown

Speaker 25

Speaker 55:35
I'm saying, you know, as long as we will continue to be able to export the data into an Excel spreadsheet. I don't see it as as a problem. You know, because we're, we're all used to manipulating the data in Excel. So it would be probably easier for us to find those anomalies. And Bob is disagreeing with me?

Unknown

Speaker 26

Speaker 56:11
No, I was looking at my screen, because there things were going bad No, no, not at all.

Unknown

Speaker 26

Speaker 56:21
So I mean, given the fact that we can export it, I'm, you know, like I said, I'm not, in the beginning, I'm probably going to flip through a few pages just to make sure that it looks like I expect it to but I'm still going to download the data. So you know, I am not going to, this is not the hill that I'm going to die on about how many records, we need to be able to flip through scroll page through.

Unknown

Speaker 26

Speaker 56:54
Yeah, I mean, it seems to me that both are valuable tools for looking at the data and what should dictate how much data should be displayed as the system performance.

Unknown

Speaker 27

Speaker 57:10
So this is this is a very good point, Bob, because we have to keep two things in mind. The performance displaying those dis records, it will definitely have an impact on on the performance. But if this is something that needs to needs to happen, we will need to take it into account. But also to develop this functionality. It's not a simple thing, this is a lot of a UI development. So

Unknown Speaker 27:50
the

Unknown Speaker 27:52
if you feel that you need this, then we will obviously us allocate resources to build it. If you think this is a nice to have that you can that you can live without it, then we will allocate the resources differently.

Unknown Speaker 28:13
I think if we can say something that's like, we will show you the first 500 records if you want. If you got more back and you want to see the entire record set, download the export. If we can be say something that concretely in the documentation, I think that is fine for where we are at right now. I think eventually people are going to expect the full pagination because that's what you get, like Gmail, that's what you get in, in, you know, other other online tools. But I think I think if you can, as long as there's a way to get to the information, and as long as it's not like here's five records, if you want the rest, download the spreadsheet. I think that is okay.

Unknown

Speaker 28

Speaker 58:57
I see Sara's question in the chat. That. Yeah, this is only my rendering of a few records. In fact, as Erin suggested, mentioned, this is the scroll down that will show you the rest. This is actually how the inventory works as well. On the screen, you don't see that 100 Unless you usually have a very small font. And I know Sarah, you were you were the person who was asking mostly about this. Do you have any other comments?

Unknown

Speaker 29

Speaker 59:37
questions, requests?

Unknown

Speaker 29

Speaker 59:41
No, I just wanted to make sure because somebody mentioned 10 or something at some point. I just wanted to make sure that we were that it was actually as I was seeing it that there was a scroll bar scroll bar would take us through the first 10 100 and that it wasn't in debate about the 100 it was We're about whether we can do pagination. I guess I'm from other systems and others here to use all of and there was like a pop up modal that would pop out. And you could scroll you could just skim through. And then you could also download it and put it in Excel to to manipulate in detail. And so I guess the pagination, and while pagination works, it is a very, very cumbersome and folio, it takes forever to get from one page to the next page to the next page, right? So.

Transcribed by https://otter.ai

Versions Compared

Old Version 22

New Version 23

Key

Attendees (please add your name):