Magda 34:27
But I would like to start a conversation about the query tool that is needed. We actually started the last meeting. I have some high-level Balsamiq mockups that I would like to walk through and get your feedback on. Please feel free to interrupt me. So this is the balsamic implementation Bulk Edit. This is something talked about--switching the order of the identifiers, putting them on top, and then having the query or the identifiers below. So this is The two links at the bottom, save the query as , and open the existing list; these are out of the scope of our conversation for now . This is something we will come back to later.  Image AddedFor the purpose of this conversation, we . There is a new button called Build query. When the user clicks the button, it opens a new pop-up that looks like this. It's a long data entry textbox with a test button disabled. is a redesign from what's currently implemented ? Magda 37:11
Not really redesigning, adding. We will get to this Unknown Speaker 7:39
this what you see here is plan would be a plan for puppy the other one. This is plan for Arcade. The difference is that in Arcade we would like to support just one record type. So you would be able to build the advanced query using only fields from the one record in Arcade it No, I'm sorry, I'm missing the releases in Arcade, there will be just one support for one record type in Poppy, you will be able to select to build the query based on several several record types. So in order to know what you have, there will be additional labels stating that you selected users because users are select that you can still edit only one record type at a time. But to build the query you have selected users. And then because you selected users user, users have access to users inventory items, circulation loads, and then circulation requests etc. This is for the demonstration purposes. And the questions on that Commons Unknown Speaker 9:11
No, I think I'm with you so far. Unknown Speaker 9:14
Okay, so we are back to arcade release user clicks on the on the textbox SEC search textbox and the list of available properties is being displayed. Unknown Speaker 9:40
So Unknown Speaker 9:42
user selects, patron group this was one of the property and then the list of identifiers and not an operator is available and the list of operators will obviously depends on the on the type of the, of the selected field. We will start obviously with equal in, not equal and not in. Unknown Speaker 10:22
So this is the same rendering by preserving the, the first option. So a user can go back and change, ask a question Unknown Speaker 10:34
Is it? Do we want to switch between words and punctuation? If if, like I understand what exclamation mark equals not, like not equals, oh, we should say not equal. Instead of, I don't know if other people have opinions on it, but just for consistency in that drop down. Just if we're gonna use word stick to words. Yeah, Unknown Speaker 11:05
I want I mean, I think there's standard search operators, but still, I think I would just do equals not equals in Unknown Speaker 11:17
Okay, thank you. So, we have the operators. Now, we selected the the operator and now we are getting the list the pre populated list of available options. So that would obviously be a in place only for the controlled vocabulary. So if in some case user selects the title, then obviously, we will not pre populate the drop down with all 10 Millions of titles that the collection may have. That would be a test text entry text where the user will be entering the data. But for for this example, we would pre populate the values with the pre populate the combo with the with the existing values for options that will have hundreds of options like locations, for example. The drop down will be a filtering drop down the same that already exists in inventory, where you select and your selection, you type and you're typing narrows down the list. Unknown Speaker 12:51
And the question No, Unknown Speaker 12:57
that all makes sense. So Unknown Speaker 12:58
next step is a user added the value. And the next option is the Boolean operator that will automatically be displayed. And user can choose to continue to add the query or just not select anything. A note, please note that once the first option is entered, the text test button is enabled. It was disabled until now. And here's the example when the user continued to build the query by adding the Boolean operator so we are back to I think this is up here we go. Here's the example that the user submitted the query, click that wrote the query, click the Test this gray area it's a progress bar that shows that there is a progress bar happening when the records are being counted. And in this example, we have more than 100,000 records returned and the notification please narrow, narrow down your search. The question here is what should be the number and how we should handle the number I do believe we talked about this at some point and was proposed that this that would be configurable but configurable, a value that the library can set up, set it up. However, we need to keep in mind that having a big number of the records that you will Like to retrieve to query, it will definitely impact the performance of bulk edit. Unknown Speaker 15:06
Is this a number across the board for queries? Or is it like record by record type or Unknown Speaker 15:11
that would be across the queries? Okay. So this is something we definitely will talk later. But I would like to you to keep it in mind. I see something in the chat. Let me take consistencies good. Unknown Speaker 15:32
Green. That's from the conversation about the Unknown Speaker 15:36
Okay. Okay, thank you. So no comments so far on this? Does anyone has like a quick feedback on the word? Oh, if this should be a limit of the bulk edit up. Unknown Speaker 15:57
I mean, I would think that it probably should, if we're talking about if we're talking about our current example, where we have 25,010 1000, I would want 10,000 As this number. Unknown Speaker 16:12
It's correction every net 25,000. But 2500 I'm sorry. So yes, this is just a proposal, as I said, we'll definitely come back to this lighter. So here what is happening right now, we run the query. And the query returns 1000 Records right now. And we see the preview. This is the hated preview that does not show all the records, but just some of them. And here you have the option of selecting columns. So this is easy to select. To see what you want to see and check and check the columns that you have. Once the preview is populated, there is a new button run the query. So the run the query will actually when the user click Run the query, what we'll do it will go back to the landing page, it closes the the pop up the login. And what is happening here the query that was created in the in the pop up is displayed here at this point is read on read only. And the list of the identify the list of matching records is here. So you see we can paginate to the to the results. Similarly, how this is done in inventory. By 100 records. There is also the option of the checkboxes at that user can narrow down their search results by selecting checkboxes. So in our example, we have 1000 records that that were found to match the query. But out of those 100 Only 10 was selected. And again here is the option of changing the columns. In this example, the user selected 100 records and on this 100 records, there will be a standard behavior of the bulk edit meaning you can save those 100 records into the CSV file. And then when you start and also when you start bulk edit the bulk edit will affect only those 100 Selected Records Unknown Speaker 19:24
comments make that does the checkboxes. When you get your search results, they automatically come pre populated like all checked and then you you select Unknown Speaker 19:40
I would say I will say yes that it will be populated. But this can be discussed in the details will not get to check boxes in Arcade that's for sure. But I would say yes let's populate and then buy sell Selecting that top checkbox, it will unselect the whole 100 that is displayed. Unknown Speaker 20:09
Okay. Yeah, I think that's a good idea too. And the other comments? Unknown Speaker 20:25
So the question I have is, if we get because this a rendering of the of the records, we can implement this also for the identifiers. Once you submit the list identifiers, instead of those top 10, you would get this behavior as well have the list that you can paginate. But the question I have for this group is, what makes what is reasonable to Unknown Speaker 21:03
paginate? Unknown Speaker 21:04
Through if you have 10,000 records that match your search criteria? Will you be paginating? By through those 10,000? By hundreds? No, Unknown Speaker 21:18
I would I I'm only paginating. To verify that. I'm, I didn't do something dominant I might page through the first part of it. But I'm not going to page the entire set. Anybody who would records? Unknown Speaker 21:35
I would think our filtering and things like that would be done in outside the system in a spreadsheet. Unknown Speaker 21:42
Right? You mean you might want to you might want to download the entire set? Yeah, which you'll be able to do, but you're not going to paginate through that in that UI. Unknown Speaker 21:52
Yeah, totally agree with that. Unknown Speaker 21:56
So how many records? Should we actually what will be the cutoff number of records that will be rendered to mean? Unknown Speaker 22:10
I? I guess I'm just thinking like, to me, that screen looks normal at paginating through 100. Right? And then it pulled 100 records or pulled 1000 records. And I would probably expect that if I clicked 10 times I would paginate through all the records. But I wouldn't probably do that very often. I'm sorry that I know, that's kind of like a wishy washy answer. No, Unknown Speaker 22:41
I was asking the same question at some point in inventory. Display. Because yet, go ahead, Unknown Speaker 22:50
I'm more likely to paginate through them, when I'm first getting used to how the bulk edit looks and works. You know, maybe the first 500 may be the first 1000. But if I'm getting consistent results that I expect to get, you know, the more that I use Bulk Edit, the less that I'm going to be scrolling through if I'm you know, if the in the beginning, it's doing what I want it to do. Unknown Speaker 23:28
Sure, that makes sense. Unknown Speaker 23:31
The thing is, I might I might want to see the end of the file results. Because just to make sure if I'm looking for any anomalies. Unknown Speaker 23:43
Yeah, that also makes sense. You know, if I'm expecting to get results all the way through to ZTE or Unknown Speaker 23:49
office, I don't know how these are. I don't know how they're coming back, you know, I'm just seeing the top of the file. So maybe there's no problems with the top of the file. But you know, halfway down the file, there's a bunch of errors for some reason, where I'm getting things back that I shouldn't that I didn't expect. Unknown Speaker 24:05
Yes. So the other option would be to sort it the the other way around. However, this is expensive operation, the sorting of 10,000 records that we need to be mindful. So it may make more sense maybe to add another link that will take you to the last one. Unknown Speaker 24:29
Can we can we export this to look at it in a spreadsheet? Unknown Speaker 24:32
Yes. So this is the Yeah, the existing behavior of the of the spreadsheet that we have is preserved. So you still can send the list of matching records. Save the list of matching record, as you do right now in bulk edit. Unknown Speaker 24:51
I would probably do that. You know, because I would want to see a whole bunch just look for any problems. So read Unknown Speaker 25:00
So do you see the value in this, of adding that preview? Or not? Really? Because you would still go to the Unknown Speaker 25:09
I think people were were expressing an interest in Yes, still seeing more than 10. You know, so I think this is valuable, but it doesn't have to go to the, you know, 200,000 Records. In my opinion, Unknown Speaker 25:25
unless you take away our exploitability, which it doesn't sound like you're going to do it, Unknown Speaker 25:32
can you Amanda, can you repeat, Unknown Speaker 25:35
I'm saying, you know, as long as we will continue to be able to export the data into an Excel spreadsheet. I don't see it as as a problem. You know, because we're, we're all used to manipulating the data in Excel. So it would be probably easier for us to find those anomalies. And Bob is disagreeing with me? Unknown Speaker 26:11
No, I was looking at my screen, because there things were going bad No, no, not at all. Unknown Speaker 26:21
So I mean, given the fact that we can export it, I'm, you know, like I said, I'm not, in the beginning, I'm probably going to flip through a few pages just to make sure that it looks like I expect it to but I'm still going to download the data. So you know, I am not going to, this is not the hill that I'm going to die on about how many records, we need to be able to flip through scroll page through. Unknown Speaker 26:54
Yeah, I mean, it seems to me that both are valuable tools for looking at the data and what should dictate how much data should be displayed as the system performance. Unknown Speaker 27:10
So this is this is a very good point, Bob, because we have to keep two things in mind. The performance displaying those dis records, it will definitely have an impact on on the performance. But if this is something that needs to needs to happen, we will need to take it into account. But also to develop this functionality. It's not a simple thing, this is a lot of a UI development. So Unknown Speaker 27:50
the Unknown Speaker 27:52
if you feel that you need this, then we will obviously us allocate resources to build it. If you think this is a nice to have that you can that you can live without it, then we will allocate the resources differently. Unknown Speaker 28:13
I think if we can say something that's like, we will show you the first 500 records if you want. If you got more back and you want to see the entire record set, download the export. If we can be say something that concretely in the documentation, I think that is fine for where we are at right now. I think eventually people are going to expect the full pagination because that's what you get, like Gmail, that's what you get in, in, you know, other other online tools. But I think I think if you can, as long as there's a way to get to the information, and as long as it's not like here's five records, if you want the rest, download the spreadsheet. I think that is okay. Unknown Speaker 28:57
I see Sara's question in the chat. That. Yeah, this is only my rendering of a few records. In fact, as Erin suggested, mentioned, this is the scroll down that will show you the rest. This is actually how the inventory works as well. On the screen, you don't see that 100 Unless you usually have a very small font. And I know Sarah, you were you were the person who was asking mostly about this. Do you have any other comments? Unknown Speaker 29:37
questions, requests? Unknown Speaker 29:41
No, I just wanted to make sure because somebody mentioned 10 or something at some point. I just wanted to make sure that we were that it was actually as I was seeing it that there was a scroll bar scroll bar would take us through the first 10 100 and that it wasn't in debate about the 100 it was We're about whether we can do pagination. I guess I'm from other systems and others here to use all of and there was like a pop up modal that would pop out. And you could scroll you could just skim through. And then you could also download it and put it in Excel to to manipulate in detail. And so I guess the pagination, and while pagination works, it is a very, very cumbersome and folio, it takes forever to get from one page to the next page to the next page, right? So. Transcribed by https://otter.ai. Basically, I'm adding a new button that will allow us to build a query. Magda 37:39
And then, in the Poppy release, we will add additional record types. In Orchid, we will support just one record type. So in order to know what you have, there will be additional labels showing that you have access to users inventory items, circulation loads, and then circulation requests, etc., to build your query. You can still only edit one record type using Bulk Edit. But in Poppy, you will be able to access additional record types to build your query. Any questions or comments?
Orchid release  Image Added
Poppy release  Image Added
Magda 9:14
The user clicks on the search textbox, and the list of available properties is displayed.  Image AddedMagda 39:42
The user selects the patron group, one of the available properties. Then the list of available operators is displayed. The list of operators will depend on the selected field. We will start with equal, in, not equal, and not in. The properties list is preserved so the user can go back and change it.  Image Added Image AddedErin 40:34
Should we use words instead of symbols for consistency in that drop-down? Thomas 41:05
I think these are standard search operators, but still, I think I would just do equals, not equals, in, and not in. Magda 41:17
Okay, thank you. We selected the operator, and now we get the list the pre-populated list of available options.  Image AddedMagda 41:19
So that would obviously be in place only for the controlled vocabulary. So if, in some cases, the user selects a title, then obviously, we will not pre-populate the drop-down with all 10 million titles that the collection may have. Instead, we would present a text entry box. But for this example, we would pre-populate the combo with the existing values. For options with hundreds of options like locations, for example, the drop-down like what already exists in inventory where what you type narrows down the list. Any question? Magda 42:58
The next option is the Boolean operator, which will automatically be displayed. And user can choose to continue to add to the query or just not select anything. Please note that once the first option is entered, the test button is enabled. It was disabled until now.  Image Added
And here's the example when the user continued to build the query by adding a boolean operator.  Image AddedHere's the example where the user wrote the query and click the test button. There is a gray progress bar that shows that the records are being counted. And in this example, we have more than 100,000 records returned with the notification, please narrow down your search. The question here is what the number should be and how we should handle that number. I do believe we talked about this at some point, and it was proposed that this would be configurable. However, we need to keep in mind that having a big number of records will definitely impact the performance of Bulk Edit.  Image AddedErin 45:06
Is this a number across the board for queries? Or is it by record type? Magda 45:11
That would be across the queries. So this is something we definitely will talk about later. But I would like you to keep it in mind. Does anyone have quick feedback on if there should be a limit for the Bulk Edit? Erin 45:57
I mean, I would think that it probably should if we're talking about our current example, where we have a 25,000 limit. I would want 10,000 as the limit here. Magda 46:12
Correction. Not 25,000 rather, it is 2,500. This is just a proposal. As I said, we'll definitely come back to this later. So we hit the test button for the query, and the preview of the resulting 1000 records displays. This preview does not show all the records, but just some of them. You have the option of selecting columns to see what you want to see by checking and unchecking the columns. Once the preview is populated, there is a new button, Run query.  Image AddedSo the run the query button will close the pop-up and bring the user back to the landing page. And what is happening here is: - The query that was created in the pop-up is displayed here on the left side as read-only.
- The list of the identified matching records is listed.
- We can paginate through the results similarly to how this is done in inventory by 100 records.
- There is also the option of checkboxes where the users can narrow down their results by selecting checkboxes. So in our example, we have 1000 records that were found to match the query. But out of those 1000, Only 100 were selected. And again, there is the option of changing the columns. In this example, the user selected 100 records, and with these 100 records, there will be the standard behavior of Bulk Edit, meaning you can save those 100 records into the CSV file. And then, when you start Bulk Edit, the edits will only affect those 100 selected records.
 Image AddedMark 49:24
When you get your search results, are the checkboxes automatically prepopulated? Magda 49:40
I would say yes they will be populated. But this can be discussed. We will not get to check boxes in Orchid, that's for sure. But I would say yes, let's prepopulate and by selecting the top checkbox, it will unselect the whole 100 that are displayed. Magda 50:25
So the question I have for this group is, what is a easonable number to paginate? If you have 10,000 records that match your search criteria, will you be paginating through those 10,000? By hundreds? Erin 51:18
I'm only paginating to verify that I didn't do something dumb. I might page through the first part of it. But I'm not going to page through the entire set. Leeda 51:35
I would think our filtering and things like that would be done outside the system in a spreadsheet. Erin 51:42
You might want to download the entire set, which you'll be able to do, but you're not going to paginate through that in that UI? Leeda 51:52
Yeah, totally agree with that. Magda 51:56
So what will be the cutoff? How many records will be rendered? Erin 52:10
To me, that screen looks normal at paginating through 100. And then it pulled 100 records or pulled 1000 records. And I would, probably expect that if I clicked 10 times I would paginate through all the records. But I wouldn't probably do that very often. I'm sorry that I know that's kind of like a wishy-washy answer. Leeda 52:50 I'm more likely to paginate through them when I'm first getting used to how the bulk edit looks and works. But if I'm getting consistent results that I expect to get, the more that I use Bulk Edit, the less that I will be scrolling through if it's doing what I want it to do.Erin 53:28
Sure, that makes sense. Bob 53:31
The thing is, I might want to see the end of the results in the file just to make sure if I'm looking for any anomalies. Erin 53:43
Yeah, that also makes sense. You know, if I'm expecting to get results all the way through to Z. Bob 53:49
I don't know how the results are coming back. I just see the top of the file. So maybe there are no problems with the top of the file. But halfway down the file, there's a bunch of errors for some reason, where I'm getting things back that I shouldn't or that I didn't expect. Magda 54:05
So the other option would be to sort it the other way around. However, this is an expensive operation, the sorting of 10,000 records that we need to be mindful of. So it may make more sense maybe to add another link that will take you to the last one. Bob 54:29
Can we export this to look at it in a spreadsheet? Magda 54:32
Yes. The existing behavior of the spreadsheet that we have is preserved. So you can still save the list of matching records, as you do right now in Bulk Edit. Bob 54:51
I would probably do that. You know, because I would want to see a whole bunch just look for any problems. Unknown Speaker 55:00
So do you see the value in adding this preview? Or not? Unknown Speaker 55:09
I think people were expressing an interest in seeing more than 10. You know, so I think this is valuable, but it doesn't have to go to 200,000 Records. In my opinion. Amanda 55:25
As long as we will continue to be able to export the data into an Excel spreadsheet, I don't see it as a problem because we're all used to manipulating data in Excel. So it would probably be easier for us to find those anomalies. I'm probably going to flip through a few pages just to make sure that it looks like I expect it to, but I will still download the data. So you know, I am not going to. This is not the hill that I'm going to die on about how many records we need to be able to scroll through. Bob 56:54
Yeah, I mean, it seems to me that both are valuable tools for looking at the data and what should dictate how much data should be displayed is the system performance. Magda 57:10
So this is a very good point, Bob, because we have to keep two things in mind, the performance displaying those records will definitely have an impact on the performance. But if this is something that needs to needs to happen, we will need to take it into account. But also to develop this functionality. It's not a simple thing, this is a lot of UI development. So if you feel that you need this, then we will obviously allocate resources to build it. If you think this is nice to have but you can live without it, then we will allocate the resources differently. Erin 28:13
If we can show the first 500 records with the option to download the entire record set if you got more than that in your results. If we can say something concretely in the documentation, I think that is fine for where we are at right now. I think, eventually, people will expect the full pagination because that's what you get in applications like Gmail and other online tools. Magda 58:57
I see Sara's question in the chat. That. Yeah, this is only my rendering of a few records. In fact, as Erin suggested, mentioned, this is the scroll down that will show you the rest. This is actually how the inventory works as well. On the screen, you don't see that 100 Unless you usually have a very small result. And I know Sarah, you were the person who was asking mostly about this. Do you have any other comments? Sara 59:41
No, I just wanted to make sure because somebody mentioned 10 or something at some point. I just wanted to ensure that we were that it was actually, as I was seeing it, that there was a scroll bar that would take us through the first 100 and that it wasn't in debate about the 100 but more about whether we can do pagination. I guess I'm from other systems and others here on Aleph there is a pop-up modal that you could scroll through. And then, you could also download it and put it in Excel to manipulate it in detail. And so I guess the pagination, while pagination works, it is a very, very cumbersome and FOLIO. It takes forever to get from one page to the next page and to the next page, right? Magda 60:01
So would you rather have a thousand on one page instead of pagination? Sara 60:33
I would prefer that, but I understand this is costly. I just did not want to go back to 10 selected. Magda 60:48 I hear you. The simpler way right now is to change the preview from 10 to 100 and then download the rest to see more. We have two minutes left, so we are not going to get to permissions. We will get to that at the next meeting, or I will reach out to you on Slack. If you have more feedback, feel free to reach out to me. Thank you. Bye. | 
