[MODRTAC-95] RTAC results order Created: 10/Apr/20  Updated: 30/Apr/23

Status: Draft
Project: mod-rtac
Components: None
Affects versions: None
Fix versions: None

Type: Story Priority: P2
Reporter: Anya Assignee: Khalilah Gambrell
Resolution: Unresolved Votes: 0
Labels: support
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: PNG File screenshot-1.png     PNG File screenshot-2.png     PNG File screenshot-3.png    
Issue links:
Defines
defines UXPROD-4036 R2 2023 | RTAC Enhancements/Bugfixes/... Open
Sprint: Prokopovych - Sprint 149, Prokopovych - Sprint 151, Prokopovych - Sprint 150
Development Team: Prokopovych
Release: Not Scheduled
RCA Group: Implementation coding issue

 Description   

The Volumes in an RTAC display are out of order ..

In FOLIO they appear in the screenshot order... and that is what is expected in the RTAC results

The Volume numbers are coming in reverse order (4, 3,2) that their different locations are appearing in separate sections (Media Collection Volumes First, 3 Hour Reserve Second), and that the volume numbers have Parentheses appearing under volumes.

<?xml version='1.0' encoding='UTF-8'?>
<holdings>
<holding>
<id>3593ddc4-3703-11ea-92ee-271862221368</id>
<callNumber>PN1998.3.L46 S473 2007 DVD</callNumber>
<location>Media Collection - Level 1</location>
<status>Available</status>
<volume>(v.4)</volume>
</holding>
<holding>
<id>3593bdda-3703-11ea-92ee-271862221368</id>
<callNumber>PN1998.3.L46 S473 2007 DVD</callNumber>
<location>Media Collection - Level 1</location>
<status>Available</status>
<volume>(v.3)</volume>
</holding>
<holding>
<id>35939fbc-3703-11ea-92ee-271862221368</id>
<callNumber>PN1998.3.L46 S473 2007 DVD</callNumber>
<location>Media Collection - Level 1</location>
<status>Available</status>
<volume>(v.2)</volume>
</holding>
<holding>
<id>35936ad8-3703-11ea-92ee-271862221368</id>
<callNumber>PN1998.3.L46 S473 2007 DVD</callNumber>
<location>3 Hour Reserve - Library Service Desk</location>
<status>Available</status>
<volume>(v.1)</volume>
</holding>
</holdings>
'



 Comments   
Comment by Anya [ 10/Apr/20 ]

Magda Zacharska - are you the RTAC PO ? if not do you know who to direct this to?

Kelly Drake for awareness

Comment by Magda Zacharska [ 10/Apr/20 ]

Anya The xml that you included in the description - does it come from EDS or from mod-rtac? Mod-rtac sits between FOLIO and RTAC in EDS so I think we should start investigating here first.

cc: Kelly Drake

Comment by Craig McNally [ 10/Apr/20 ]

I'd have to look to be sure, but I don't think mod-rtac even attempts to order it's results, either across locations or volumes... It sounds like the desired order is first by location descending, then by volume ascending when there are multiple items in a given location. Is that right?

Just curious... Is the problem that they're not shown in the desired order in EDS, or is there another consumer of RTAC that needs the results in a particular order?

Comment by Craig McNally [ 10/Apr/20 ]

in fact, mod-rtac doesn't attempt to provide consistent order of results across multiple requests, In the example below I make 2 requests for the same instance and get results in different order (both across locations and within a location). I don't think this was a requirement at the time of the initial implementation.

Example:

$ curl $OKAPI/rtac/7f380ee2-67d8-4d00-8435-a3af64b10b31 -H "X-Okapi-Token: $TOKEN" -w'\n'
{
  "holdings" : [ {
    "id" : "14e71ce6-465f-409a-b93a-e8037e4be0c7",
    "location" : "Annex",
    "callNumber" : "HOB2",
    "status" : "Available"
  }, {
    "id" : "ed82acd8-42d4-456b-b3ed-930585d45d69",
    "location" : "Annex",
    "callNumber" : "F TOLKIEN U alsoTOLKIEN",
    "status" : "Available"
  }, {
    "id" : "8866c05e-164b-42bd-9628-1d42cd4540ec",
    "location" : "Annex",
    "callNumber" : "HOB",
    "status" : "Available"
  }, {
    "id" : "0d96635c-c0ca-44f7-bd95-bb1dd2733fc5",
    "location" : "Annex",
    "callNumber" : "YAF TOLKIEN",
    "status" : "Available"
  }, {
    "id" : "6288b86f-6201-4555-a0c4-be458ce18b26",
    "location" : "Annex",
    "callNumber" : "YAF TOLKIEN",
    "status" : "Available"
  }, {
    "id" : "96739c43-7bcf-482a-8ab0-41e32fbac5ef",
    "location" : "Main Library",
    "status" : "Available"
  } ]
}

$ curl $OKAPI/rtac/7f380ee2-67d8-4d00-8435-a3af64b10b31 -H "X-Okapi-Token: $TOKEN" -w'\n'
{
  "holdings" : [ {
    "id" : "0d96635c-c0ca-44f7-bd95-bb1dd2733fc5",
    "location" : "Annex",
    "callNumber" : "YAF TOLKIEN",
    "status" : "Available"
  }, {
    "id" : "6288b86f-6201-4555-a0c4-be458ce18b26",
    "location" : "Annex",
    "callNumber" : "YAF TOLKIEN",
    "status" : "Available"
  }, {
    "id" : "96739c43-7bcf-482a-8ab0-41e32fbac5ef",
    "location" : "Main Library",
    "status" : "Available"
  }, {
    "id" : "8866c05e-164b-42bd-9628-1d42cd4540ec",
    "location" : "Annex",
    "callNumber" : "HOB",
    "status" : "Available"
  }, {
    "id" : "ed82acd8-42d4-456b-b3ed-930585d45d69",
    "location" : "Annex",
    "callNumber" : "F TOLKIEN U alsoTOLKIEN",
    "status" : "Available"
  }, {
    "id" : "14e71ce6-465f-409a-b93a-e8037e4be0c7",
    "location" : "Annex",
    "callNumber" : "HOB2",
    "status" : "Available"
  } ]
}
Comment by Craig McNally [ 10/Apr/20 ]

oh, interesting... I'm seeing the same thing in the UI as well:

Comment by Anya [ 10/Apr/20 ]

This is ticket is on behalf of Simmons who is using EDS as a discovery platform, and Blacklight as its OPAC.

Kelly Drake is Simmons IC and Sean Tivian at EBSCO is their EDS IPM. I believe that the scheme i provided was from the rtac call- but I could be wrong - it was given to me.

I do think it is odd that holding is in random order

Comment by Magda Zacharska [ 10/Apr/20 ]

Thank you Craig McNally for looking into that issue so quickly! I guess the issue of the consistency will need to be resolved on FOLIO's inventory side. Charlotte Whitt - the order of returned locations and items within them?

When it comes to parentheses surrounding volume information - it was a work done for MODRTAC-14 Closed that was needed for Item Level requests in EDS for Chalmers. I'm also curious as Craig in his comments above:

Just curious... Is the problem that they're not shown in the desired order in EDS, or is there another consumer of RTAC that needs the results in a particular order?

Is there a need for a different way of rendering volume information?

Comment by Craig McNally [ 10/Apr/20 ]

Disclaimer - this gets into the weeds a bit....

FWIW it is possible to sort the results when calling the holdings API, but the holdings record itself doesn't directly have anything valuable to sort on - just references (UUIDs) to other records. For instance it has a reference to a location via permanentLocationId, and there's a foreignKey there, but there isn't an alias in the schema.json, so you can't use cross-table subqueries to get/sort holdings by location.name...
i.e. you can't currently do:
GET /holdings-storage/holdings?query=instanceId==5bf370e0-8cca-4d9c-82e4-5170ab2a0a39 and permanentLocation.name=Annex
or
GET /holdings-storage/holdings?query=instanceId==5bf370e0-8cca-4d9c-82e4-5170ab2a0a39 sortBy permanentLocation.name.
However, I don't think it would be hard to add this functionality.

All that aside, I don't think we can assume that holdings and/or items should always be sorted the same way, even within the context of RTAC results... EDS might expect them to appear ordered one way, and another consumer of RTAC might expect them ordered another way, etc. It may make more sense to add the ability to specify how you want results sorted in the RTAC API, which would then be used when retrieving records from inventory.

One more gotcha here - as Matt Reno mentioned to me, volume is derived by mod-rtac, so sorting by volume would need to happen in the code. It couldn't be done by the database.

Comment by Craig McNally [ 10/Apr/20 ]

regarding the searching for holding and sorting by say permanentLocation.name... See https://folio-org.atlassian.net/browse/RMB-454

Comment by Kelly Drake [ 10/Apr/20 ]

Regarding the code snipped supplied by Anya this was indeed from an rtac call to folio (Edelweiss).

Comment by Ann-Marie Breaux (Inactive) [ 04/Oct/21 ]

Hi Anya This is an old bug that was on UXPROD. I moved it to SUP. Not sure if it has been resolved and can be closed, or if it needs attention. Thank you!

Comment by Ann-Marie Breaux (Inactive) [ 30/Nov/21 ]

Hi Khalilah Gambrell Not sure if this belongs to Prokopovych or some other team. It might just need to be moved to a UI or MOD project. Could you review and decide? Thank you!

Comment by Charlotte Whitt [ 03/Jan/22 ]

Support SIG: Hi Anya you are the report on this bug from 4/10/2020 - before we do any further planning on this work, I think we should verify if this bug is still valid. Will you re-test and confirm if it can be reproduced.

Comment by Anya [ 10/Jan/22 ]

will reopen if it comes up again.

Comment by Anya [ 17/Jun/22 ]

Repoening as it is becoming an issue 

Comment by Craig McNally [ 21/Jun/22 ]

It's worth pointing out that this will almost certainly require work in edge-rtac as well.

FWIW similar functionality was added to edge/mod-patron 6 months ago. Using a similar approach here may make sense, though someone will need to take a closer look.

See https://folio-org.atlassian.net/browse/MODPATRON-79 and https://folio-org.atlassian.net/browse/EDGPATRON-62

Comment by Charlotte Whitt [ 19/Sep/22 ]

Support SIG: Charlotte Whitt to pull it into Prokopovych Sprint 149.

Comment by Kyle Felker [ 26/Sep/22 ]

Anya

I need some clarification. You state here:

> In FOLIO they appear in the screenshot order... and that is what is expected in the RTAC results

The screenshots in this issue are inaccessible, so you may need to re-attach them. I did check the inventory UI of our reference environment (snapshot) and did some experimenting with volume numbers and locations. The inventory UI seems to display items in the order they were created, with the oldest item at the top of the list, and the most recently added at the bottom. This is the same order that seems to be employed in the RTAC listing, and is the ordering I would expect. Volume number and location seems to have no impact at all on order of display.

Edit: Further investigation shows that volume is an alphanumeric field. the following are all acceptable values for that field:
"volume one"
"vol. 1"
"1"
"thrtgrvrbtn"

That makes sorting on that field nearly impossible, since I can't expect a simple numeric value.

Comment by Kyle Felker [ 27/Sep/22 ]

After reading through all the comments on this, I think this issue needs a more clearly-stated scope of work. Is the desire here to add query parameters to the URL specifying a field to sort on that would be passed on to the CQL? Or do you want a hardcoded sort? If the latter, what fields am I sorting on, and in what order? As I point out above, the volume info attached to item records is alphanumeric, so it won't always sort cleanly. Location has been discussed a lot-is that a criteria? If so, I'm assuming effective location? Which one, it can appear in different records (instance, holdings, location)?

Comment by Khalilah Gambrell [ 27/Sep/22 ]

Charlotte Whitt / Kyle Felker  / this story is not ready for development.  Please re-assign to me and set to Draft. 

Comment by Craig McNally [ 27/Sep/22 ]

Kyle Felker I think we need to make this flexible.  Looking at the code, I'm not sure we can take the same approach that we used in mod-patron/edge-patron (see my earlier comment above).  The reason being that mod-rtac is presently getting information from multiple places here, and not all of them accept CQL queries...  

Inventory:  POST /inventory-hierarchy/items-and-holdings – this endpoint does not support CQL.  It takes a payload with one or more instanceIDs and returns holding and items associated with that instance.  

Circulation: GET /loan-storage/loans – this API does support CQL (and sorting), but really I think the only thing we're getting from this is the "status" info.  I'm going to guess that sorting by status does not satisfy the requirements, but I'll defer to Anya and others on that.

So, assuming we need to sort by fields other than just status, the way I see it the options are as follows: 

Allow sort to be specified in the RTAC request and...

  1. Continue to call POST /inventory-hierarchy/items-and-holdings and perform the sort in mod-rtac (as opposed to delegating sorting to the storage module/database) prior to forming a response
  2. Adjust mod-rtac to call different APIs which support CQL, e.g. GET /item-storage/items & GET /holdings-storage/holdings
  3. Delegate sorting to POST /inventory-hierarchy/items-and-holdings.  Update that endpoint to allow for a sort clause to be specified, and pass along the sort clause provided in the RTAC request.
  4. Maybe there are other options I'm missing... ?

Other considerations:

  • Performance... will the approach hurt performance?
  • I suspect there will also be a request (if there isn't already) to support filtering of RTAC results (e.g. only return holdings w/ certain locations, etc.).  Technically this doesn't preclude any of the options listed above, but I thought it was worth mentioning.
  • Backwards compatibility.  It's crucial that we maintain backwards compatibility so we're not breaking existing integrations with FOLIO RTAC.  Fortunately, these integrations work through edge-rtac, so we have an isolation layer there which can adapt to some breaking changes in mod-rtac, but this is still something to keep in mind.
Comment by Anya [ 28/Nov/22 ]

Support: Khalilah Gambrell  this ticket has been open since April 2020 - support would like to move this to a P2 in order to get more eyes on it, and for it to be fixed. Please update this ticket if this has had progress. 

Comment by Erin Nettifee [ 28/Nov/22 ]

Why can't the discovery layer tool do the sorting? Or is that a dumb question?

Comment by Marie Widigson [ 17/Jan/23 ]

We discussed RTAC sorting of monographs (where all items on one instance are equivalent, not multi-volume monographs) in the Discovery group with Khalilah Gambrell today. In the group we encouraged institutions to provide input on what we would like RTAC to sort on. Having a custimizable primary and secondary sorting was also discussed. 

So here is some quick sorting input from Chalmers. 

  • Item status (Available first. Not sure about the rest, but same status kept together)
  • Effective location (Based on hierarchy Campus - Library - Location. Perhaps in customizable order)

and maybe also

  • Item loan type

Not sure if we would want to sort on Location or item status primarily+

Some discovery layers displays only the first item and a "See all" option. If a checked out item is displayed first, this could put users off from placing a request, even though there may be available copies further down in the list. Therefore I think we would like to display any of the available items first.

For multi volume instances and serials:
The Enumeration field(s) used by the library.

Comment by Erin Nettifee [ 17/Jan/23 ]

I would add my support for sort based on location and loan type. Item status is also helpful, though I don't know that FOLIO should necessarily be compensating for the behavior of the discovery layer per se, since FOLIO doesn't require a particular discovery layer to work.

Comment by Charlotte Whitt [ 06/Feb/23 ]

Khalilah Gambrell - this work seams a bit super optimistic that this should be done by Orchid. Please update the release target.

Comment by Theodor Tolstoy (One-Group.se) [ 07/Feb/23 ]

It feels like this has been said before, but I could not see it.

Since the response is in XML, you by definition cannot rely on the order of the nodes in the XML response, right? 

Also, I have not been involved in the discussions about the sorting in the community, but I recently wen through a couple of scenarios with a colleague and we found valid use cases for sorting on almost all of the properties.

Erin Nettifee I agree that this logic should reside in the Discovery layer for the above mentioned reasons.

Comment by Kyle Felker [ 08/Feb/23 ]

Theodor Tolstoy (One-Group.se)

The inventory UI seems to display items in the order they were created, with the oldest item at the top of the list, and the most recently added at the bottom. This is the same order that seems to be employed in the RTAC listing, and is the ordering I would expect. I think the ordering is consistent, it's just not the type of ordering people want.

Comment by Debra Howell [ 20/Mar/23 ]

Khalilah Gambrell FROM SUPPORT SIG: Please assign a release for this bug.

Comment by Khalilah Gambrell [ 22/Mar/23 ]

Hey Debra Howell - I know this is desired but I cannot assign a release until I have time to write and validate requirements.

Comment by Anya [ 24/Apr/23 ]

Support: THe dev team is not decided yet .. Khalilah Gambrell 

Comment by Brooks Travis [ 30/Apr/23 ]

Is this really a bug?

Generated at Fri Feb 09 00:23:28 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.