[MODINVSTOR-650] Bugfest - will solve using Elasticsearch. Item segment. Filtering by Effective location and Item Status returns unexpected additional records Created: 17/Dec/20  Updated: 01/Nov/21  Resolved: 01/Nov/21

Status: Closed
Project: mod-inventory-storage
Components: None
Affects versions: None
Fix versions: None

Type: Bug Priority: P3
Reporter: Lisa Sjögren Assignee: Unassigned
Resolution: Won't Do Votes: 0
Labels: elastic-search, support
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: PNG File Skärmavbild 2020-12-17 kl. 19.53.23.png     PNG File Skärmavbild 2020-12-17 kl. 19.53.43.png     PNG File Skärmavbild 2020-12-17 kl. 21.02.21.png     PNG File Skärmavbild 2020-12-17 kl. 21.22.06.png     PNG File Skärmavbild 2020-12-17 kl. 21.40.03.png     PNG File Skärmavbild 2021-02-02 kl. 13.02.14.png     PNG File Skärmavbild 2021-02-02 kl. 13.04.31.png     PNG File instance-item-based-view.png     PNG File screenshot-1.png     PNG File screenshot-2.png     PNG File screenshot-3.png     PNG File screenshot-4.png    
Issue links:
Relates
relates to MSEARCH-47 Item - combine Item's effective locat... Closed
relates to UIIN-1343 Sort on/off option for Instance Query... Draft
relates to UIIN-1343 Sort on/off option for Instance Query... Draft
Sprint: Prokopovych - Sprint 106, Prokopovych - Sprint 105
Development Team: Prokopovych
Affected Institution:
Chalmers, Simmons
Tester Assignee: Charlotte Whitt

 Description   

Overview:
I want to get a list of all items at a certain library location that have status missing. I search in Inventory by selecting the Item segment, and filtering on Item Status and Effective location. However, this search unexpectedly also returns titles with items missing at other library location, in cases where the given instance has multiple items, and other items than the missing items, are at the location I searched for.

Customer Priority is set to be Low since Chalmers currently has a workaround using APIs for creating lists of missing items for each library. It would have been prioritized higher had it been possible to display item data – status + data identifying and locating the item – in the results list (which would make Inventory an attractive option for generating and working with precise lists of e.g. missing items).

Steps to reproduce:

  1. Login to FOLIO Snapshot, go to Inventory and select the Item segment
  2. Select Item status Missing and Effective location Main Library
  3. This is the URL: https://folio-snapshot.dev.folio.org/inventory/view/f31a36de-fcf8-44f9-87ef-a55d06ad21ae?filters=effectiveLocation.fcd64ce1-6995-48f0-840e-89ffa2288371%2CitemStatus.Missing&segment=items&sort=title
    {{okapi-url}}/inventory/instances?limit=100&query=(item.status.name=="Missing" and item.effectiveLocationId="fcd64ce1-6995-48f0-840e-89ffa2288371") sortby title
    

Expected result:
The result list contains only instances with item which are missing AND at the selected library (location). In this case, instances which have at least one item that has both effectiveLocation Main library and status "Missing". The number of instances returned should only be 1 - The semantic web.

Actual result:
The result list contains not only instances with items which are missing at the chosen library location, but also titles where the item at the selected location has a different Item status, but mulitiple items and one of these item has the item status as missing.
In this case: all instances which have at least one item with effectiveLocation Main library and at least one item with status Missing.

Additional comments:
The search a staff thinks he/she is doing is find all items matching both criteria, but the search is looking for items matching both criteria, and also instance with multiple items, where some items matches one criteria, and other items matches the second criteria.

Marc Johnson wrote up following scope for the expectations:

  • Any criteria expressed for an item must match the same item
  • Any criteria expressed for a holdings record must match the same holdings record
  • If both item and holdings criteria is included, then the item must also be part of that holding
  • it must be able to work with large sets of search results (maybe in the 1000s?)
  • it must allow the same sorting that UI inventory does at the moment

Interested parties: Maryam Vardeh (Chalmers)



 Comments   
Comment by Kelly Drake [ 17/Dec/20 ]

Charlotte Whitt - Is this something you are aware of?

Comment by Charlotte Whitt [ 17/Dec/20 ]

Kelly Drake, Lisa Sjögren - no this is a new bug, which I have not heard about until now.
Which release are you testing in Goldenrod? By looking at your screendumps, it can not be Honeysuckle.

This problem is not about that the item segment is selected but actually searching instances ... but I would need to look into this, and see if I can mimik the error in Bugfest Honeysuckle.

Comment by Charlotte Whitt [ 17/Dec/20 ]

Bugfest Honeysuckle behaves as expected:
I marked three items having the effective location: AcqMono BO and item state: Missing

Steps:

  1. In the item segement
  2. When filtering first on: Item state Missing, then I get 38 results found
  3. Then filtered on the location: AcqMono BO, then I get 3 results as expected - https://bugfest-honeysuckle.folio.ebsco.com/inventory?filters=itemStatus.Missing%2CeffectiveLocation.39a16452-b2da-490b-8148-41c09781e4ab&segment=items&sort=title
    So I get the expected result, and can not reproduce the error.

Lisa Sjögren Could you maybe try the same steps in the environment you are testing, and check if you still see any errors.

Please notice, that the result list always will return a list of instances, and then you need to verify each result for which of the instance has the item status missing. (That's the current display, but only an interim solution).

Comment by Lisa Sjögren [ 17/Dec/20 ]

Hi Charlotte Whitt! Sorry for not being clear about the version. I was testing in Snapshot Stable.

Comment by Charlotte Whitt [ 17/Dec/20 ]

Hi Lisa Sjögren you're screen dump looks a bit old. Please notice that FOLIO Snapshot Stable is not maintained since 9/29/2020 - https://dashboard.folio.ebsco.com/

Try FOLIO Snapshot instead: https://folio-snapshot.dev.folio.org/

Comment by Lisa Sjögren [ 17/Dec/20 ]

Charlotte Whitt Are you sure that, when you did your search, there existed a title where AcqMono BO's item was available but another location's item was missing?

When I tried your search in Honeysuckle I did initially only get the three results, but once I added a missing item for another location to an instance that has an available AcqMono BO item, that title showed up in the result list.

SInce AcqMono BO item is not missing, I would not expect that to be returned when I filter on Missing and AcqMono BO.

Comment by Charlotte Whitt [ 17/Dec/20 ]

And you did clear your searches between the two searches? That's important in order to make sure there is nothing in the catch spoking?

Comment by Charlotte Whitt [ 17/Dec/20 ]

It works for me in Bugfest Honeysuckle. Now I don't get the missing item with 'new' location. Do you want me to call you, and I can share my screen.

Comment by Lisa Sjögren [ 17/Dec/20 ]

Thanks for the Snapshot link, Charlotte Whitt! I'll use that one in the future.

It looks like this one and Stable have the same okapi URL though – folio-snapshot-okapi.dev.folio.org – so I would assume that the API behaviour would have been the same.

I am getting the same result:

Comment by Charlotte Whitt [ 17/Dec/20 ]

Yes, and the title Semantic web, has also a Missing item and location Main library right. Then that's the expected result.
The item detail list shows all items, not just the Missing item.

That's the way it works right now, until we can get it refined, right.
(Having this improved, I have fought for all year, but the developer resources were never available to improve Inventory's MCL result list, or even better implement hierarchical display of the Result list)

https://folio-snapshot.dev.folio.org/inventory/view/5bf370e0-8cca-4d9c-82e4-5170ab2a0a39?filters=itemStatus.Missing%2CeffectiveLocation.fcd64ce1-6995-48f0-840e-89ffa2288371&segment=items&sort=title

Comment by Lisa Sjögren [ 17/Dec/20 ]

Charlotte Whitt I'm in another meeting, but I think we are speaking past each other.

The issue is not with the Semantic Web, but The Girl on the Train. Main Library's copy of that title is not missing.

Where the use case is to find all missing items at a certain library, titles where only another library's copy is missing are not relevant.

Comment by Charlotte Whitt [ 17/Dec/20 ]

Lisa Sjögren - re.

It looks like this one and Stable have the same okapi URL though

Says nothing about the build and the current version of the software. Here you need to check: https://folio-snapshot.dev.folio.org/settings/about

Comment by Charlotte Whitt [ 17/Dec/20 ]

The issue is not with the Semantic Web, but The Girl on the Train. Main Library's copy of that title is not missing.

okay, let me check

Comment by Charlotte Whitt [ 17/Dec/20 ]

Lisa Sjögren - I know the explenaition now:

I tried a new search in Bugfest Honeysuckle (and Bugfest Snapshot - is the same). The search do correctly search: All items missing and effective location, e.g. acq bind https://bugfest-honeysuckle.folio.ebsco.com/inventory/view/22f98c5a-d4ea-433f-81fe-7859c5a9ea9d?filters=itemStatus.Missing%2CeffectiveLocation.fac5de34-26ee-456d-86b1-f04fdf680d65&segment=items&sort=title*https://bugfest-honeysuckle.folio.ebsco.com/

But because one specific instance has two items

  1. Item status Missing at location Book stacks and
  2. Item status In process at acq bind.
    Then the search do not find an item with Item status Missing at acq bind, but find one item with item status missing, and another item at location acq bind.
    The result list displays it as one out of three instances matching the search criteria.

That's a bug, and we must look into how to fix that.
I'll clean up the reproduction steps, so the confusion of the different environments, and possible use of API get cleared out.

Comment by Lisa Sjögren [ 18/Dec/20 ]

Great! That is exactly what I was trying to say.

I do think it's worth noting that the query

{{okapi-url}}/inventory/items?limit=100&query=(status.name=="Missing" and effectiveLocationId="fcd64ce1-6995-48f0-840e-89ffa2288371") sortby title

does return the expected results, in case a library that runs into this decides to do the search outside the UI.

Comment by Charlotte Whitt [ 18/Dec/20 ]

Cool, sounds all good. I'll see if I can get it groomed at today's Core Functional meeting.

CC: Holly Mistlebauer

Comment by Zak Burke [ 18/Dec/20 ]

This looks to me like a back-end issue. e.g. the query generated for an item query when both "Item status" and "Effective location" filters are present contains an "and" clause like this:

GET https://folio-snapshot-okapi.dev.folio.org/inventory/instances?limit=100&query=%28item.status.name%3D%3D%22Available%22%20and%20item.effectiveLocationId%3D%3D%22fcd64ce1-6995-48f0-840e-89ffa2288371%22%29%20sortby%20title

{
  "instances" : [ {
    "@context" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/context",
    "id" : "69640328-788e-43fc-9c3c-af39e243f3b7",
    "hrid" : "inst000000000001",
    "source" : "FOLIO",
    "title" : "ABA Journal",
    "alternativeTitles" : [ ],
    "editions" : [ ],
    "series" : [ ],
    "identifiers" : [ {
      "identifierTypeId" : "913300b2-03ed-469a-8179-c1092c991227",
      "value" : "0747-0088"
    }, {
      "identifierTypeId" : "c858e4f2-2b6b-4385-842b-60732ee14abb",
      "value" : "84641839"
    } ],
    "contributors" : [ ],
    "subjects" : [ ],
    "classifications" : [ ],
    "publication" : [ {
      "publisher" : "American Bar Association",
      "place" : "Chicago, Ill.",
      "dateOfPublication" : "1915-1983",
      "role" : null
    } ],
    "publicationFrequency" : [ ],
    "publicationRange" : [ ],
    "electronicAccess" : [ ],
    "instanceTypeId" : "6312d172-f0cf-40f6-b27d-9fa8feaf332f",
    "instanceFormatIds" : [ ],
    "physicalDescriptions" : [ ],
    "languages" : [ ],
    "notes" : [ ],
    "discoverySuppress" : false,
    "statisticalCodeIds" : [ ],
    "statusUpdatedDate" : "2020-12-18T01:43:49.852+0000",
    "metadata" : {
      "createdDate" : "2020-12-18T01:43:49.852+00:00",
      "createdByUserId" : null,
      "updatedDate" : "2020-12-18T01:43:49.852+00:00",
      "updatedByUserId" : null
    },
    "tags" : {
      "tagList" : [ ]
    },
    "natureOfContentTermIds" : [ "0abeee3d-8ad2-4b04-92ff-221b4fce1075" ],
    "links" : {
      "self" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/69640328-788e-43fc-9c3c-af39e243f3b7"
    }
  }, {
    "@context" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/context",
    "id" : "7fbd5d84-62d1-44c6-9c45-6cb173998bbd",
    "hrid" : "inst000000000006",
    "source" : "FOLIO",
    "title" : "Bridget Jones's Baby: the diaries",
    "alternativeTitles" : [ ],
    "editions" : [ "First American Edition" ],
    "series" : [ ],
    "identifiers" : [ {
      "identifierTypeId" : "5d164f4b-0b15-4e42-ae75-cfcf85318ad9",
      "value" : "ocn956625961"
    } ],
    "contributors" : [ {
      "contributorNameTypeId" : "2b94c631-fca9-4892-a730-03ee529ffe2a",
      "name" : "Fielding, Helen",
      "contributorTypeId" : null,
      "contributorTypeText" : null,
      "primary" : null
    } ],
    "subjects" : [ "Jones, Bridget", "Pregnant women", "England", "Humorous fiction", "Diary fiction" ],
    "classifications" : [ {
      "classificationNumber" : "PR6056.I4588",
      "classificationTypeId" : "ce176ace-a53e-4b4d-aa89-725ed7b2edac"
    } ],
    "publication" : [ {
      "publisher" : "Alfred A. Knopf",
      "place" : "New York",
      "dateOfPublication" : "2016",
      "role" : null
    } ],
    "publicationFrequency" : [ "A frequency description" ],
    "publicationRange" : [ "A publication range" ],
    "electronicAccess" : [ {
      "uri" : "http://www.folio.org/",
      "linkText" : "Electronic resource (audio streaming)",
      "materialsSpecification" : "Novel",
      "publicNote" : "Access to audio file",
      "relationshipId" : null
    } ],
    "instanceTypeId" : "6312d172-f0cf-40f6-b27d-9fa8feaf332f",
    "instanceFormatIds" : [ ],
    "physicalDescriptions" : [ "219 pages ; 20 cm." ],
    "languages" : [ "eng" ],
    "notes" : [ {
      "instanceNoteTypeId" : "6a2533a7-4de2-4e64-8466-074c2fa9308c",
      "note" : "Bridget Jones finds herself unexpectedly pregnant at the eleventh hour. However, her joyful pregnancy is dominated by one crucial but awkward question --who is the father? Could it be honorable, decent, notable human rights lawyer, Mark Darcy? Or, is it charming, witty, and totally despicable, Daniel Cleaver?",
      "staffOnly" : false
    } ],
    "previouslyHeld" : true,
    "staffSuppress" : true,
    "discoverySuppress" : true,
    "statisticalCodeIds" : [ ],
    "statusUpdatedDate" : "2020-12-18T01:43:49.701+0000",
    "metadata" : {
      "createdDate" : "2020-12-18T01:43:49.701+00:00",
      "createdByUserId" : null,
      "updatedDate" : "2020-12-18T01:43:49.701+00:00",
      "updatedByUserId" : null
    },
    "tags" : {
      "tagList" : [ ]
    },
    "natureOfContentTermIds" : [ ],
    "links" : {
      "self" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/7fbd5d84-62d1-44c6-9c45-6cb173998bbd"
    }
  }, {
    "@context" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/context",
    "id" : "eb5dc7cf-5c81-4172-b2e3-f0cc30dc1c6c",
    "hrid" : "in00000000005",
    "source" : "FOLIO",
    "title" : "New test title",
    "alternativeTitles" : [ ],
    "editions" : [ ],
    "series" : [ ],
    "identifiers" : [ ],
    "contributors" : [ ],
    "subjects" : [ ],
    "classifications" : [ ],
    "publication" : [ ],
    "publicationFrequency" : [ ],
    "publicationRange" : [ ],
    "electronicAccess" : [ ],
    "instanceTypeId" : "a2c91e87-6bab-44d6-8adb-1fd02481fc4f",
    "instanceFormatIds" : [ ],
    "physicalDescriptions" : [ ],
    "languages" : [ ],
    "notes" : [ ],
    "previouslyHeld" : false,
    "staffSuppress" : false,
    "discoverySuppress" : false,
    "statisticalCodeIds" : [ ],
    "statusUpdatedDate" : "2020-12-18T02:14:17.089+0000",
    "metadata" : {
      "createdDate" : "2020-12-18T02:14:17.089+00:00",
      "createdByUserId" : "a5a9f6c5-30c1-57d6-b4a8-0fe0d40bb319",
      "updatedDate" : "2020-12-18T02:14:17.089+00:00",
      "updatedByUserId" : "a5a9f6c5-30c1-57d6-b4a8-0fe0d40bb319"
    },
    "tags" : {
      "tagList" : [ ]
    },
    "natureOfContentTermIds" : [ ],
    "links" : {
      "self" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/eb5dc7cf-5c81-4172-b2e3-f0cc30dc1c6c"
    }
  }, {
    "@context" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/context",
    "id" : "cf23adf0-61ba-4887-bf82-956c4aae2260",
    "hrid" : "inst000000000024",
    "source" : "FOLIO",
    "title" : "Temeraire",
    "alternativeTitles" : [ ],
    "editions" : [ ],
    "series" : [ ],
    "identifiers" : [ {
      "identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
      "value" : "1447294130"
    }, {
      "identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
      "value" : "9781447294130"
    } ],
    "contributors" : [ {
      "contributorNameTypeId" : "2b94c631-fca9-4892-a730-03ee529ffe2a",
      "name" : "Novik, Naomi",
      "contributorTypeId" : null,
      "contributorTypeText" : null,
      "primary" : null
    } ],
    "subjects" : [ ],
    "classifications" : [ ],
    "publication" : [ ],
    "publicationFrequency" : [ ],
    "publicationRange" : [ ],
    "electronicAccess" : [ ],
    "instanceTypeId" : "6312d172-f0cf-40f6-b27d-9fa8feaf332f",
    "instanceFormatIds" : [ ],
    "physicalDescriptions" : [ ],
    "languages" : [ ],
    "notes" : [ ],
    "discoverySuppress" : false,
    "statisticalCodeIds" : [ ],
    "statusUpdatedDate" : "2020-12-18T01:43:50.025+0000",
    "metadata" : {
      "createdDate" : "2020-12-18T01:43:50.025+00:00",
      "createdByUserId" : null,
      "updatedDate" : "2020-12-18T01:43:50.025+00:00",
      "updatedByUserId" : null
    },
    "tags" : {
      "tagList" : [ ]
    },
    "natureOfContentTermIds" : [ ],
    "links" : {
      "self" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/cf23adf0-61ba-4887-bf82-956c4aae2260"
    }
  }, {
    "@context" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/context",
    "id" : "30fcc8e7-a019-43f4-b642-2edc389f4501",
    "hrid" : "inst000000000003",
    "source" : "FOLIO",
    "title" : "The American Journal of Medicine",
    "alternativeTitles" : [ {
      "alternativeTitleTypeId" : null,
      "alternativeTitle" : "The American journal of medicine (online)"
    }, {
      "alternativeTitleTypeId" : null,
      "alternativeTitle" : "Am. J. med"
    }, {
      "alternativeTitleTypeId" : null,
      "alternativeTitle" : "Green journal"
    } ],
    "editions" : [ ],
    "series" : [ ],
    "identifiers" : [ {
      "identifierTypeId" : "593b78cb-32f3-44d1-ba8c-63fd5e6989e6",
      "value" : "AJMEAZ"
    }, {
      "identifierTypeId" : "913300b2-03ed-469a-8179-c1092c991227",
      "value" : "0002-9343"
    }, {
      "identifierTypeId" : "c858e4f2-2b6b-4385-842b-60732ee14abb",
      "value" : "med49002270"
    } ],
    "contributors" : [ ],
    "subjects" : [ "Clinical medicine-Periodicals", "Medicine", "Geneeskunde" ],
    "classifications" : [ {
      "classificationNumber" : "RC60 .A5",
      "classificationTypeId" : "ce176ace-a53e-4b4d-aa89-725ed7b2edac"
    }, {
      "classificationNumber" : "W1 AM493",
      "classificationTypeId" : "a7f4d03f-b0d8-496c-aebf-4e9cdb678200"
    } ],
    "publication" : [ {
      "publisher" : "Dun-Donnelley Pub. Co. ",
      "place" : "New York",
      "dateOfPublication" : "1946-",
      "role" : null
    } ],
    "publicationFrequency" : [ ],
    "publicationRange" : [ ],
    "electronicAccess" : [ ],
    "instanceTypeId" : "6312d172-f0cf-40f6-b27d-9fa8feaf332f",
    "instanceFormatIds" : [ ],
    "physicalDescriptions" : [ "v., ill. 27 cm." ],
    "languages" : [ "eng" ],
    "notes" : [ {
      "instanceNoteTypeId" : "6a2533a7-4de2-4e64-8466-074c2fa9308c",
      "note" : "Print subscription cancelled by Dec. 2016.",
      "staffOnly" : false
    }, {
      "instanceNoteTypeId" : "6a2533a7-4de2-4e64-8466-074c2fa9308c",
      "note" : "May 1988-: A Yorke medical. Also known as the Green journal",
      "staffOnly" : false
    }, {
      "instanceNoteTypeId" : "6a2533a7-4de2-4e64-8466-074c2fa9308c",
      "note" : "Publisher: Excerpta Medica, 2008-; New York, NY : Elsevier Inc. 2013-",
      "staffOnly" : false
    }, {
      "instanceNoteTypeId" : "6a2533a7-4de2-4e64-8466-074c2fa9308c",
      "note" : "Supplements issued irregularly, 1982.",
      "staffOnly" : false
    }, {
      "instanceNoteTypeId" : "6a2533a7-4de2-4e64-8466-074c2fa9308c",
      "note" : "Official journal of the Association of Professors of Medicine 2005-",
      "staffOnly" : false
    }, {
      "instanceNoteTypeId" : "6a2533a7-4de2-4e64-8466-074c2fa9308c",
      "note" : "Indexed quinquennially in: American journal of medicine 5 year cumulative index",
      "staffOnly" : false
    } ],
    "discoverySuppress" : false,
    "statisticalCodeIds" : [ ],
    "statusUpdatedDate" : "2020-12-18T01:43:49.645+0000",
    "metadata" : {
      "createdDate" : "2020-12-18T01:43:49.645+00:00",
      "createdByUserId" : null,
      "updatedDate" : "2020-12-18T01:43:49.645+00:00",
      "updatedByUserId" : null
    },
    "tags" : {
      "tagList" : [ ]
    },
    "natureOfContentTermIds" : [ "0abeee3d-8ad2-4b04-92ff-221b4fce1075" ],
    "links" : {
      "self" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/30fcc8e7-a019-43f4-b642-2edc389f4501"
    }
  }, {
    "@context" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/context",
    "id" : "f31a36de-fcf8-44f9-87ef-a55d06ad21ae",
    "hrid" : "inst000000000012",
    "source" : "FOLIO",
    "title" : "The Girl on the Train",
    "alternativeTitles" : [ {
      "alternativeTitleTypeId" : null,
      "alternativeTitle" : "First alternative title"
    }, {
      "alternativeTitleTypeId" : null,
      "alternativeTitle" : "Second alternative title"
    } ],
    "editions" : [ ],
    "series" : [ ],
    "identifiers" : [ {
      "identifierTypeId" : "7f907515-a1bf-4513-8a38-92e1a07c539d",
      "value" : "B01LO7PJOE"
    } ],
    "contributors" : [ {
      "contributorNameTypeId" : "2e48e713-17f3-4c13-a9f8-23845bb210aa",
      "name" : "Creator A",
      "contributorTypeId" : null,
      "contributorTypeText" : null,
      "primary" : null
    }, {
      "contributorNameTypeId" : "e8b311a6-3b21-43f2-a269-dd9310cb2d0a",
      "name" : "Creator B",
      "contributorTypeId" : null,
      "contributorTypeText" : null,
      "primary" : null
    } ],
    "subjects" : [ ],
    "classifications" : [ ],
    "publication" : [ ],
    "publicationFrequency" : [ ],
    "publicationRange" : [ ],
    "electronicAccess" : [ ],
    "instanceTypeId" : "6312d172-f0cf-40f6-b27d-9fa8feaf332f",
    "instanceFormatIds" : [ ],
    "physicalDescriptions" : [ ],
    "languages" : [ ],
    "notes" : [ ],
    "discoverySuppress" : false,
    "statisticalCodeIds" : [ ],
    "statusUpdatedDate" : "2020-12-18T01:43:49.683+0000",
    "metadata" : {
      "createdDate" : "2020-12-18T01:43:49.683+00:00",
      "createdByUserId" : null,
      "updatedDate" : "2020-12-18T01:43:49.683+00:00",
      "updatedByUserId" : null
    },
    "tags" : {
      "tagList" : [ ]
    },
    "natureOfContentTermIds" : [ "44cd89f3-2e76-469f-a955-cc57cb9e0395" ],
    "links" : {
      "self" : "http://folio-snapshot-okapi.dev.folio.org/inventory/instances/f31a36de-fcf8-44f9-87ef-a55d06ad21ae"
    }
  } ],
  "totalRecords" : 6
}
Comment by Kelly Drake [ 04/Jan/21 ]

Support SIG - this issue would appear to effect all production libraries. Should it be higher priority?

Comment by Charlotte Whitt [ 04/Jan/21 ]

Yes, Kelly Drake - the ranking was based on Chalmers comment re. their work around, but do agree, and will bump it up.

CC: Holly Mistlebauer Marc Johnson Zak Burke

Comment by Marc Johnson [ 05/Jan/21 ]

Holly Mistlebauer Charlotte Whitt

This has been placed in Sprint 105 for the Core Functional team. Does that mean that the team should be actively picking up this issue?

If so, is it more important than the upgrading of modules to RAML Module Builder 32 for 2021 R1?

Comment by Marc Johnson [ 05/Jan/21 ]

I agree with Zak Burke's statement above that this is likely a back end issue.

It was estimated on the 18th December (at a session I wasn't present for) yet the issue was not moved to a back end project. Can anyone provide me context as to whether the estimate was based upon it being front end or back end work?

Comment by Marc Johnson [ 05/Jan/21 ]

If I were to speculate (and we would want to investigate and confirm this), my thought would be that this CQL translates to find any instance where there is an item that is missing and an item whose effective location is the chosen location, however these might not be the same item.

If this turns out to be the case, this would be a side effect of how multi-record CQL is translated to SQL and would likely need Core Platform involvement to resolve.

Comment by Charlotte Whitt [ 05/Jan/21 ]

I think you are right Marc Johnson (the comment above) - should we ask if Julian Ladisch could help out here?

Comment by Holly Mistlebauer [ 05/Jan/21 ]

Marc Johnson: Chalmers has a workaround, so I would not describe this as a higher priority than upgrading modules for RAML.

Comment by Marc Johnson [ 08/Jan/21 ]

Holly Mistlebauer

Chalmers has a workaround, so I would not describe this as a higher priority than upgrading modules for RAML.

Should it be in the current sprint for developers to pick up?

Comment by Charlotte Whitt [ 08/Jan/21 ]

Yes, good point Marc Johnson. This is very important to get fixed, while it affects all libraries in production, and of course also all libraries testing their sandbox environments, preparing to go live some time this year.

Comment by Marc Johnson [ 08/Jan/21 ]

Charlotte Whitt Holly Mistlebauer

This is very important to get fixed, while it affects all libraries in production, and of course also all libraries testing their sandbox environments, preparing to go live some time this year.

Does that mean it should be in the current sprint and take priority over other back end work for 2021 R1?

Comment by Charlotte Whitt [ 11/Jan/21 ]

Marc Johnson - it means that if at all possible we should fit it into the current sprint, or the following sprint. And we definitely should solve it for 2021 R1.
The libraries in production are all affected by this, and the large libraries as uChicago, Cornell and others will also be very much affected by this, if we do not get it solved.

Comment by Marc Johnson [ 11/Jan/21 ]

Charlotte Whitt Thank you for providing more context for the prioritisation of this.

it means that if at all possible we should fit it into the current sprint, or the following sprint. And we definitely should solve it for 2021 R1.

The libraries in production are all affected by this, and the large libraries as uChicago, Cornell and others will also be very much affected by this, if we do not get it solved.

Holly Mistlebauer

What do you want to do? Should we leave this in the current sprint?

If the cause of this turns out to be the SQL generated from the CQL query, there might be a significant lead time to resolve this (assuming we can).

If the resolution requires changes to RAML Module Builder it is unlikely that these will be completed in time for the freeze on the 15th January.

Comment by Julian Ladisch [ 11/Jan/21 ]

Marc is correct that this CQL query

{{okapi-url}}/inventory/instances?limit=100&query=(item.status.name=="Missing" and item.effectiveLocationId="fcd64ce1-6995-48f0-840e-89ffa2288371") sortby title

is translated into this this database query:
Find an instance where

  • that instance has at least one item that is missing and
  • that instance has at least one item that has the effective location.

If "item.*" is used several times it can refer to different items of the instance.
This is by design and intended because it is a query against the /inventory/instances API.

If multiple filters should apply to the same item use the /inventory/items API.

Lisa already provided the solution:

{{okapi-url}}/inventory/items?limit=100&query=(status.name=="Missing" and effectiveLocationId="fcd64ce1-6995-48f0-840e-89ffa2288371") sortby title
Comment by Marc Johnson [ 11/Jan/21 ]

Julian Ladisch

Marc is correct

Thanks, I was just setting out to investigate and replicate this.

If multiple filters should apply to the same item use the /inventory/items API

As convenient a solution as that would be, I don't think that would meet the expected behaviour.

As I understand it, all searches in the inventory UI are intended to find instances not items, which I think would mean this approach isn't appropriate as that API responds with items.

Charlotte Whitt Holly Mistlebauer Zak Burke Is this the case? Is it a sensible understanding that the behaviour we are trying to achieve is to find all instances which are associated to an item with a given status and effective location?

Comment by Charlotte Whitt [ 11/Jan/21 ]

Hi Marc Johnson - yes, that's correct:

Charlotte Whitt Holly Mistlebauer Zak Burke Is this the case? Is it a sensible understanding that the behaviour we are trying to achieve is to find all instances which are associated to an item with a given status and effective location?

We have labelled this ticket as a SPIKE, while this work might reveal that this is a general issue when we are performing a search as a combinations of filters.

Comment by Charlotte Whitt [ 11/Jan/21 ]

Marc Johnson - should I assign the ticket to you?

Comment by Marc Johnson [ 11/Jan/21 ]

Charlotte Whitt

We have labelled this ticket as a SPIKE, while this work might reveal that this is a general issue when we are performing a search as a combinations of filters.

I believe Julian Ladisch has confirmed that is the case in his comment above.

Julian Ladisch Would it be reasonable to generalise what you said above that any combination of search clauses e.g. filters that refer to any record type other than instances e.g. items, holdings records will result in the behaviour where each is considered a separate match on these related records?

should I assign the ticket to you?

I don't think so, I just unassigned myself from it after Julian Ladisch confirmed my hypothesis.

Julian Ladisch has outlined that this is intended behaviour in the tooling (RAML Module Builder) that mod-inventory-storage uses. Without changes to that tooling, I don't think it is likely we would be able to meet the expected behaviour.

What are you expecting the next steps to be for this work?

cc: Jakub Skoczen

Comment by Lisa Sjögren [ 11/Jan/21 ]

As I understand it, all searches in the inventory UI are intended to find instances not items, which I think would mean this approach isn't appropriate as that API responds with items.

While that is indeed how inventory works, I think it's worth noting that this "intention" might not be shared by the user. As I pointed out, Chalmers expected to be able to use the combination item segment + effective location filter + status to identify items that belong to a certain location and have a certain status – to a librarian that is a search intended to find items, although Inventory (here, with some difficulty) translates it into a search for instances.

The need to search for specific items in Inventory was also highlighted (so to speak) in https://folio-org.atlassian.net/browse/UIIN-906.

Comment by Marc Johnson [ 11/Jan/21 ]

Lisa Sjögren

While that is indeed how inventory works, I think it's worth noting that this "intention" might not be shared by the user. As I pointed out, Chalmers expected to be able to use the combination item segment + effective location filter + status to identify items that belong to a certain location and have a certain status – to a librarian that is a search intended to find items, although Inventory (here, with some difficulty) translates it into a search for instances.

Agreed, that may well be the case. My understanding comes from Charlotte Whitt's comment above and what we have been asked to build (it saddens me when that doesn't match what folks actually need). Alas, I get to have very little direct contact with potential or actual users of the system :-/

Given Charlotte Whitt answer above and your suggestion that behaviour might not be what is wanted, I think this becomes more of a user needs and experience conversation. Does that make sense? And fit with your understanding?

Comment by Charlotte Whitt [ 11/Jan/21 ]

hmm Lisa Sjögren - I'm sure you are aware that the current solution, displaying instance in the result list is the thin thread implementation using the MCL component. Eventually (multiple fingers crossed we will end up with having search results displaying in a hierarchical display, as envisioned by the UX developers.

Having the search term being highlighted not just for barcodes, but as a general solution, is definitely a vision we share. But that work was paused due to Elastic Search. When we did UIIN-906 Closed the scope was decided to be kept narrow, to only solve barcode highlight - please see my comment more than a year ago:

Charlotte Whitt added a comment - 12/Dec/19 4:36 PM
Hi Cate Boerema - the highlight functionality was exactly what I was talking about, and explaining, when I yesterday demoed the hierarchical search.
The thing is, that this highlight functionality should correspond any searches, not just search on an item barcode - and properly this highlight mechanism is not just relevant in Inventory, but FOLIO wide.
Then this definitely is becoming a feature, and maybe several features. I'd like to get some thoughts from Khalilah Gambrell and John Coburn to hear if this is something they own, or want to define, and then write up stories, which each of the apps are to implement.

We could use search on barcode as a SPIKE for this functionality.

CC: Siska Humlesjö

Comment by Charlotte Whitt [ 11/Jan/21 ]

Marc Johnson - please see my comment to Lisa above.

Lisa Sjögren comments are only to be understood as 'statements' re. where we eventually want to go - and this work is both agreed upon by the MM-SIG and definitely expected to happen long term.
But for now we have a more obvious bug, which we need to solve.

Comment by Lisa Sjögren [ 11/Jan/21 ]

So, this is what I originally wrote in the description and what I understand the overarching, bordering on philosophical, issue to be:

It looks to me like the issue/source for confusion here is that, although the user has selected the Item segment, implying to the user that they'll be searching for items matching selected criteria, what is actually being searched is instances.

So I absolutely think this belongs in a greater discussion about what you should expect to be able to do in Inventory.

The Customer priority for Chalmers on this is Low, so they do not expect it to be fixed soon.

Comment by Charlotte Whitt [ 11/Jan/21 ]

Lisa Sjögren - maybe it's low for Chalmers due to you have a work around, and are can search in the Union Catalogue.
But this is an issue for all libraries in production, and soon to be in production.

We have a Support SIG meeting later today - and I'll loop in Kelly Drake on this.

Comment by Charlotte Whitt [ 11/Jan/21 ]

Lisa Sjögren - I'd be happy anytime to give you a walk through of the planned work for Inventory, Inventory Search, and Inventory UI
Let's do that one-on-one - and not in this specific ticket.

Comment by Lisa Sjögren [ 11/Jan/21 ]

I'm wondering if there needs to be several Customer priority fields, to reflect the fact that different customers have different priorities (which may also be different from development priority). Is this something you have discussed in the Support SIG, Kelly Drake?

hmm Lisa Sjögren - I'm sure you are aware that the current solution, displaying instance in the result list is the thin thread implementation using the MCL component. Eventually (multiple fingers crossed we will end up with having search results displaying in a hierarchical display, as envisioned by the UX developers.

Yes, and I believe that could open up a world of exciting uses for Inventory search – at which point this issue would become much more tangible.

Comment by Marc Johnson [ 11/Jan/21 ]

Charlotte Whitt Lisa Sjögren

But for now we have a more obvious bug, which we need to solve.

It seems to me that we are talking about two different things

  • Within the current behaviour, folks want to be able to find instances associated with an item that matches all of the criteria, not only one of them
  • There is a future need to be able to search for items rather than instances

Is this a reasonable statement?

Is this issue specifically about the former?

Comment by Charlotte Whitt [ 11/Jan/21 ]

Marc Johnson - yes this issue is solemnly about:

Within the current behaviour, folks want to be able to find instances associated with an item that matches all of the criteria, not only one of them

The future need, and when work on these features are to be expected, we can not say anything more concrete about until we get an updated road map for 2021 and 2022.

Comment by Marc Johnson [ 11/Jan/21 ]

Charlotte Whitt Thanks for confirming my understanding.

Based upon Julian Ladisch comments above, we need the involvement of the Core Platform, as (if my understanding is right) resolving this with the current tooling means changing shared behaviour that has been stated to work as expected.

Jakub Skoczen please can you advise how we move this work forward.

Are folks thinking we need to either change the shared behaviour or introduce a new mechanism?

I think there could be some challenges in this, as the search

instances?query=(item.status.name=="Missing" and item.effectiveLocationId="fcd64ce1-6995-48f0-840e-89ffa2288371")

is sufficiently ambiguous that it could legitimately mean either:

  • find instances with an associated item that is missing and has the specified effective location
  • find instances with an associated item that is missing and an item that has the specified effective location

I'm not sure how we would disambiguate this.

Are proposing changing the interpretation of CQL involving multiple record types to have to understand that all of the criteria for a specific record type are intended to be used together.

Comment by Julian Ladisch [ 11/Jan/21 ]

The same ambiguity can happen when querying both holding and item:

instances?query=(item.status.name=="Missing" and holdingsRecords.permanentLocationId="fcd64ce1-6995-48f0-840e-89ffa2288371")

In current CQL implementation this means:
Find an instance that

  • has at least one missing item and
  • has at least one holding with the specified permanent location (this might be a different holding that the holding of the missing item).
Comment by Marc Johnson [ 11/Jan/21 ]

Julian Ladisch

The same ambiguity can happen when querying both holding and item

Agreed. And it gets even more ambiguous, in that the item might not be part of that specific holdings record.

Comment by Julian Ladisch [ 11/Jan/21 ]

No back-end change is needed if the front-end runs this query

{{okapi-url}}/inventory/items?limit=100&query=(status.name=="Missing" and effectiveLocationId="fcd64ce1-6995-48f0-840e-89ffa2288371") sortby title

and displays the result.
This has the additional benefit that an instance is showed two times if it has two missing items at that location.

Comment by Marc Johnson [ 11/Jan/21 ]

Julian Ladisch

No back-end change is needed if the front-end runs this query and displays the result.

Agreed. However I believe Charlotte Whitt has confirmed that this is an unsuitable resolution for this issue as the search still has to be for instances (for the moment).

This has the additional benefit that an instance is showed two times if it has two missing items at that location.

How would that query present instances, it is a query for items?

Comment by Julian Ladisch [ 11/Jan/21 ]

The GET /inventory/items API fetches the instance title and merges it into the item record as a read-only field: https://s3.amazonaws.com/foliodocs/api/mod-inventory/inventory.html#inventory_items_get

Comment by Marc Johnson [ 11/Jan/21 ]

Julian Ladisch

The GET /inventory/items API fetches the instance title and merges it into the item record as a read-only field

It does. I don't think that means this API can be used to present the same search results as the instance API. Is that what you are suggesting?

Also, I thought that only properties on the primary record type could be used for sorting. Have I remembered that correctly?

Comment by Julian Ladisch [ 11/Jan/21 ]

This is correct for RMB CQL in RMB based storage modules.
Business logic modules like mod-inventory may add sort functionality that is not available in RMB CQL but mod-inventory doesn't.
"sortBy title" should be removed from the query:

{{okapi-url}}/inventory/items?limit=100&query=(status.name=="Missing" and effectiveLocationId="fcd64ce1-6995-48f0-840e-89ffa2288371")

This avoids sorting on the "title" field that doesn't exists at the time when running the CQL query against the mod-inventory-storage item API.

Comment by Julian Ladisch [ 11/Jan/21 ]

present the same search results as the instance API

The item record contains the holdingsId. The UI can fetch the holding record that contains the instanceId. The UI can fetch the instance record. This should be sufficient.
If latency is a concern mod-inventory can easily be changed to also take the instance id from the instance record and merge an "instanceId" field into the item record.

Comment by Zak Burke [ 12/Jan/21 ]

Julian Ladisch, yes, I think latency is a concern here, though I'm not certain how moving the instance-id up into the item record would address that. That would allow us to generate an un-ordered list of IDs, but then we'd have to write a second query with those IDs to actually get at the data we wanted. This is problematic for multiple reasons:

  • queries containing > ~40 IDs (I forget the exact number) exhaust the length of the query-string
  • given the initial (potentially batched) list is unsorted, it becomes impossible to correctly sort the follow up queries except by retrieving all batches and sorting them client side.

In other words, we could change mod-inventory as you suggest, but it would (a) require substantial UI work in order to actually function and (b) doing so would go against established UI patterns of loading records in batches and providing a "Load more" button for additional batches which would (c) compound they latency problem.

What I am hearing is basically this: while the end-user expectation is that multiple filters of are joined with "AND" (this is how instance-level filtering works), this is not possible for holdings-level and item-level filters applied to the inventory/instances endpoint (effectively, multiple filters are joined with "OR").

Comment by Julian Ladisch [ 12/Jan/21 ]

Yes, it gets that difficult if the items should be sorted by instance title.
If this requirement is removed it gets simple.
We may sort by item id to ensure a consistent order allowing paging ("Load more").

Comment by Marc Johnson [ 12/Jan/21 ]

Zak Burke

What I am hearing is basically this: while the end-user expectation is that multiple filters of are joined with "AND" (this is how instance-level filtering works), this is not possible for holdings-level and item-level filters applied to the inventory/instances endpoint (effectively, multiple filters are joined with "OR").

That is basically my understanding.

Julian Ladisch stated that this behaviour is the intended behaviour provided by the shared tooling (RAML Module Builder), which means that we would need to change that across the board to change it.

Comment by Marc Johnson [ 12/Jan/21 ]

Julian Ladisch

Yes, it gets that difficult if the items should be sorted by instance title.
If this requirement is removed it gets simple.

Do you mean instance rather than items, as the search presents instances not items?

I understand you are trying to come up with solutions that don't involve changing that shared underlying tooling and appreciate the endeavour. My concern is that the solutions presented have compromises that when discussed previously with product owners, have been unacceptable e.g. disabling sorting search results by the properties of the instance presented.

Comment by Julian Ladisch [ 12/Jan/21 ]

Do you mean instance rather than items, as the search presents instances not items?

I mean that the search run against the item endpoint as proposed by Lisa is a solution that can be implemented without any back-end development. However, it requires that title sort is disabled. If the result list should show the instance title there will be some latency.

Running the query against the item endpoint is the natural way when all sub-expressions should match the same item.

This solution requires some changes in the UI but has low development effort, it can be released earlier than other solutions.

It's up to the implementing organizations and the PO to weight the pros and cons.

Comment by Marc Johnson [ 12/Jan/21 ]

Julian Ladisch

I mean that the search run against the item endpoint as proposed by Lisa is a solution that can be implemented without any back-end development.

Apologies for my misunderstanding.

However, it requires that title sort is disabled. If the result list should show the instance title there will be some latency.

Charlotte Whitt Lisa Sjögren is it acceptable that sorting be disabled for some search results in the inventory UI?

Based upon previous conversations, my understanding is that this limitation is not acceptable.

This solution requires some changes in the UI but has low development effort, it can be released earlier than other solutions.

I believe Zak Burke has already raised concerns about the front end work involved in this approach

Running the query against the item endpoint is the natural way when all sub-expressions should match the same item.

How would this work if the search also included properties from the instance or the holdings record? As I understand it, we would encounter the same challenge in reverse.

Comment by Julian Ladisch [ 12/Jan/21 ]

Querying against the item endpoint does NOT match different items at the same time. It matches one item only. Adding subqueries with holdingsRecords.* and instance.* to the query matches the holding and instance the item belongs to but does NOT allow to match other items.

This is the reason why Lisa and I point to this approach.

Comment by Zak Burke [ 12/Jan/21 ]

I have strong reservations about what I'm about to suggest, but in the interest of moving this conversation forward rather than debating whether it's necessary to be able to sort the results of a search in a library's catalog:

If we point the holdings and items queries at the holdings and items endpoints, respectively, in order to have multiple filters combine in a way that meets user expectations, we could process that output through a secondary query to the instances endpoint, as is done to handle client-side joins in ui-agreements. md331 noted in the PR where he implemented this pattern that it's cleaner than fetching directly via GET mutators in lifecycle methods. The downside, as I noted above, is that this doesn't accommodate "large" result sets (> ~40 records). Maaaaybe we could figure out how to show a "narrow your filter; too many results" warning if the initial query returns too many rows.

Or maybe we suck it up and use those lifecycle methods, along with a query that hangs off the results of a holdings/items query, to retrieve all related instance records (up to some threshold, 100? 1000?) in a loop. More gross, but also more powerful, at least from a search results point of view. To be clear, I don't love this, but we have some patterns for it (even if they're really anti-patterns sprung from the need to handle client-side joins) so maybe it's worth a shot. i.e. having already gone through the Gedankenexperiment of how to make this happen, maybe that hard part is mostly done.

Comment by Charlotte Whitt [ 13/Jan/21 ]

Thank you so much Zak Burke for bringing the conversation forward, and pointing at a solution

I have only one comment and that is, that a result set as > ~40 records, is probably to expect, so we probably from the very beginning need to go for the more powerful solution from a search result point of view.

CC: Holly Mistlebauer lew235 and Magda Zacharska (if this is to be addressed by Elastic Search)

Comment by Marc Johnson [ 13/Jan/21 ]

Charlotte Whitt

a result set as > ~40 records, is probably to expect, so we probably from the very beginning need to go for the more powerful solution from a search result point of view.

I want to check my understanding of this statement.

Are you saying that search results of more than 40 records should be expected? And that would make the solution that Zak Burke proposed an unacceptably limited solution for this?

Comment by Charlotte Whitt [ 13/Jan/21 ]

Marc Johnson -

Are you saying that search results of more than 40 records should be expected?

Yes that's correct, and it would be an unacceptable limited solution.

Comment by Marc Johnson [ 13/Jan/21 ]

Charlotte Whitt Lisa Sjögren Holly Mistlebauer

I'm getting the sense that both Julian Ladisch and Zak Burke's proposed solutions are too limited to be acceptable.

My understanding of the expectations for this are are as follows:

  • Any criteria expressed for an item must match the same item
  • Any criteria expressed for a holdings record must match the same holdings record
  • If both item and holdings criteria is included, then the item must also be part of that holding
  • it must be able to work with large sets of search results (maybe in the 1000s?)
  • it must allow the same sorting that UI inventory does at the moment

Is this a reasonable summary of the expectations?

Comment by Charlotte Whitt [ 13/Jan/21 ]

Thanks Marc Johnson - yes that's the expectations.
I can add that to the description of the ticket.

Comment by Julian Ladisch [ 13/Jan/21 ]

Are there use cases where different items of one instance should be queried?
For example find an instance that has one item at location A and a different item at location B?

item.effectiveLocationId="fcd64ce1-6995-48f0-840e-89ffa2288371"
AND
item.effectiveLocationId="8188760d-2641-44ca-a1f2-c111810201db"

If this is the case then the current implementation is still useful and this Jira should be changed to be a feature request (and not a bug).

Comment by Charlotte Whitt [ 13/Jan/21 ]

Hi Julian Ladisch - this ticket is about:

  • Search for item records which are subject for following two requirements: Item status Missing AND has the Effective location Main Library.

The behavior documented here is clearly a bug, while an unwanted instance (The girl on the train) > holdings > items is returned.

Comment by Julian Ladisch [ 13/Jan/21 ]

Charlotte Whitt You do not answer my question, sorry that I did not explain it clearly enough.
I understand that there are use cases where the instance search should match the same item. This Jira is about this single-item use case.
However, can there be a different use case, a multi-item use case, where matching different items of the same instance can be useful?
If yes then this Jira is requesting a new additional feature and the current multi-item match should be kept as a different search option because the multi-item match is not always a bug (better documentation should be added though) and this Jira should be changed to a feature request.
If no then this Jira is a valid bug and RMB should remove the multi-item feature.

Do I understand correctly that there will never be a multi-item use case where searching different items of the same instance is a useful feature like the example I posted above, and that the RMB CQL multi-item feature is completely wrong and should be thrown away?

Comment by Charlotte Whitt [ 13/Jan/21 ]

Hi Julian Ladisch - you are right I did not answer your question, and that is because what you describe is a different scenario, and do not really have anything to do with this bug ticket.

But I can write up another ticket for your question re multi-item use case - no problem.

Real live Use case:
Find me all results for Items which have item status missing at Main library, and Annex library.

Comment by Marc Johnson [ 13/Jan/21 ]

Charlotte Whitt

I can write up another ticket for your question re multi-item use case - no problem.

Does that mean that there are circumstances in the inventory UI where folks want to find instances that have an item that matches only some of the criteria (e.g. only one of two filters specified)?

Comment by Charlotte Whitt [ 13/Jan/21 ]

This is three different searches, right?

  1. Find me all results for Items which have item status missing
  2. Find me all results for Items which have item status missing at Main library
  3. Find me all results for Items which have item status missing at Main library, and Annex library

Marc Johnson and Julian Ladisch - I expect that

  1. Find me all results for Items which have item status missing - works well

When we have solved:
2. Find me all results for Items which have item status missing at Main library
Then that hopefully lead to that we do not see any new bug for
3. Find me all results for Items which have item status missing at Main library, and Annex library (which I of course will double check to be sure, and to catch and write up if that bug do exist)

Comment by Zak Burke [ 14/Jan/21 ]

Find me all results for Items which have item status missing at Main library, and Annex library

I think this should be "Find me all results for Items which have item status missing AND (location is Main library or location is Annex library). i.e. that multiple filters are AND-ed together (any matching must match all filters) while multiple values within a filter are OR-ed together (matching a filter means matching any one of its selected values).

This is the behavior, for example, in ui-users when ticking the boxes for Status - Active, Patron group - Faculty, and Patron group - Staff. Selected users must be active AND must belong to faculty or staff. I understand ui-users is a simpler use case (there is no join from instance to holdings to item) but from the point of view of a choosing the "Item" pill and then choosing some filters, I see why folks would expect the same behavior (filter-values OR-ed, multiple filters AND-ed).

Comment by Marc Johnson [ 01/Feb/21 ]

Holly Mistlebauer Lisa Sjögren Charlotte Whitt

This issue is in the Core Functional team's current sprint. What are folks expectations about progressing this?

From what I can tell from the conversations above. The current tooling (RAML Module Builder's implementation of multi-record type queries) won't allow us to satisfy this kind of search (except for the potential workaround to use the items API which sacrifices sorting and may have challenges when combined with other record types).

I think that leaves us with the following choices:

  • Change the current tooling (likely to be 2021 R2 at the earliest, given the 2021 R1 tool deadline is 5th February 2021 and may not be prioritised given the deprecation of RAML Module Builder)
  • Wait till the elastic search based search is ready (assuming that can address this kind of search, not till at least 2021 R2)
  • Change all of the searches that include items (on any of the tabs) to use the workaround (need to figure out how to communicate changes in behaviour, and will need significant testing for unexpected impact)

Jakub Skoczen Julian Ladisch is that a reasonable summary of the situation?

Comment by Lisa Sjögren [ 01/Feb/21 ]

Marc Johnson

As I've stated above, Chalmers who reported this to EBSCO does not consider this issue urgent (thus their Customer priority: Low), since for their intents and purposes Inventory search, as long as it only returns instances, does not lend itself very well to the type of searches that would be the most affected by this.

What I expect, considering the nature of the problem, is a solid long term solution which is well anchored in the architecture of Inventory. I cannot really speak to what that solution should be.

Comment by Marc Johnson [ 01/Feb/21 ]

Lisa Sjögren

As I've stated above, Chalmers who reported this to EBSCO does not consider this issue urgent (thus their Customer priority: Low), since for their intents and purposes Inventory search, as long as it only returns instances, does not lend itself very well to the type of searches that would be the most affected by this.

What I expect, considering the nature of the problem, is a solid long term solution which is well anchored in the architecture of Inventory. I cannot really speak to what that solution should be.

I apologise for asking a question that you had answered previously

I had gotten the impression, from the current priority (P2, meaning it needs resolving this release), the current customer priority (Important) and the involved conversation, that there was a need to move this issue forward soon.

Charlotte Whitt Holly Mistlebauer Kelly Drake

Please could you advise on this? It seems that EBSCO / Chalmers (based upon what Lisa Sjögren just said) consider this a low priority piece of work, yet it has been classified as an urgent and important piece of work.

Comment by Lisa Sjögren [ 01/Feb/21 ]

Absolutely no need to apologize Marc Johnson! Lots of comments to keep track of here. 🙂

The current Customer Priority value was added by Charlotte Whitt to reflect other libraries' needs.

Comment by Charlotte Whitt [ 01/Feb/21 ]

Yes, this ticket has been discussed at the past 3-4 Support SIG meetings ... and the talks have been lengthy.

At today's Support SIG meeting we decided that a talk with relevant MM-SIG SMEs, Elastic Search people and the developers who have been involved in the current Search in Inventory should be held, and then we must review our options, and decide for how to move along.

I'll try if I can find a suitable meeting time this week. I'll send out a doodle poll ASAP.

Comment by Charlotte Whitt [ 02/Feb/21 ]

Zak Burke - I had to test using Query Search:

Following query search gives the expected result (test data in FOLIO Snapshot as of today)
(item.status = "Missing" AND item.permanentLocationId = "fcd64ce1-6995-48f0-840e-89ffa2288371") NOT item.status = "Available"

URL: https://folio-snapshot.dev.folio.org/inventory?filters=effectiveLocation.fcd64ce1-6995-48f0-840e-89ffa2288371&qindex=querySearch&query=%28item.status%20%3D%20%22Missing%22%20AND%20item.permanentLocationId%20%3D%20%22fcd64ce1-6995-48f0-840e-89ffa2288371%22%29%20NOT%20item.status%20%3D%20%22Available%22&segment=items&sort=title

I do (of corse) understand, that eventually all other item states should also be listed with NOT Order closed, NOT Awaiting delivery, etc. etc.

If I do the filtering on Item status: Missing, and Effective location: Main library, then I (with my test data) get the not wanted instance title: The girl on the train (as we already have documented above):

Comment by Julian Ladisch [ 02/Feb/21 ]

This query doesn't solve the problem.
It means:
Find an instance that

  • has at least one item with the permanentLocationId and
  • has at least one item with status "Missing" and
  • has no item with status "Available"

This is a different query.

If you change the status of the "The Girl on the Train" barcode=7031 item from "Available" to some other status the query will also return the instance "The Girl on the Train".

Comment by Charlotte Whitt [ 03/Feb/21 ]

Julian Ladisch - I did also write:

I do (of corse) understand, that eventually all other item states should also be listed with NOT Order closed, NOT Awaiting delivery, etc. etc.

So I'm fully aware that the query can be more complicated – but my approach do solve the issue here.

Julian Ladisch I'm no super expert on CQL, so there might be away to express: that only item status Missing is wanted here. That would make the CQL much prettier.

Comment by Holly Mistlebauer [ 03/Feb/21 ]

Charlotte Whitt: If this issue doesn't need to be fixed for R1 2021, or isn't ready to be fixed in R1 2021, we need to change the status from P2 to P3. P2 is reserved for bugs that must be fixed in the next release. Thanks...

Comment by Charlotte Whitt [ 03/Feb/21 ]

Okay, I can lower it to P3.

Comment by Julian Ladisch [ 10/Aug/21 ]

If I search on an item attribute and an instance has both a matching item and a non-matching item should both been shown or only the matching item?
The back-end (RMB, mod-inventory-storage) supports both.
The front-end only supports the instance based view that always shows all items of an instance, even items that doesn't match the item search criterium.
Please consider adding an item based front-end view where only matching items are shown.

Comment by Charlotte Whitt [ 11/Aug/21 ]

Hi Julian Ladisch - we should not invest more effort in fixing this in the current search solution. We are planning for doing the swap of Inventory using PostgreSQL to be using Elastic Search.

I have added `Elastic-search` as a label - so we keep track, and make sure that it's not being dropped under the radar.

Comment by Charlotte Whitt [ 11/Aug/21 ]

Anya - should I remove the `support` label?
I don't think we should try and solve it using the old search technology. This will be throw away work, and instead we should focus on getting it right when using Elasticsearch
CC: Magda Zacharska

Comment by Julian Ladisch [ 11/Aug/21 ]

Charlotte Whitt: The PostgreSQL tables will continue to exists, Elastic Search doesn't replace them, Elastic Search adds an additional search option.

Adding an additional front-end view where only matching items are shown is needed regardless of the underlying search machine.

Comment by Marc Johnson [ 28/Oct/21 ]

Charlotte Whitt Lisa Sjögren

This is currently marked as needing to be fixed for 2021 R3 Bug Fix.

The title suggests that the intention was to resolve this in the ES based search rather than regular inventory. As MSEARCH-47 Closed is closed, should this also be closed as won't do because we've chosen a different solution?

Comment by Marc Johnson [ 01/Nov/21 ]

Superseded by ES based work.

Charlotte Whitt Lisa Sjögren please reopen if this is incorrect.

Generated at Thu Feb 08 22:21:24 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.