Elasticsearch (UXPROD-2591)

[UXPROD-2806] Create Elasticsearch indexes for Inventory search Created: 10/Nov/20  Updated: 17/Jan/22  Resolved: 19/Apr/21

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: R1 2021
Parent: Elasticsearch

Type: New Feature Priority: P2
Reporter: Magda Zacharska Assignee: Magda Zacharska
Resolution: Done Votes: 0
Labels: r1-2021-at-risk
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
blocks FOLIO-2573 Paused due to POC of Elastic Search. ... Closed
blocks MODINVSTOR-683 Wait for POC of Elastic Search: Slow ... Closed
blocks UIIN-1049 Paused due to POC of Elastic Search. ... Closed
blocks UIIN-1070 Paused due to POC of Elastic Search. ... Closed
blocks UIIN-1258 Paused due to POC of Elastic Search. ... Closed
blocks UXPROD-2276 Extend the keyword index. Add barcode... Closed
blocks UXPROD-2687 Wait for POC of Elastic Search. Inven... Closed
blocks UXPROD-2712 Inventory Elastic Search (Lotus): Tim... Closed
blocks MODINVSTOR-692 Unresponsive Instance Query Search Closed
blocks UIIN-1365 Paused due to POC of Elastic Search. ... Closed
blocks UXPROD-2369 Wait for POC of Elastic Search - Impl... Closed
blocks UXPROD-2623 Wait for POC of Elastic Search - Roun... Closed
blocks RMB-685 totalRecordsRounded Blocked
Defines
is defined by MSEARCH-3 Instance - hierarchical filter by i... Open
is defined by MSEARCH-32 Holdings - hierarchical facet by per... Open
is defined by MSEARCH-61 Item - effective location autocomplete Open
is defined by MSEARCH-62 Holdings - permanent location autoco... Open
is defined by UISEES-8 Effective locations - hierarchical facet Open
is defined by MODINVSTOR-674 Implement bulk instance endpoint Closed
is defined by UISEES-26 Replace Query Search with Advance Sea... Closed
is defined by UISEES-28 Create CheckboxFilter component with ... Closed
is defined by UISEES-29 Update filters options for Instances ... Closed
is defined by UISEES-30 Update filters options for Items with... Closed
is defined by UISEES-31 Update filters options for Holdings w... Closed
is defined by MODINVSTOR-647 Refactor ItemStorageAPI and HoldingsS... Closed
is defined by MSEARCH-1 Instance - support keyword search f... Closed
is defined by MSEARCH-2 Instance - HRID searches Closed
is defined by MSEARCH-4 Instance - filters/facets Closed
is defined by MSEARCH-9 Tenant level language support Closed
is defined by MSEARCH-10 Indexing of instances Closed
is defined by MSEARCH-12 Items - keyword search by effective ... Closed
is defined by MSEARCH-13 [Spike]: Instance - investigate per... Closed
is defined by MSEARCH-15 Instance - support keyword search ... Closed
is defined by MSEARCH-16 Instance - support title keyword se... Closed
is defined by MSEARCH-17 Instance - support contributors key... Closed
is defined by MSEARCH-19 Instance - support keyword search ... Closed
is defined by MSEARCH-21 Instance - support keyword search b... Closed
is defined by MSEARCH-22 Instance - support keyword search ... Closed
is defined by MSEARCH-23 Instance, holdings and items - supp... Closed
is defined by MSEARCH-24 Instance - fields supporting phrase... Closed
is defined by MSEARCH-25 Instance - UUID search Closed
is defined by MSEARCH-26 Holdings - HRID searches Closed
is defined by MSEARCH-27 Items - HRID searches Closed
is defined by MSEARCH-31 Item - keyword search by barcode Closed
is defined by MSEARCH-33 Item - suppress from discovery filter... Closed
is defined by MSEARCH-34 Holdings- suppress from discovery fil... Closed
is defined by MSEARCH-35 Holdings - additional fields support... Closed
is defined by MSEARCH-36 Item - additional fields supporting a... Closed
is defined by MSEARCH-43 Instance - reference data - filters... Closed
is defined by MSEARCH-44 Item - material types - filters/facets Closed
is defined by MSEARCH-47 Item - combine Item's effective locat... Closed
is defined by MSEARCH-48 Instance - search by item's effecti... Closed
is defined by MSEARCH-55 Instance - filter/facet by item eff... Closed
is defined by MSEARCH-56 Holdings - filter/facet by permanent ... Closed
is defined by MSEARCH-58 Support advanced search Closed
is defined by MSEARCH-63 Item - status filters/facets Closed
is defined by MSEARCH-81 Holdings - keyword search by call nu... Closed
is defined by UISEES-1 Elasticsearch for Inventory search - ... Closed
is defined by UISEES-2 Autocomplete for reference data - Ins... Closed
is defined by UISEES-3 Reference data - filters/facets - Ins... Closed
is defined by UISEES-4 Prepare environment with forked inven... Closed
is defined by UISEES-5 Add link to the existing inventory se... Closed
is defined by UISEES-6 Spike - UI for Facet component Closed
is defined by UISEES-7 Inventory clone powered by Elasticsea... Closed
is defined by UISEES-9 Keyword search by title, contributor,... Closed
is defined by UISEES-10 Onboarding Closed
is defined by UISEES-11 Spike - Investigate approach for the ... Closed
is defined by UISEES-12 Update icon on Inventory-es Closed
is defined by UISEES-13 Add "Proof of concept" label Closed
is defined by UISEES-14 Instance - language filter/facet Closed
is defined by UISEES-15 Instance - Tags filter/facet Closed
is defined by UISEES-16 Instance - Source filter/facet Closed
is defined by UISEES-17 Instance - Suppress from discovery fi... Closed
is defined by UISEES-18 Instance - Staff suppress filter/facet Closed
is defined by UISEES-19 Autocomplete for reference data - Ite... Closed
is defined by UISEES-20 Reference data - filters/facets - Ite... Closed
is defined by UISEES-21 Item - Tags filter/facet Closed
is defined by UISEES-22 Holdings - Tags filter/facet Closed
is defined by UISEES-23 Item - Suppress from discovery filter... Closed
is defined by UISEES-24 Holdings - Suppress from discovery fi... Closed
is defined by UISEES-32 Replace input-inventory-search contro... Closed
is defined by UISEES-34 Add the supported search options Closed
is defined by UISEES-35 Add suggestions to the “Advanced sear... Closed
is defined by UISEES-36 Creating a parser and rendering the r... Closed
is defined by UISEES-37 Replacing existing query search Closed
is defined by UISEES-39 Missing dependency in package.json Closed
is defined by UISEES-40 Remove Inventory- es settings Closed
is defined by MSEARCH-18 Instance - support publisher keywo... Closed
is defined by MSEARCH-20 Instance - support keyword search ... Closed
Relates
relates to MSEARCH-57 Instance, holdings and items - suppor... Closed
Epic Link: Elasticsearch
Front End Estimate: XXL < 30 days
Front End Estimator: Magda Zacharska
Front-End Confidence factor: Low
Back End Estimate: XXL < 30 days
Back End Estimator: Magda Zacharska
Development Team: Falcon
Rank: Chicago (MVP Sum 2020): R1
Rank: Cornell (Full Sum 2021): R1
Rank: Duke (Full Sum 2021): R1
Rank: 5Colleges (Full Jul 2021): R1
Rank: GBV (MVP Sum 2020): R2
Rank: Lehigh (MVP Summer 2020): R2
Rank: TAMU (MVP Jan 2021): R2
Rank: U of AL (MVP Oct 2020): R2
Score: 16
Showstopper for Summer 2021 Implementers?: No
Showstopper Comments from Summer 2021 Implementers: [Steven from Five Colleges: Five College Showstopper based the complexity of our locations] [Magda responded to Steven: Hi Steven Bischof could you provide some use cases where the current implementation of Inventory search is insufficient for Five Colleges' needs?] [Tod from Chicago: The issue for us is around needed improvements to search and search results in Inventory, many of which have become tied to Elasticsearch implementation. The problem with search in Inventory is that there are many cases where the available identifying text is not enough distinguish between the results. Without those needed improvements, we will be having some difficult conversations about whether we can support our workflows.][Shawn & Kay from MI St U. Need efficient search to enable pre-order work, adding cataloging to complicated sets, such as government documents.]
Showstopper December 11 Meeting Summary: UPDATE: This feature was changed to NOT be a showstopper given a thin-thread version of UXPROD-2712 is the actual showstopper.
The understanding by the implementers is that this feature needs to be completed before they can the improved searching that they need (as described in UXPROD-2712). Right now users cannot tell the difference between an ebook and a regular book. Magda, the PO, pointed out that this feature is to implement Elasticsearch in addition to the existing search-it will not be integrated as part of R1 2021. So, we need to figure out what the Iris summer 2021 implementers REALLY need. Charlotte recommends that we look at the user stories that define UXPROD-2712 and decide which ones are 'showstoppers'. We will move the 'showstoppers' to a separate UXPROD feature and estimate that work. This feature would be re-planned for R2 2021.

 Description   

Current situation or problem:

The existing inventory search does not provide enough flexibility in building the search criteria, does not perform as expected when searches in large collection takes long to return the data. Also, in searches that have more than 1000 records fulfilling the search, the count of the matching records is incorrect.

In scope
Searches against inventory instances, holdings and items:

Instance Holdings Items
Keyword search (title, contributor, identifier) Effective location Item Status
Effective location Holdings permanent location Effective location
Language Suppress from discovery Holdings permanent location
Resource type   Material type
Format   Suppress from discovery
Mode of issuance    
Nature of content    
Staff suppress    
Suppress from discovery    
Date created (from, to)    
Date updated (from, to)    
Source    

Out of scope

  1. Other than inventory records searches
  2. Updates to existing inventory search.

Proposed solution/stories

1. Define mappings that will determine

  • which fields should be treated as full text search fields
  • which fields contain numbers and dates and their format
  • custom rules for dynamically added fields

2. Define approach to text analysis (tokenization and normalization)

Links to additional info

Questions



 Comments   
Comment by Steve Bischof [ 09/Dec/20 ]

Five College Showstopper based the complexity of our locations

Comment by Magda Zacharska [ 09/Dec/20 ]

Hi Steve Bischof could you provide some use cases where the current implementation of Inventory search is insufficient for Five Colleges' needs?

Comment by Tod Olson [ 10/Dec/20 ]

Chicago - The issue for us is around needed improvements to search and search results in Inventory, many of which have become tied to Elasticsearch implementation. The problem with search in Inventory is that there are many cases where the available identifying text is not enough distinguish between the results. Without those needed improvements, we will be having some difficult conversations about whether we can support our workflows.

Comment by Magda Zacharska [ 11/Dec/20 ]

The initial implementation of Elasticsearch for inventory does not mean to replace the existing Inventory search but add additional option for the user. Steve Bischof and Tod Olson if this is showstopper for your institutions please provide use cases so we can prioritize.

Comment by Steve Bischof [ 11/Dec/20 ]

Probably the most obvious use case for Five Colleges is limiting by location in combination with things like identifiers, title, author. With multiple cataloging, acquisition and circulation units, this becomes a critical tool for finding the appropriate instance/holding/item

Generated at Fri Feb 09 00:27:00 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.