Background:
In October of 2020, during the Technical Council meeting it was decided to look into Elasticsearch as a possible alternative for the existing Inventory search due to its poor performance and inability to provide exact count of records matching search criteria.
The development team has been formed in December 2020 and a PO was assigned to the team. The scope of the work was limited to what was limited to the back-end work due to the limited time for delivery. The details of what was discussed are here
Scope:
- Deliverables - back-end:
- Sending update notification messages from inventory and source record storage (SRS)
- Providing Inventory and SRS APIs for fetching view for indexing by ids
- Extract common library for using it in other modules
- Build infrastructure necessary to support Elasticsearch
- Deliverables - front-end:
- Update Stripes Components to support new Search API
- Provide “switch” to allow using Elasticsearch or existing search
- Deliverables - infrastructure:
- Add Elasticsearch cluster to CI/CD and setup it on environments (k8s conf)
- Check configuration of existing Kafka cluster
Sending update notification messages from source record storage (SRS) - determined as out of scope for Elasticsearch UXPROD-2791
Delivered functionality:
- Back-end:
- Sending add/update/delete notification messages from Inventory
- Built Search APIs for searching and faceting
- Combined instances + holding + items into a single index
- Implemented reindex process for existing inventory DB
- Front-end
- Built UI for alternate Inventory search app that represents searching capabilities of Elasticsearch
- Infrastructure:
- Added Elasticsearch cluster to CI/CD and set it up on reference environments
- Updated existing Kafka cluster configuration
- Introduced option of setting up performance testing environment in the community
Instance | Holdings | Items |
Keyword search (title, contributor, identifier) | Keyword search (title, contributor, identifier) | Keyword search (title, contributor, identifier) |
Contributors | ISBN | Barcode |
Title (all) | ISSN | ISBN |
Identifiers (all) | Call Number | ISSN |
ISBN | Holdings HRID | Material type |
ISSN | Call Number | |
Subject | Item HRID | |
Instance HRID | ||
Instance UUID | ||
Electronic access (all fields) |
Instance | Holdings | Items |
Effective location | Effective location (item) | Item Status |
Language | Holdings permanent location | Effective location |
Resource type | Suppress from discovery | Holdings permanent location |
Format* | Tags | Material type |
Mode of issuance | Suppress from discovery | |
Nature of content | Tags | |
Staff suppress | ||
Suppress from discovery | ||
Date created (from, to)* | ||
Date updated (from, to)* | ||
Source | ||
Tags |
POC evaluation:
Proposed next steps:
- Use mod-search endpoints for searching
- Redesign Inventory UI Search component so that it can include new UI components created by POC, especially filters and facets
- Conduct usability study for advanced search textbox
- Conduct analysis of ranking refinements (weights and boosts)
- Conduct analysis of further search refinements
- Define and prioritize work for cross app/cross record types searches
- Define UI for cross app/cross record types searches
- Define requirements for cross-tenant searches