| Table of Contents |
|---|
Scope
In order to address issues related to To address the poor performance of the Inventory appinventory search, the Technical Council decided during the meeting in October 2020 to implement an alternative Inventory search using Elastic searchrecommended to investigate a possible Elasticsearch implementation. The scope of the work was determined to include following deliverables for the Iris release:
Back-end:
- Sending update notification messages from inventory and source record storage (SRS)
- Providing Inventory and SRS APIs for fetching view for indexing by ids
- Extract common library for using it in other modules
- Build infrastructure necessary to support Elasticsearch
...
In December 2020 it was determined that sending update notification messages from source record storage (SRS) are search is out of scope for Elasticsearch the Elasticsearch POC and it became a separate feature (UXPROD-2791)
Delivered functionality
The work delivered by Falcon team as a part of the Iris release includes:
Back-end:
- Sending add/update/delete notification messages from Inventory
- Built Search APIs for searching and faceting
- Combined instances + holding + items into a single index
- Implemented re-index process for existing inventory DB
- Spring base implementation that supports:
- Up to five language-specific analyzers configured on the tenant level
- Near real-time inserts, updates and deletions
- Boolean operators (AND, OR, NOT)
- Nested search using brackets
- All or Any keyword search
- Exact phrase search
- Left-, right-hand truncation, wildcards searches in some fields
...
- Added Elasticsearch cluster to CI/CD and set it up on the reference environments
- Updated existing Kafka cluster configuration
- Introduced option of setting up performance testing environment in the community
As the a result of , the work, following search options and filters are supported:
...
The evaluation of the POC took place from April 5th to April 9th, 2021 and it was conducted in the the Bugfest environment (~8 millions records) by eEight eight evaluators representing:
- Chicago University (2 participants)
...
- Duke University
...
- Missouri State University (2 participants)
...
- Simmons University
...
- EBSCO
...
- Index Data
...
Almost all evaluation happened was conducted trough UI. 75% of those who participate, found the implementation POC successful but all participants saw the room for improvements. The team addressed following issues that were raised:
Issue | Solution |
Noisy search results | Implemented searches supporting keyword “all” or “any” limiting the number of matches: MSEARCH-91 |
Expected results not found | All provided examples were related to the special characters in the Title that were searched using ASCII representation. The problem will be addressed in scope of MSEARCH-67 |
Bug in sorting by title | |
Support phrase search | |
Ranking refinement | Refinement of the default ranking system will require further analysis to be in the scope of a separate feature |
Discrepancy in saving UUIDs from Action menu | |
UI enhancements and bugfixes | UISEES-47, UISEES-57, UISEES-61, UISEES-62, UISEES-48, UISEES-49 |
...
- Expected to perform complex queries of multiple fields and across record types (including MARC fields).
- Expected a different UI more like a catalog or discovery system advanced search.
- Expected support for additional operators (not equal to, starts with, etc.).
- UI not user friendly.
- Preferred a simple left-anchored search than the provided relevancy ranking
...
- TC Meeting on October 14, 2020
- UXPROD-2791 SRS MARC Query API
- Search – Technical documentation
- POC Evaluation survey
- Detailed responses to raised issues
- POC overview Overview for MM SIG (presentation)POC overview for MM SIG slides) and (recording)
- 110 -111 Sprints Demo