Concept work: Remove leading and trailing whitespace from fields and search query
This page contains information about a phased approach to implement a better user experience when working with and searching for Inventory records (instances, holdings, and items). It is based on the consensus of the MM SIG (
Version of this page: 2025-09-08, updated 2025-12-04
Problem definition
Currently, the instance, holdings and item records store whitespace characters that are present at the beginning or end of a field value in the database.
Whitespace character | Stored field value |
|---|---|
U+0020 : Space |
|
U+0009 : Tab |
|
U+000A: Line Feed + U+000D: Carriage Return |
|
U+000A: Line Feed |
|
With sunflower, records with those control characters can be retrieved even if the control character is missing from the query. That is expeceted behavior.
In the interests of clean metadata, however, such control characters should not be stored in the records in the first place.
Concept idea
Split
MVP
Modify ui-inventory so that whitespace is removed when new records are created and existing records are updated. Add one feature for each record type (instance, holdings, item) in Inventory. For each feature, create subsequent stories that further specify the details of the work, e.g. a list of fields that have to be changed.
The UI will already remove white spaces from the CQL query, so no mod-search ticket needed.
Update existing metadata
A library might want to correct faulty data in existing records. Investigate options:
Update through a manual process by searching for them via the API in instance-storage, holdings-storage, and items-storage.
Create a script to automatically update records, either
invidivual at each library or
community-wide using a migration script during Flower upgrade.
Add a new feature that handles the correction in existing Inventory records.
Implement safety mechanism in the module’s backend
Add a feature to modify mod-inventory to validate and either
reject a JSON that contains any whitespace at the beginning/end or
remove leading/trailing whitespace from JSON (preferred).
Good to know
A user can retrieve records that contain such control characters not directly via the Inventory UI search, but can use the API’s or an external solution like metaDB.
Using Inventory’s storage API’s
Add the control characters in the percent encoded form:
%20 (Space)
%09 (Tab)
%0A (LF)
%0D (CR)
Using metaDB
<tbd>