Search - using PostgreSQL (all releases up to Juniper)
NOTE
This page describes the search in Inventory UI up to and including the Juniper release. Since the Kiwi release, Inventory UI uses Elastic/Open Search and no longer PostgreSQL search.
This page can still be used for CQL queries at mod-inventory APIs (/inventory/...) and mod-inventory-storage APIs (/inventory-storage/...).
Please keep this information in mind when using the page.
Search and filters
FOLIO’s current technical approach for search is using PostgreSQL, JSONB columns, CQL and RAML Module Builder. Using Elastic Search as a long term solution is being explored Autumn 2020 - Spring 2021 - see more.
In inventory, we have implemented segmented control search where it's possible to toggle between searching on an Instance, Holdings or individual Item level. Search in the Instance segment supports search on bibliographic data. Search in the Holdings and Item segments are a combination of key data from the Instance record, combined with holdings and item specific data elements. The filters reflect each of the individual segments and include search on effective location.
This is work in development, but as of end of January the following search and filter options are present in FOLIO Snapshot:
| Instance | Holdings | Item |
|---|---|---|---|
Search options |  | | |
Filter options | | | |
Search and filter options can be combined.
When searching in a filter, the search typed will be highlighted. E.g.:
If more than 100 hits are retrieved, the first 100 will load, and then you will see a "Load more" button at the bottom of the page.
When clicked, it will display the next 100:
From the FOLIO version Juniper onwards the concept of loading more records has been redesigned. You can navigate through the records in steps of 100 by clicking the "next/previous" buttons at the bottom of the page.
There is also a Query search function, which can search across records on all data elements. See below.
Search using Query language (Instance, Holdings, Item segment)
See list of property names in Query Search - Inventory Metadata Elements, or in the API Documentation in the /instance-storage/instances POST, /holdings-storage/holdings POST and /item-storage/items POST.
See more CQL tips written up by @Julian Ladisch Explain CQL string matching. For experts: RAML Module Builder CQL documentation.
See: Tips: Use Query Search in Inventory for export MARC records from FOLIO
See list of defined supporting database indexes: db_scripts/schema.json – Skip to the "tableName": "instance", "tableName": "holdings_record" or "tableName": "item" section and look into the "uniqueIndex", "index", and "fullTextIndex" subsections. The id field is automatically indexed and not listed in schema.json.
See misc. test of search performance in Bug fest Goldenrod (https://bugfest-goldenrod.folio.ebsco.com/) in Document Inventory Search Response time (Q3 Timebox)
Tips: When searching by an ID, in the Query search it's better to use "==" as it uses the B-tree index to search, whereas a CQL query using "=" will result in a full text search, which is not efficient for searching when there is already an index on the ID.
| | |
|---|
Examples on CQL syntax (WIP)
Query search | Search result |
|---|---|
id == "f31a36de-fcf8-44f9-87ef-a55d06ad21ae" | give me the instance record where UUID = f31a36de-fcf8-44f9-87ef-a55d06ad21ae |
hrid == "in00000009" | give me the instance record where the HRID = in00000009 (this number will later be an 11 digit sequence) |
source = "folio" | give me all instance records where metadata source = FOLIO |
source = "marc" | give me all instance records where metadata source = MARC, which means that the instance record has an underlying MARC record in SRS |
contributorsNames all "Sosa Keita" | give me all instance records where the contributor names contain both Sosa and Keita (any position, any order) |
contributorsNames adj "Sosa Omar" | give me all instance records where the word Sosa is immediately followed by the word Omar in the contributor names ignoring whitespace and punctuation between words (phrase search) |
identifiers =/@value/@identifierTypeId="01ca9cda-7027-4d64-abed-9e3c4943daf2" "(OCoLC)670473988" | give me all instance records where OCLC number = (OCoLC)670473988 - result in BugFest Iris environment |
identifiers =/@value/@identifierTypeId="01ca9cda-7027-4d64-abed-9e3c4943daf2" ("806014467" OR "560591246") | give me the two instance records with OCLC number = (OCoLC)806014467, (OCoLC)560591246 by using the OR boolean operator - result in BugFest Goldenrod environment |
statisticalCodeIds ="c6de6928-bb6e-4458-97f4-9145b27abb24" | give me all instance records where the statistical code = books (University of Chicago) - result in BugFest Iris environment |
title == dave * | give me all instance records where the resource title exactly matches against the resource title data element as: dave * |
alternativeTitles=ATLA Proceedings | give me all instance records where the alternative title = ATLA Proceedings |
series = "cooperative information systems" | give me all instance records where the series title = Cooperative information systems |
publication = "MIT Press" | give me all instance records where the publisher = MIT Press |
publication = "c2004" | give me all instance records where the publication year = c2004 |
physicalDescriptions = "bände" | give me all instance records where the physical description = bände |
subjects = "history" | give me all instance records where the subject heading = history |
classifications =/@classificationNumber "025.04" | give me all instance records whit classification code = ”025.04” (this is supported since Iris release) |
classifications = "025.04" | give me all instance records whit classification code = ”025.04” (returns false positives if the code equals a UUID substring in the record) |
instanceFormatIds = "5cb91d15-96b1-4b8a-bf60-ec310538da66" | give me all instance records where format term = audio disc |
keyword all oclc | give me all instance records where oclc is mentioned in title, contributor, or identifier |
notes = "Volltext auch als Teil einer Datenbank verfügbar" | give me all instance records where the note = Volltext auch als Teil einer Datenbank verfügbar |
catalogedDate == "yyyy-mm-dd" | give me all instance records catalogedDate on a given date |
metadata.updatedByUserId="aeaf920a-ecd5-5d97-a95a-7aaa3f542cbd" | give me all instance records that were last changed by the user "diku-admin" |
discoverySuppress=="false" NOT holdingsRecords.permanentLocationId==("89560105-164b-4b1b-a887-be89ede65b58" OR "f34d27c6-a8eb-461b-acd6-5dea81771e70") | give me all instance records that are not suppressed and that have at least one holding with permanentLocationId other than the listed values. |
holdingsRecords.fullCallNumber == "TK5105.88815 . A58 2004 FT MEADE" OR holdingsRecords.callNumberAndSuffix == "TK5105.88815 . A58 2004 FT MEADE" | give me all holdings records where the call number = TK5105.88815 . A58 2004 FT MEADE |
holdingsRecords.copyNumber = c.2 | give me all holdings records where the copy number = c.2 |
holdingsRecords.holdingsStatements = "v.70-84" | give me all holdings records where the holdings statement = v.70-84 (1984-1998) |
holdingsRecords.notes = "CharlotteTest" | give me all holdings records where the holdings notes = CharlotteTest |
holdingsRecords.metadata.createdDate = "2020-07-13" | give me all holdings records created on, before or after 2020-07-13; use "2020-07-13T23:10:37" for time (UTC) |
holdingsRecords.metadata.updatedDate = "2020-07-13" | give me all holdings records updated on, before or after 2020-07-13; use "2020-07-13T23:10:37" for time (UTC) |
holdingsRecords.holdingsTypeId == "373ae405-37b3-4ff5-98c6-e9842d2d6687" | give me all holdings records where holdings type is Electronic - result in BugFest Iris environment |
holdingsRecords.permanentLocationId == "590e4d5c-6369-4bcc-8467-c0cfa03bf21e" | give me all holdings records where holdings permanent location id is X location |
item.enumeration = "v.71" | give me all item records where the enumeration = v.71 |
item.accessionNumber == 2424242424 | give me all item records with accession number 2424242424 |
item.electronicAccess = "table" | give me all item records where there in the electronic access elements is matching value = table |
item.notes = "note" | give me all item records where the item note = note |
item.circulationNotes = missing pages* | give me all item records with check out/check in note about having missing pages |
item.permanentLoanTypeId == "2b94c631-fca9-4892-a730-03ee529ffe27" | give me all item records where the permanent item loan type is Can circulate |
item.metadata.createdDate = "2020-07-13" | give me all item records created on, before or after 2020-07-13; use "2020-07-13T23:10:37" for time (UTC) |
item.metadata.updatedDate = "2020-07-13" | give me all item records updated on, before or after 2020-07-13; use "2020-07-13T23:10:37" for time (UTC) |
Boolean search within records and across instance/holdings/item:
Query search | Search result |
|---|---|
publication = "MIT Press" and publication = "c2004" | give me all instance records where publisher = MIT Press AND publication year is c2004 |
publication = "MIT Press" and holdingsRecords.callNumber = "TK5105.88815 . A58 2004 FT MEADE" | give me all instance records where publisher = MIT Press AND the associated holdings records has holdings level call number = TK5105.88815 . A58 2004 FT MEADE |
subjects = "history" or identifiers = "OCoLC" not publication = "2017" | give me all instance records where publisher = MIT Press AND the resource identifier is starting with OCoLC and does NOT has publication year = 2017 |
languages="eng" AND item.itemLevelCallNumber="TK5105*" | give me all instance records where language = English AND the associated item records has item level call number starts with TK5105. The query includes wildcard search with an asterisk |
identifiers =/@value/@identifierTypeId == "216b156b-215e-4839-a53e-ade35cb5702a" "10419/192087" | give me all instance records where resource identifier type = handle AND resource identifier = 10419/192087 |
source = "*" not source = "folio" Alternative CQL: id=* not source = "folio" cql.allRecords = "1" not source = "folio" | give me all instance records where metadata source is NOT Folio (but: MARC, and other source formats) |
item.status="available" AND statusId = "26f5208e-110a-4394-be29-1569a8c84a65" | give me all records where item status = available AND instance status term = Uncataloged |
item.barcode="10101" | give me the instance record the item with barcode 10101 belongs to |
Search queries that currently do not seem to work
Query search | Expected search result | Actual search result | Tested by / on environment | Comments |
|---|---|---|---|---|
discoverySuppress="false" AND holdingsRecords.discoverySuppress="false" AND item.discoverySuppress="false" AND item.barcode="null" | Instance, Holdings, Item are not suppressed (for discovery?) AND item barcode = blank example from 2020-01-30 Metadata Management Meeting notes |
|
| The instance record JSON includes a boolean "discoverySuppress" which is set to false in my instance example record. Holding and item records don't have this elements in their response. Use Chrome's dev tools to verify. I'm not sure how to search for NULL values. If an item has no barcode assigned, the element is not included in the JSON response either. cc @Charlotte Whitt |
|
|
|
|
|
|
|
|
|
|
Item search
Query search | Search result |
|---|---|
id=* NOT item.barcode="" NOT discoverySuppress="true" NOT holdingsRecords.discoverySuppress="true" NOT instance.discoverySuppress="true" | give me all item records that have no barcode and are not suppressed for discovery on item, holdings and instance level. |
Sort
Change the sort field in the URL.
Example:
https://folio-snapshot.dev.folio.org/inventory?query=*&sort=title becomes
https://folio-snapshot.dev.folio.org/inventory?query=*&sort=publicationPeriod.start
We need to use publicationPeriod.start because there can be multiple dateOfPublication entries in the publication array.
Retrieve UUIDs
For some CQL queries you need the UUID of the value, because you cannot search for the actual term. To do this, you can use the developer tools that are built into your browser. In Chrome, press F12 to open the developer tools and switch to the "network" tab. Clear out any previous data by pressing "clear" and refresh the Folio page. Please note, that you'll first have to open a detail instance record in the third pane. After the page has loaded, you should look in the network data for the record UUID. Select it to see the record data in JSON format.
RefData | Value | UUID |
|---|---|---|
Instance status types | Batch Loaded | 52a2ff34-2a12-420d-8539-21aa8d3cf5d8 |
Cataloged | 9634a5ab-9228-4703-baf2-4d12ebc77d56 | |
Not yet assigned | f5cc2ab6-bb92-4cab-b83f-5a3d09261a41 | |
Other | 2a340d34-6b70-443a-bb1b-1b8d1c65d862 | |
Temporary | daf2681c-25af-4202-a3fa-e58fdf806183 | |
Uncataloged | 26f5208e-110a-4394-be29-1569a8c84a65 | |
|
|
|
More documentation on search indexes (from a developer's perspective)
Following list of resources is gathered by @Julian Ladisch:
Existing documentation:
All fields can be indexed, simply add an index entry to schema.json on GitHub. The indexing is the same for all institutions and a change requires a new release of the module.
Currently institutions cannot customize indexing at runtime and cannot have different indexing. GBV libraries have extensive indexing customization in the inventory of their current system (OCLC LBS) and will likely request this for FOLIO. Index customization use cases for inventory should be discussed in the metadata management sig.