Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Use ILIKE for content column of srs_marctab table

Description

Background:  LDP's ldpmarc tool generates a table called srs_marctab from SRS MARC records.  This table is very large and cannot be queried within a short time unless indexes are used.  The tool has recently been integrated with folio-snapshot so that it can be tested with this app.

Problem:  The column "content" of table "public.srs_marctab" has a specialized index that does not work with the equals (=) operator but only works with LIKE or ILIKE.  This will cause a query on that column to be prohibitively slow with a significant amount of data.

Proposed solution:  Hardcode this particular column to use ILIKE instead of equals.

Fallback:  If the solution is not workable or acceptable, then disable filtering on srs_marctab.content or implement a general, correct solution..

In the future we will need a general approach, because Metadb will at some point support a choice of index types.

Environment

None

Potential Workaround

None

relates to

Checklist

hide

TestRail: Results

Activity

Show:

Mike Taylor November 4, 2024 at 7:46 PM

Thanks, — now closed.

Nassib Nassar November 4, 2024 at 7:05 PM

I made this issue three years ago and a lot has changed since then. At this point I would suggest closing the issue if there has been no recent discussion on it.

Mike Taylor November 4, 2024 at 5:23 PM

I have no idea why I moved this to MODLDP back in April 2022. I can make a UI-side change so that when the user enters a filter on public.srs_marctab (or its Metadb equivalent) with the = relation, we silently rewrite it to that the submitted version uses ILIKE instead.

, do you agree that this suffices? Or do you think we to prevent client software from causing an = search on this index by disallowing it on the server side? Given how many ways there are to cause denial-of-service, this feels like a pretty futile thing to do.

Charlotte Whitt November 4, 2024 at 2:40 PM

- this we just reviewed …

Mike Taylor April 4, 2022 at 3:34 PM

Seems to me that this can only be fixed on server side, since that's where the SQL is generated. I will move it across to the MODLDP project.

Won't Do

Details

Assignee

Reporter

Priority

Sprint

Development Team

Thor

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs
Created December 14, 2021 at 7:54 PM
Updated November 5, 2024 at 11:40 AM
Resolved November 4, 2024 at 7:46 PM
Configure
TestRail: Cases
TestRail: Runs

Flag notifications