Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

POC: Add /authorities/ids API for Authority Records search

Description

 Requirements/Scope:

  1. Extend mod-search.yaml with new API /authorities/ids

  2. Add unit/integration tests

Acceptance criteria:

  • New API for /authorities/ids is exist

  • Unit/Integration tests created

Proposed solution:
A possible solution for streaming large volumes of IDs. To begin with, it is worth dividing streaming into two operations. Stream ID from the ES to temporary storage, and the next step is to stream unique IDs to the end-user. As temporary storage, it is most convenient to use the Postgres database.
At the request, it is possible to create a temporary table in which to add the IDs from the ES. To optimize export, name the table with the CQL query hash and reuse it. A periodic job can delete result tables after it is no longer needed.

Environment

None

Potential Workaround

None

CSP Request Details

None

CSP Rejection Details

None

CSP Approved

None

Checklist

hide

TestRail: Results

Activity

Show:

Mikhail Fokanov May 6, 2022 at 2:31 PM

The implementation looks good to me.

Magda Zacharska March 18, 2022 at 7:12 PM

- please coordinate with Pavlo Smahin (Spitfire Dev lead)

Oleksii Kuzminov February 9, 2022 at 10:24 AM

A possible solution for streaming large volumes of IDs. To begin with, it is worth dividing streaming into two operations. Stream ID from the ES to temporary storage, and the next step is to stream unique IDs to the end-user. As temporary storage, it is most convenient to use the Postgres database.
At the request, it is possible to create a temporary table in which to add the IDs from the ES. To optimize export, name the table with the CQL query hash and reuse it. A periodic job can delete result tables after it is no longer needed.
CC

Magda Zacharska December 10, 2021 at 3:45 PM

The story is needed for exporting of authority records - the implementation will be revisited  once the work on exports starts.

Natalia Zaitseva December 9, 2021 at 10:01 AM

This story needs to be re-estimated on the next grooming 10 Dec 2021

Done

Details

Assignee

Reporter

Labels

Priority

Story Points

Sprint

Development Team

Falcon

Fix versions

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created October 1, 2021 at 12:12 PM
Updated July 8, 2022 at 11:48 AM
Resolved May 6, 2022 at 2:32 PM
TestRail: Cases
TestRail: Runs