[Instance/Item/Holding/Bib/Authority Audit] Extend a scheduled job for partitions deletion/creation

Description

Extend a scheduled nightly job(which is used for expired records cleanup) to clean up empty partitions preceding current period and create new if required (TBD, probably one quarter in advance)
Acceptance Criteria:

  1. If the related subpartition of the table is empty, then the subpartition should also be dropped

  2. The job should create additional partitions by date when last date partition is month or greater after current date.

  3. The job is applied to all entity types

Environment

None

Potential Workaround

None

Attachments

26

Checklist

hide

Activity

Show:

Valery_Pilko March 7, 2025 at 10:33 AM
Edited

- tested retention period and his tests are passed.
I moving this ticket in PO review.

But, I will use comments of this ticket to document testing scenarios related to MARC records.
Note: Testing of specified below scenarios could take several days, that’s why I suggest closing this ticket and in the case of any issue we will create a separate one.

I’ve checked retention policy of “Version history” for MARC records on Eureka Sprint testing environments:

  1. Retest - NON-ECS MARC bibliographic case:
    The following record was created:


    999 $i c5415001-ae59-4e35-89e5-d0a1846bb2e9 $s 7316c92e-552f-4cd2-817a-bc8916462191
    3 versions in total
    Retention period was set for 1 day.
    Expected/Actual result: After 1 day: No versions are shown in “Audit log” API for that record, see attached screenshot:
    Works as expected:



  2. RETEST - NON-ECS MARC authority case:
    The following record was created:

    999 $s cf875338-f692-4db9-b410-7bdab8c8850f $i b26361f3-4240-4e24-b4c9-84a01d653cd4
    3 versions total
    Retention period was set for 1 day.
    Expected/Actual result: After 1 day: No versions are shown in “Audit log” API for that record, see attached screenshot:
    Works as expected:

     

  3. NON-ECS MARC authority case "Version history" of Deleted "MARC authority" record is cleared when retention period is set to "1" day:
    The following record was created and deleted:


    $s 5ad1842b-b6c3-4886-9680-604d48bb6133 $i 5928ef11-1657-4cd1-92c6-0795fd3eca5f
    3 versions total
    Retention period for “Version history” was set for 1 day.
    Expected: After 1 day: No versions are shown in “Audit log” API for that record.
    Actual result: Only 1 “version” is shown in “Audit log” API for that record with “Deleted” action - Works as expected.

     



  1. NON-ECS MARC authority case "Version history" of Deleted "MARC authority" record is cleared when retention period of Authority archive expiration is set to "1" day:
    The following record was created and deleted:


    $s 5efa65d5-bc56-4ada-8cc9-71733c887cc3 $i 6db5e9d4-63bb-42e0-aa67-303914c2ee17
    Retention period for “Authority storage archive expiration” was set for 1 day.
    Expected: After 1 day: No versions are shown in “Audit log” API for that record.
    TBD - 15 march

  2. ECS MARC bib case "Version history" of Shared "MARC bibliographic" record is cleared when retention period is set to "1" day for Central tenant:
    The following record were created:
    SHARED BIB

    $i 3ed51ece-a855-461f-b14d-2aa62eb5d189 $s d9cbbdac-0638-442a-b900-cf46ad21fbae
    3 vers

    LOCAL BIB

    $i 25fea9e8-4673-411f-8f2f-6df0e578d727 $s 8b3e01c3-6eb3-44ac-8ebd-f8b3ef068b03
    3 vers
    Retention period was set for 1 day only on Central tenant
    Expected result: After 1 day: No versions are shown in “Audit log” API for Shared record, but versions still shown for Local record.
    Actual result: Works as expected for Shared and Local MARC bibs.




  3. ECS MARC bib case - "Version history" of Local "MARC bibliographic" record is cleared when retention period is set to "1" day for Member tenant -
    The following record were created:
    SHARED BIB

    $i 6e4d9dd8-7b48-49a6-819a-f5e5281d0a6c $s 81b34ccd-f76a-499a-9194-904d3e5c4b03
    2 versions

    LOCAL BIB


    $i 25fea9e8-4673-411f-8f2f-6df0e578d727 $s 8b3e01c3-6eb3-44ac-8ebd-f8b3ef068b03
    3 vers
    Retention period was set for 1 day only on Member tenant
    Expected result: After 1 day: No versions are shown in “Audit log” API for Local record, but versions still shown for Shared record.
    Actual result: Works as expected. Only versions of Local records are deleted.

     




  4. ECS MARC authority case - "Version history" of Shared "MARC authority" record is cleared when retention period is set to "1" day on Central tenant -
    The following record were created:
    SHARED

    $s 30f683d4-35fe-49db-9f5c-9d0983d67d78 $i f0d96c04-780e-45e6-abab-8219c610cc58
    3 vers


    LOCAL


    $s bf59cc56-3d49-40e6-b8da-757e2a6c728a $i 8cded3ae-c51c-43ec-aa55-b25ad31c5009
    3 vers
    Retention period was set for 1 day only on Central tenant
    Expected result: After 1 day: No versions are shown in “Audit log” API for Shared record, but versions still shown for Local record.
    Actual result: Works as expected for Shared/Local MARC authority records:




  5. ECS - MARC authority case "Version history" of Local "MARC authority" record is cleared when retention period is set to "1" day on Member tenant
    The following record were created:
    SHARED

    $s 5e54b55d-746e-4044-b788-0defca9877cc $i d4f465e7-a3df-4a25-a8e8-0e03b39d7ac3
    2 vers

    LOCAL

    $s bf59cc56-3d49-40e6-b8da-757e2a6c728a $i 8cded3ae-c51c-43ec-aa55-b25ad31c5009
    4 vers
    Retention period was set for 1 day only on Member tenant
    Expected result: After 1 day: No versions are shown in “Audit log” API for Local record, but versions still shown for Shared record.
    Actual result: Works as expected. Only versions of Local record are deleted.


Viacheslav Kolesnyk February 19, 2025 at 10:24 AM

How much partitions in advance do you think we need? One quarter in advance?

Done

Details

Assignee

Reporter

Priority

Story Points

Sprint

Development Team

Spitfire

Fix versions

Release

Sunflower (R1 2025)

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created February 19, 2025 at 10:03 AM
Updated March 14, 2025 at 8:12 AM
Resolved March 7, 2025 at 1:27 PM
TestRail: Cases
TestRail: Runs