Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Option

Description

Pros & Cons

1

RDBMS

The audit database should persist a snapshot of the entity. The queries are made mostly by the entity's unique identifier. Thus partitioning by UUID can be applied to audit tables

Pros:

  • allows flexible access to versioning data

Cons:

  • limited scaling options

  • negative impact on Postgres that is used by all others modules

2

Object Storage

AWS S3-like storage can be used to persist snapshots because audit events can be stored as plain-text (JSON) documents

Pros:

  • allows scaling almost indefinitely

Cons:

  • requires an additional solution for complex queries

  • might cause high costs for storage bc of a large amount of operations on small files

2. Version history display: This should be done on demand comparing each consecutive snapshot of the entity to the previous

...

Question

Answer

Comment

1

Should failure in sending audit message block the create/update/delete operation?

Hey Kalibek Turgumbayev - what happens today when an update is made and the create/update date and time stamp is not updated?

The question is related to transactional outbox pattern implementation

2

What would be the period of retention for Audit records?

Dennis Bridges has this requirement come up for you with respect to Acq’s change tracker?

The storage options depend on this question:

  • Postgres for ~ 1-3year

  • Postrgres with Partitioning by UUID 5-15 years

  • NoSQL options for 20+ years

3

In what form should we show the changes to non-marc fields (e.g. staffSuppress, administrative Notes, etc.) in MARC instances?

Kalibek Turgumbayev - I am unsure I understand this question. Can you review this mockup of how to display updates made to a FOLIO instance record?

Instance with source FOLIO or MARC from inventory is a separate object than SRS record and should be tracked separately

4

If only the order of fields in a MARC record is changed, should it be logged?

Kalibek Turgumbayev - Good question - I need to ask users but unless it is a significant to implement, answer is Yes.

Dennis Bridges has this requirement come up for you with respect to Acq’s change tracker?

5

Do we have scenarios where the audit log is exported in batches for some period of dates?

It is possible that a library may want to do so but I do not think it is a requirement for Sunflower. Dennis Bridges has this requirement come up for you with respect to Acq’s change tracker?

If the solution uses Postgres with partitioning by entity key, such exports would cause significant performance issues.

6

Do we need to have remapping or other technical updates as auditable events?

We might need a list of actions leading to an event in the audit log.

7

Can we show only the last 20 records (last 3 months)?

Display of the whole history of the entity will impact performance negatively.

8

What should be tracked with BIBFRAME? Should we track the original BIBFRAME record or related MARC records is enough.

  1. Acquisition event log - data retention period is 20 years

  2. Transactional outbox pattern

  3. Orders Event Log

  4. Javers - Java object diff library