Update links after Authority source file's base URL update

Scope

1. Find a solution to update $0 value of instance-authority links when base url of authority source file changed.
   Cover scenario from  MODELINKS-188 - Getting issue details... STATUS :

   Create/Edit/Derive a MARC bib record

   AND user manually links bib field 100 $a Coates, Ta-Niehsi AND subfield $0 displays http://id.loc.gov/authorities/names/n2008001084

   When the protocol is updated to https://

   AND user views linked bib field 100 $a Coates, Ta-Niehsi

   Then subfield $0 displays https://id.loc.gov/authorities/names/n2008001084

2. Suggest a solution that will unify approach for updating links for both cases: when authority changed or authority source file changed.

Inputs

1. Authority and authority source file changes
2. Authority create/update/delete should be catched almost in real-time in mod-search
3. Authority update/delete should trigger updates in MARC bibs' links in mod-source-record-storage -> mod-inventory-storage
4. Authority source file update should trigger updates in MARC bibs' links in mod-source-record-storage -> mod-inventory-storage
5. Authority could have a lot of links (many events)
6. Authority source file could be related to millions of authorities, each of could have a lot of links (huge amount of events)

Solution

Phase 1: Use same approach as for authority changes

Approach

1. Production of Domain Events: The system triggers the production of domain events when changes are detected in the Authority Source File (ASF).

2. Consumption of Domain Events: ASF-linked events are consumed by mod-entities-links for processing.

3. Verification of BaseUrl Changes: A mechanism is implemented to detect whether or not there is any modification in the baseUrl of the ASF.

4. Production of Link Update Events: If a difference in the baseUrl is detected, the system responds by generating link update events. It does this by leveraging its knowledge of the assignment between authorities and the ASF and authorities and links.

Pros/Cons

Details

@startuml
actor User as u
participant SourceFileService as sfs
participant SourceFileChangeService as sfcs
database Database as d
queue "authorities.authority" as qaa
queue "authorities.authority-source-file" as qaasf
participant "mod-source-record-storage" as srs

u -> sfs : create/update/delete
sfs -> qaa : produce domain event
qaa -> sfcs : consume domain event
loop in batches
  sfcs -> d : fetch links (based on assigned authority)
  sfcs -> sfcs : prepare batch message for links update
  sfcs -> qaasf : produce link update event
end
qaasf -> srs : consume link update event
srs -> srs : update source records

@enduml

Phase 2: Use transactional outbox pattern as extension for existing approach

Details

1. Production of Domain Events: Just as with Option 1, changes to the ASF spark the creation of domain events.

2. Utilization of the Transactional Outbox Pattern: Unlike Option 1, this approach employs the Transactional Outbox Pattern. This pattern is utilized for both changes to the authorities and the ASF.

3. Consumption of Domains Events: The created domain events are consumed for processing.

4. Integration of Transactional Outbox Pattern for Link Updating: An engine implementing the Transactional Outbox Pattern is established to ensure that all link update events are handled efficiently and effectively, especially in the event of system failures.

5. Detailed approach is described in Implementing the Transactional Outbox pattern

Pros/Cons