High-level solution structure

Target architecture

The following table shows how entities from Conceptual diagram (see Conceptual model page) are mapped to the solution components:

Key decisions

1. ElasticSearch should be used to store data because of:
        - Audit logs may contain 100s of millions of records
        - Audit data is immutable. Audit records shouldn't be updated. The records should be deleted/archived in batches once they expire.
        - A full-text search is required to be used to find Audit records.
        - It is required to return the total number of records found for a search query. The number should be included in response to the original query. Query response time should be less than a few seconds. 

2. mod-audit-log-interceptor should be implemented as a separate module and be deployed as a side-car with modules that contain Audit data. It should work as a proxy that intercepts these modules requests/responses. This approach allows to add the interceptor without modifying target modules.
   However, it should be also possible to use the Audit logs interceptor library as an alternative. The library should be built into the Vertex request-response pipeline. As a result, there will be different tools to collect Audit data.

3. mod-audit-log-interceptor should only capture Audit data and transfer it as-is via Kafka to mod-circulation-audit-logs-aggregator. mod-circulation-audit-logs-aggregator in its turn should convert source data into the target schema and format and post it to ElasticSearch. As a result, the interceptor should spend little time for data capturing.

Transient architecture

The following table shows how entities from Conceptual diagram (see Conceptual model page) are mapped to the solution components: