Target architecture
The following table shows how entities from Conceptual diagram (see Conceptual model page) are mapped to the solution components:
Conceptual diagram entity | Solution component |
---|---|
Audit Logs interceptor | mod-audit-logs-interceptor, Audit logs interceptor library |
Audit Log storage | Kafka |
Audit Logs aggregator | mod-circulation-audit-logs-aggregator |
Materialized view | ElasticSearch |
Audit Reports provider | mod-circulation-audit-logs-provider |
Folio UI | Folio UI |
Key decisions
ElasticSearch should be used to store data because of:
- Audit logs may contain 100s of millions of records
- Audit data is immutable. Audit records shouldn't be updated. The records should be deleted/archived in batches once they expire.
- A full-text search is required to be used to find Audit records.
- It is required to return the total number of records found for a search query. The number should be included in response to the original query. Query response time should be less than a few seconds.mod-audit-log-interceptor should be implemented as a separate module and be deployed as a side-car with modules that contain Audit data. It should work as a proxy that intercepts these modules requests/responses. This approach allows to add the interceptor without modifying target modules.
However, it should be also possible to use the Audit logs interceptor library as an alternative. The library should be built into the Vertex request-response pipeline. As a result, there will be different tools to collect Audit data.
Transient architecture
The following table shows how entities from Conceptual diagram (see Conceptual model page) are mapped to the solution components:
Conceptual diagram entity | Solution component |
---|---|
Audit Logs interceptor | Audit logs interceptor library |
Audit Log storage | mod-pubsub, Kafka |
Audit Logs aggregator | mod-circulation-audit-logs-aggregator |
Materialized view | PostgreSQL |
Audit Reports provider | mod-circulation-audit-logs-provider |
Folio UI | Folio UI |