Reporting: Analytics and Audit Data Logging for External Reporting (UXPROD-330)

[UXPROD-1222] Implement Message Stream Filters Created: 03/Oct/18  Updated: 17/Jan/19  Resolved: 17/Jan/19

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: None
Parent: Reporting: Analytics and Audit Data Logging for External Reporting

Type: Story Priority: P3
Reporter: VBar Assignee: Hongwei Ji
Resolution: Done Votes: 0
Labels: analytics, transactdata
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
blocks UXPROD-343 Partition Transactions into Kafka Topics Closed
Defines
defines UXPROD-380 Configuration setting for Audit facility Closed
Relates
relates to UXPROD-331 Okapi Transactional Data Extraction Closed
relates to UXPROD-1221 Implement support for AES Message Str... In Review
Epic Link: Reporting: Analytics and Audit Data Logging for External Reporting
Back End Estimate: Medium < 5 days
Back End Estimator: VBar
Development Team: EBSCO - FSE

 Description   

Provide the ability to filter (select) transaction for AES Message Streams. The functionality belongs in the AES Message Stream Processor component.

The selection criteria consist of a single Boolean expression that will include (or exclude) individual transactions based on what they match to.

  • The rules support basic Boolean operations (OR, AND, NOT), precedence rules
  • Boolean monotone laws (Association, Commutation, Distribution, Identity, Annihilation)
  • The terms of the expression consist of:
    1. name-value pairs from HTTP headers;
    2. metadata available to Okapi through its discovery service (e.g. a module name or an interface id, identified using a reserved word vocabulary).
  • The terms of the selection criteria Boolean expression consist of name value pairs.
  • Values are expressed as a string.

The initial implementation will support exact matching only and stemming. Stemming (a.k.a “begins with”) is used to match module or version names, while excluding the version numbers that exist on the tail of the string.

For example,

 `“folio_search-*”` will match `“folio_search-1.1.100088”`. 

Later versions may introduce support for regular expression matching.



 Comments   
Comment by Hongwei Ji [ 03/Dec/18 ]

As discussed today, the Stream Filters will be implemented as

{"criteria":"some json path", "target":"some Kafka topic name"}

and stored in mod-config. The mod-aes will retrieve (and refresh periodically) the filter rules from mod-config and dispatch messages as defined by filter rules.

Generated at Fri Feb 09 00:13:47 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.