Proposal is already in use in inventory, changes in mod-inventory-storage send notifications to be consumed by mod-search
Q&A:
The message passing via Kafka introduces dependencies between modules. Do we have a mechanism for documenting / declaring the dependencies. Discussed, but intentionally out-of-scope for the group.
Are there guidelines on how retention period may change? Trade-offs, retention periods may be different based on use case, no one approach for all cases. Deliberately does not allow for replay.
Full-body vs. diff message: there are message sizes, and Kafka works better for small events. 10-100KB cover current known use cases.
Regarding outbox pattern, was the Debezium idea evaluated? No, just came up in brainstorming.
Debezium is used in LDP/Metadb. Debezium/WAL solutions do generate A LOT of events, which heavily impacts Cloud hosting costs.
Are there performance issues with Outbox pattern? Did not do specific testing, existing uses do not have numbers, but also did not state concerns about performance. Outbox will introduce some delay in notifying consumer. Different use cases have their own acceptable delays.
Do we need to propose a mechanism for synchronizing data in cases where events get lost.
What are next steps? This process pre-dates ADR/RFC process.
Action: group will create and ADR and ask for TC approval
POC & performance testing? existing work in inventory could be considered POC, but do not use Outbox pattern.
Can discuss next steps after TC approval
Group was starting from current state of FOLIO and current use cases, was working from a more constrained view than green field.
Current reindexing approach undermines some of the is proposal, relies upon consumer knowing the source of truth.
The next Quarterly Community Update is approaching (last day of WOLFcon). I will be pulling together lists of accomplishments, goals, etc. soon. Please let me know if you have any ideas/thoughts. Once I have something I will share it at an upcoming TC meeting.
Remaining Time
Other topics
All
Several topics have been raised recently that deserve attention/discussion. The goal here is to figure out how to best move these forward, without digging into the topics themselves. Is there a priority here? Should we spin up working groups for any of these? etc.
How/when to make significant technology changes in coordination with the release schedules of other tools, e.g. keeping in sync with Node LTS releases, or Java 17 or Postgres 14, etc. - Marc Johnson