2022-08-03 Meeting notes

Owned by Craig McNally
Last updated: Aug 18, 2022 by Raman Auramau

Date

Attendees 

Discussion items

TimeItemWhoNotes
1 minScribeAll

 Tod Olson is next, followed by Ankita Sen 

1 minTCR Board Review

All


15-20 min

Technical Council Sub Groups Updates

All


TC Goals/Objectives: not met as group, Pain Points document still forming, using that to drive some Architecture and Tech Debt topics at WOLFcon
Translation Subgroup: got started, beginning discussion
FOLIO Scope Criteria: Two tracks: (1) Harry has revised the draft practical approach to evaluating modules, (2) still struggling with the scope and what the group is empowered to do
AWS costs: group struggling to meet
Breaking changes: conversation about rules do define breaking change for front-end and back-end modules. Looking at mechanisms for communicating about breaking changes.
Elasticsearch vs Open Search: CC setting up Slack channel (#cc-elasticsearch-opensearch-discussion) to discuss.
1 minRFCsAll
  • Nothing to review as of  
25-30 minCross-App data sync

Raman Auramau 

  • Presentation (15m-20m) & Q/A (10m): Data sync updates and outcome (TC)
  • Looking for solutions that are generally applicable and preserve loose coupling between modules
  • Changes in source of truth are to be done atomically, and then notifications sent to consumers
  • Details in proposal FOLIO Cross-Application Data Sync Solution
  • Proposal is already in use in inventory, changes in mod-inventory-storage  send notifications to be consumed by mod-search 
  • Q&A:
    • The message passing via Kafka introduces dependencies between modules. Do we have a mechanism for documenting / declaring the dependencies. Discussed, but intentionally out-of-scope for the group.
    • Are there guidelines on how retention period may change? Trade-offs, retention periods may be different based on use case, no one approach for all cases. Deliberately does not allow for replay.
    • Full-body vs. diff message: there are message sizes, and Kafka works better for small events. 10-100KB cover current known use cases.
    • Regarding outbox pattern, was the Debezium idea evaluated? No, just came up in brainstorming.
      • Debezium is used in LDP/Metadb. Debezium/WAL solutions do generate A LOT of events, which heavily impacts Cloud hosting costs.
    • Are there performance issues with Outbox pattern? Did not do specific testing, existing uses do not have numbers, but also did not state concerns about performance. Outbox will introduce some delay in notifying consumer. Different use cases have their own acceptable delays.
    • Do we need to propose a mechanism for synchronizing data in cases where events get lost.
    • What are next steps? This process pre-dates ADR/RFC process.
      • Action: group will create and ADR and ask for TC approval
      • POC & performance testing? existing work in inventory could be considered POC, but do not use Outbox pattern.
      • Can discuss next steps after TC approval
    • Group was starting from current state of FOLIO and current use cases, was working from a more constrained view than green field.
    • Current reindexing approach undermines some of the is proposal, relies upon consumer knowing the source of truth. 


Remaining TimeOther topicsAll

Several topics have been raised recently that deserve attention/discussion.  The goal here is to figure out how to best move these forward, without digging into the topics themselves.  Is there a priority here?  Should we spin up working groups for any of these?  etc.

Action Items

  •