2022-10-19 Meeting notes: Cross-app data sync update and outcomes

Date

19 Oct 2022

Housekeeping

• Convener and notes: Martina Schildt
• Next meeting: Oct 24th | topic: UX patterns for common cross-app tasks

Discussion items

1. • Cross-app data sync update and outcomes of working group Owen Stephens 
   • AI Slack channel: open vs closed - if time allows

Minutes

• working group has been working to find a common pattern for data sync use cases that have been collected in this group (and other groups)
• Raman has written a proposal based on Kafka < does not solve all given use cases, but most of them
• approach: ADR-000004 - Cross application data sync proposal
• approach was presented to the TC | recorded as ADR = Architecture Decision Record
• TC recommends this approach | use of Kafka to keep data consistency
• it was a long way around approving a practice that is already in use in FOLIO
• new pattern: inbox/outbox pattern | audit whether changes are read by subscriber | TC did not adopt that because it has not been used so far
• it seems as if teams just need to do something to have it approved by TC | we need to discuss whether this is a good way
• Brooks in chat: Not having outbox also means that you could encounter a situation where a record is not updated in the database, but an update event is published and consumed… | Charlotte agrees
• Owen & Marc: what we gain with this approach is better than what we have before
• Brooks agrees
• Marc in chat: Whilst the outbox pattern helps, it is not sufficient to guarantee the consistent state (which is generally very hard to achieve)
• E.g.: the Circ log app not knowing about an item being deleted: UXPROD-3738 Circulation log update: deleted item records
  
  • Marc in chat: You can mitigate/minimize that possibility…
    Not sure how that applies here…
  • Marc: all of the technical background stuff is in place (since the search engine based search work)
• ADR gives us a way of how changes can be communicated
• Up to POs and dev teams to decide when it is time to do this
• to keep in mind: this approach does not give us one way
• consumer is more interested in keeping data in sync than the producer | this is a challenge | need to talk about prioritization | apps are dependant on development on both sides | this is not solved through this solution
• what can users expect from this solution
  • data update not in real time | most likely not very delayed (timely, but not instantaneous)
  • cannot know definitely | will depend on lots of variables
  • Marc in chat: For folks who want to understand the rough expectations of these designs, they can consider the search in inventory. For example, folks know there is a discrepancy when returning to a search immediately after editing a record.
  • Dung-Lan in chat: What people can expect should be made as clear as possible to reduce confusion or frustration | the group agrees
• idea: expose to the user whether looking at original data or copy of data → discuss in this group in a future meeting
• new pattern is in use for communication between Inventory and Elastic Search | Marc will check and give more information
• possible next step: find dev team to start using the new approach for one of our use cases | maybe modorders-642 would be a good starting point | we need Dennis to decide on this | discuss in PO meeting? | not needed, promotion is sufficient
• Next steps: 
  • we need to promote these things are unblocked | PO meeting
  • feedback somehow the frustrations of the process | maybe in PC with TC
  • present approach and what to expect in PC call with SIG conveners are present (SIG convener updates)

Next steps

• we need to promote new approach | in PO meeting
• feedback somehow the frustrations of the process | maybe in PC with TC
• present approach and what to expect in PC call with SIG conveners are present (SIG convener updates)

Future topics

• Topic proposal by Owen Stephens for October:
  • UX patterns for common cross-app tasks (e.g. would it be useful to have a UX for a 'quick add' task that could be used in different contexts when you need to create something in another app). I think it would be good to have UX specialists (Kimie, Gill) present for this, and possibly also some Stripes devs (e.g. John C/Zak)
    • fast-add is a good model - Laura will demo 
    • Owen has some examples from invoice creation to talk about
    • Martina S. will check whether Gill and Kimie will be available
    • Date: Oct 24th
  • Use of shortcut keys and macros for more effective cross-app working  - it also be good to have UX and Stripes/dev knowledge for this discussion I think. I know @Laura (she/they) uses macros so might have insights into the potential for cross-app working
  • Potential for external 'workflow' solutions for cross-app interactions
    • I think 'workflow' is a dangerous term here - in this context it's more about automation than user workflows, although I think there is overlap
    • I was particularly struck by the solution in production at TAMU (Jeremy Huff and Sebastian Hammer presented, the recording is at https://prod-zoom-recordings-openlibraryfoundation-org.s3.amazonaws.com/50dc6c87-3912-43fa-8287-56ec73b12bbb%2Fshared_screen_with_speaker_view%28CC%29.mp4 starting at 3 hrs, 14 min) - I think getting someone from TAMU to talk about how this is used would be v interesting
    • There was also a presentation on the use of a tool called Airflow at Stanford for "bibliographic workflow" but I've not watched that yet so not 100% sure if it is completely applicable - I think the core use case there was systems migration but it may go beyond that
• UX/UI and implementers topics
  • should be Wednesdays

Attendees

Action items