2024-09-27 Meeting notes (In Person)

Date

Attendees 

Discussion items

TimeItemWhoNotes
1 minScribeAll

Julian Ladisch is next, followed by Maccabee Levine

Reminder:  Please copy/paste the Zoom chat into the notes.  If you miss it, this is saved along with the meeting recording, but having it here has benefits.

*WOLFcon

All

Open discussion

  • Noteworthy sessions/discussions/etc?
    • Session by Index Data / Hebis (Maccabee)
      • Some good things came out of it
      • Positive feedback on the TCR process seemed to go well
      • We stuck to the prescribed timeline - improves our credibility
      • Useful feedback was given to the team
    • Workflow session (Jason)
      • Good input from SMEs
      • Jeremy was able to capture it
      • If/when workflows are successful, the project should discuss what that means for Folio deployment and related topics.
        • it may lead to complications on the project mgmt side
        • TC might want to discuss if the workflow engine should be added to the infrastructure stack (OST?)
        • Depending on the workflows chosen, your Folio experiences may be quite different from other libraries
    • Eureka - Several sessions
      • Including from SysOps perspective
      • Sunflower is the target
        • A little concerning to some that it seems this in on the community
          • Some of this is on the community, some on EBSCO
            • TC, Tri-council working group
          • EBSCO is going to do what they need to in order to meet its commitments.
          • It does seem like there's some momentum growing behind Eureka adoption.
        • What exactly does need to happen?
          • RFC is still required.  This is on Craig/Vince.
            • What is the driver behind this?
              • Documentation?
              • Feedback?
              • Both!
            • Getting the scope right is tricky but very important.
          • Update processes (e.g. TCR, etc.)
          • Documentation
            • On many fronts - user facing, sysops, developer, etc.
          • Application composition needs to be sorted out
        • There's some acknowledgement that there needs to be support for both Okapi & Eureka for some period of time
          • Various impacts include:  
            • Cost of running hosted reference/test/etc. environments for both platforms
            • Cognitive complexity - having devs, sysops, users, etc. needing to know both
        • The adoption or large architectural changes have been a bit messy in the past (Kafka/pub-sub, etc.)
      •  Application formalization
        • Several sessions on this
        • Being looked at by different groups, from different perspectives - Functional & Technical
        • We need to be careful to not conflate the application composition topic with the adoption of Eureka in general
          • It's possible to adopt Eureka, even without having ideal applications
        • Who actually decides what the applications composition looks like?
          • Next steps?
            • Find the ideal composition from the functional angle
            • Do the analysis on the technical side and get us to a good starting point - what can be done w/o code changes?
            • Identify the gaps between the two and plan/prioritize the work
            • Generate documentation for how to simplify the dependency tree, pros/cons, when each technique is applicable, etc.
              • Eureka will seed this
            • Try to find a team to attempt to resolve some of the issues in their modules
              • We can then point to this as concrete examples
    • Spokane public library developed (or are developing) a couple modules (Maccabee)
      • Languages used include Python, and possibly Go
      • It isn't clear if the intent is to contribute these modules to the community 
      • Are there ways to communicate to the broader community about technical things - e.g. OST, best practices, processes, etc.
        • Can/should we try to be more proactive on this stuff?
      • Language selection does matter
        • It may impact hosting cost - both for libraries/hosting providers, but also the community
      • We need to be careful not to scare away (potential) contributors
      • It would be great if there was more information sharing about some of these "outside the community" development activities on an ongoing basis, rather than waiting until WOLFcon.
        • Not only modules, but also external tooling, etc.
        • Relates to something discussed during the national library panel
      • A lot of emphasis is on code quality, but interfaces are not scrutinized as much
        • Maybe we should, but not at the cost of making the TCR process more complex
        • Consistent API design, dependency tree implications
          • Many APIs are structured around CRUD operations
            • Sometimes this is a square peg/round hole situation.
            • It isn't always clear looking at an API exactly what the behavior is behind the scenes (without looking at the code, or even in some cases in the database queries)
            • Documentation can help here
              • It would be helpful if the project could provide some guidance on API design - dev.folio.org?
          • APIs can and do evolve
            • Maccabee cited the instance-view API as an example
          • There was a situation discussed about a higher level circulation API
            • Devs were dissuaded from even creating JIRAs on this, which shouldn't be the case.
          • API documentation can be improved
            • It isn't always clear what the behavior of an API is
            • It would be helpful for the project to provide (better) guidance on how to document APIs
              • e.g. with details in the README, referred to in the generated API docs
              • The best place to document something depends on the target audience
                • READMEs are more developer-facing, whereas the wiki is more likely to be used by others, e.g. Product Owners, etc.
                • Cross-referencing is a good idea and should help people find the information they're looking for easier.
            • Some teams/modules have done a good job of this
              • mod-Z39.50 was mentioned as an example of a place where this was done well.
              • Pointing to examples would also be helpful
        • Not necessarily being prescriptive about this, but at least have a conversation
    • Data Security for the Lists App (Jenn)
      • Weren't we supposed to get an RFC on this?
      • Now it's built and presented on.  Too late?
        • Maybe - even if only for documentation purposes
        • There were some questions in the session, so there could still be valuable feedback
    • Onboarding development teams (Jenn/Maccabee)
      • The process seemed very prescriptive, and parts of it were a little odd.  Examples:
        • Why do teams go to the RMS group?
        • "You build it, you own it"
      • If these processes are working out, maybe they should be formally captured / embraced by the community
      • Theme - communication is very important
      • Related discussion - not a new module, but a developer/developers contributing to an existing module
        • Often a bad start, but happy ending.
        • Can/should we try to define/formalize recommended approaches/processes for this too?
        • Not specific to just developers, but also other roles (PO for examples)
  • Any next steps/action items for the TC?
    • Keep doing:
      • All: honor our timelines (e.g. for TCRs, etc.)
    • Start doing:
      • All: try to stick to our timelines where we haven't in the past (e.g RFCs, etc.)
      • Action: (Jenn/Craig) ensure there are discussions around how processes may change wrt Eureka adoption (e.g. TCR, etc.) 
        • Form a new working group after the existing tri-council application-formalization working group wraps up?
      • Action: (Eureka/Early Adopters/FSE/Kitfox) to generate Eureka documentation (end-user facing, dev facing, sys-ops facing, etc.)
      • Action: (Craig) ensure that there's alignment on expectations with Early adopters - meetings are likely required.
      • Action: (...) 
        • (App formalization working group) Find the ideal composition from the functional angle
        • (Eureka) Do the analysis on the technical side and get us to a good starting point - what can be done w/o code changes?
        • (Next iteration of a tri/bi-council working group) Identify the gaps between the two and plan/prioritize the work
          • Depends on the previous two activities
        • (Eureka/Craig) See documentation for how to simplify the dependency tree, pros/cons, when each technique is applicable, etc.
        • (Craig) Try to find a team to attempt to resolve some of the issues in their modules
      • Action: (Craig/Vince) - get the Eureka RFC created ASAP!
      • Action: (Jenn - via app formalization working group) - Enumerate/document the decisions (both made and todo) around Eureka
      • Action: (Craig/Jenn) - schedule one (or more) dedicated TC discussions on interfaces, API design guidance, documentation guidance (see notes above).
      • Action: (Craig/Jenn) - schedule a dedicated TC discussion on dev team onboarding and cross-dev team collaboration - generating guidance
      • Action: (Craig) - As a first step, Ask Lee to get the information in his slides (dev team onboarding) formalized on the wiki so it's more prominent and easier to find. 
    • Follow-up:
      • Action: Discuss if the workflow engine should be added to the infrastructure stack (OST?). E.g. particular workflow engine will affect how workflows are implemented (what programming languages tasks may be programmed in, perhaps other effects.)
      • Action: (Craig) RFC for data security of Lists App/FQM
  • Tri-council meeting prep
    • Discuss next steps for Eureka adoption, including timelines, action items, etc.
    • Raise the desire/need for more awareness of development efforts outside "the community" teams - maybe the CC can help with this.
      • 2-directions: Awareness of development activity occurring at different organizations, and awareness of best practices for FOLIO (languages, frameworks, conventions...)
    • Suggest the idea of periodic meetings to "show and tell"/raise awareness of development efforts - scripts/tooling/modules/etc.
      • Including ideas, and work in progress.  Not just finished work
    • Raise the topic of dev team onboarding - is there a better starting point for those wanting to join than the RMS group?
      • Maybe RMS group is the place, but with more involvement from others, e.g. PC/CC?
    • Meeting hygiene - TC and PC dipped their toe in the "international meeting times" pool... how'd it go?  Next steps?
      • Maybe try alternate approaches:  e.g. ask those who can't usually attend council meetings (e.g. Asia/Pacific) to drive this; generate the agenda, and inform the councils when meetings are needed. 
NAZoom Chat



Topic Backlog

Decision Log ReviewAll

Review decisions that are in progress.  Can any of them be accepted?  rejected?

Translation SubgroupAllSince we're having trouble finding volunteers for a subgroup, maybe we can make progress during a dedicated discussion session?
Communicating Breaking ChangesAll

Currently there is a PoC, developed by Maccabee Levine, of a utility to catalog Github PRs that have been labeled with the "breaking change" label. We would like to get developer feedback on the feasibility of this label being used more often, and the usefulness of this utility. 

Officially Supported Technologies - UpkeepAll

Previous Notes:

  • A workflow for these pages. When do they transition from one state to another. Do we even need statuses at all ?

Stripes architecture group has some questions about the Poppy release.

Zak: A handshake between developers, dev ops and the TC. Who makes that decision and how do we pass along that knowledge ? E.g. changes in Nodes and in the UI boxes. How to communicate this ? We have a large number of teams, all have to be aware of it.  TC should be alerted that changes are happening. We have a couple of dedicated channels for that. Most dev ops have subscribed to these channels. How can dev ops folk raise issues to the next level of community awareness ? There hasn't been a specific piece of TC to move that along.

Craig: There is a fourth group, "Capacity Planning" or "Release Planning". Slack is the de facto communication channel.  There are no objections to using Slack. An example is the Java 17 RFC. 

Craig: The TC gets it on the agenda and we will discuss it. The TC gets the final say.

Marc Johnson: We shouldn’t use the DevOps Channel. The dev ops folks have made it clear that it should only be used for support requests made to them.

Jakub: Our responsibility is to avoid piling up technical debt.

Marc: Some set of people have to actually make the call. Who lowers the chequered flag ?

Craig: It needs to ultimately come to the TC at least for awareness. There is a missing piece. Capacity Planning needs to provide input here. 

Marc: Stakeholders / Capacity Planning could make that decision. Who makes the decision ? Is it the government or is it some parts of the body ?

Marc: the developers community, the dev ops community and sys ops are involved. For example the Spring Framework discussion or the Java 17 discussion. But it was completely separate to the TC decision. It is a coordination and communication effort.

Marc: Maybe the TC needs to let go that they are the decision makers so that they be a moderating group.

Jakub: I agree with Marc. But we are not a system operating group. Dependency management should be in the responsibility of Release management. There are structures in the project for that.

Jason Root: I agree with Jakub and with Marc also. Policies should drive operational/release/support aspects of Folio.

Jason Root: If the idea of “support” is that frameworks are supported, then of course the project should meet that.

Marc Johnson
Some group needs to inform OleksAii when a relevant policy event occurs.
These documents effectively ARE the manifestation of the policy.

Craig: This is a topic for the next Monday session.

Craig to see if Oleksii Petrenko could join us to discuss the process for updating the officially supported technologies lists.


Dev Documentation VisibilityAll

Possible topic/activity for a Wednesday session:

Discuss/brainstorm:

  • Ideas for the type of developer-facing documentation we think would be most helpful for new developers
  • How we might bring existing documentation up to date and ensure it's consistent 
  • etc.
API linting within our backend modulesAll

https://folio-project.slack.com/archives/CAQ7L02PP/p1713343461518409


Hello team, I would like to discuss API linting within our backend modules. Some time ago, we transitioned our linting process from Jenkins to GitHub Actions as outlined in https://folio-org.atlassian.net/browse/FOLIO-3678. I am assuming that this move was done via some technical council decision. Please correct me if I'm wrong.
In my observations, I've found two problems:
  1. Schema linting does not occur if the schemas are in YAML format.
  2. There are issues with resolving some deeper references during API linting.
Although I'm unsure about how to improve the existing linting implementations within Folio, I propose to consider an open-source solution that handles OpenAPI linting effectively and allows us to define custom rules. For your reference: https://stoplight.io/open-source/spectral A test of this solution can be found in this PR: https://github.com/folio-org/mod-search/pull/567. The same PR also provides an example of custom rule definition: https://github.com/folio-org/mod-search/pull/567/files#diff-d5da7cb43c444434994b76f3b04aa6e702c09e938de09dbc09d72569d611d9ab.Also, by employing 'Spectral', I discovered AsyncAPI (https://www.asyncapi.com/en), an API design tool similar to OpenAPI but for asynchronous interactions. I suggest that we consider using AsyncAPI in FOLIO to generate documentation for Kafka interactions.


PR TemplatesAll

https://folio-project.slack.com/archives/CAQ7L02PP/p1713445649504769

Hello team, Small request to consider.
Regarding pr templates.
  1. From my perspective, pr template is not good idea. Even the biggest open source projects that are contributed by many people don't have any pr template. Currently what we have for acq modules https://github.com/folio-org/mod-orders-storage/blob/master/PULL_REQUEST_TEMPLATE.md
  2. These pr template is inconsistent in different teams.
What I suggest is that, pr template shouldn't be any instructions, because most developer who are creating pr have already understand the rules. If we put just two section into template, it will encourage developers to write more about their work and that lead to knowledge  sharing among developers.
Proposed Mod KafkaAll

https://folio-project.slack.com/archives/CAQ7L02PP/p1714471592534689

Mike Taylor

Proposal. If and only if a FOLIO instance is running Kafka, it should insert and enable a module called mod-kafka, which consists entirely of a module descriptor that says it provides the interface kafka. The purpose is so that other modules can use the standard <IfInterface> and similar tools to determine whether they should attempt Kafka operations. Rationale: the FOLIO ILS depends absolutely on Kafka, but other uses of the platform will not. One such example: a dev platform that includes only mod-users, used as a source of change events for Metadb.

Action Items