2024-09-11 Meeting notes

Owned by Jenn Colt
Last updated: Sep 12, 2024 by Tod Olson

Date

Attendees 

Discussion items

TimeItemWhoNotes
1 minScribeAll

Ingolf Kuss is next, followed by ????

Reminder:  Please copy/paste the Zoom chat into the notes.  If you miss it, this is saved along with the meeting recording, but having it here has benefits.

5-10 minLiaison Updates

Maccabee Levine

Tod Olson

Jakub Skoczen

Craig McNally

Jenn Colt 

  • CC: Maccabee Levine
    • There was a presentation and discussion about AWS costs, which are significantly over budget.  CC decided on an amount to limit the monthly spend, that ACRG or others would figure out how to achieve, and they are writing something up for clarity. 
    • They want to send out a detailed message, haven't done so yet.
  • PC: Tod Olson
    • Presentations on mod-marc-migrations and mod-record-specifications
    • Voting ongoing, so far seems positive
    • UPDATE (2024-09-12): mod-marc-migrations and mod-record-specifications approved by PC
  • RMS Group:
    • No meeting
  • Security Team: 
    • Nothing special to report.  Business as usual
  • Tri-council Application Formalization:  
    • No meeting last week. Discussions about Eureka impacts as they surface.
1 minUpcoming MeetingsAll
  •  - Regular TC Meeting
  •  - Regular TC Meeting
  •  - Dedicated discussion: Continuation of architectural decision-making discussion
  •  - Cancelled: WOLFcon
  •  - Cancelled: WOLFcon
  •  - TC & WOLFcon
5 minTCR Board ReviewAll
  • ui-reading room to be presented at PC on 12 Sep
  • mod-marc-migrations; Jeremy was out last week. No news. Need to respect deadline for release. Jason will get in touch with Jeremy.
  • mod-record-specifications. Evaluation is going on; looks good, don't expect blockers. Ingolf will make a proposal to change the criterion for SNAPSHOT-builds. Julian made a porposal to expand the criterion for consumed APIs. The TCR-45 should be decided on by the TC before WolfCon. Jenn schedules a discussion (next Monday).
5 min

Technical Council Sub-Groups Updates

All

  • Reference Data Upgrades: last week a subgroup was suggested, need to come back to this.
  • Static Code Analysis: 2024-08-20 Static Code Analysis Meeting notes . The group investigated the current Sonar Cloud settings. Only a few changes to the Default Settings have been made be the development teams. These changes do not apply to the whole project, but have been made on a per-module basis. The group proposes the following set of guidelines:
    • Default rules for each code scanner per language are the standard.
    • The team is free to change the rules according to their development needs.
    • The team is free to set up a new code scanner without explicit TC approval
    • The changes need to be made explicit to the reviewer and reasonably justified.
    • The reviewer is a member of the TC who does the code review

    Disagreement as to whether one is obliged to use a certain analysis tool for certain projects (e.g. Sonar Cloud for Java Projects). Some say yes, some say no.

    Marc: Development teams need more guidance as to what tool and what set of rules to use.

    Others say it should be up to the team (and the duty of the team) to choose a tools and setup and to document and justify their choice.

    The Pull Request to folio-org/tech-council needs to be worked on before we can decide on it. E.g. discussions need to be closed. The 3% figure for code duplication needs to be dropped if teams are allowed to fine-tune theirselves.

    Some say if a new development language comes up, the first team who uses it must propose a code analysis tool and then all other teams must use it (for this programming language). Caveats because projects can have multiple languages and different coding styles. Contradictory to the current proposal that the team is free to set up a new code scanner (even for an existing language).

    Jenn: The proposal of the subgroup is not ready to be voted on. There is disagreement among the group members.

    The group must meet again. Results will not be brought back to TC before WolfCon.


1 minRFCs

All

Reminder(s)

  • Review of open items, such as the need for decision log records and the application of the "Go" RFC.
    • Still need merge and decision log entry
1 minDecision LogAll

Need to log decisions for the following:  (see above)

25 min

Officially Supported Technologies (OST)

All

still open:

  • Java 21 discussion from the DevOps perspective.
  • Vertex.5 - we will wait for the capacity discussion

Last week:

Check Recurring Calendar...

  • Spring Boot 3.4 (and Spring Framework 6.2)
    • Today RMS group has made the decision, see Liaison Updates section above.
  • Sunflower
    • The Spring Boot support period of 12 months for a single minor release has the same length as FOLIO's support period of ~ 12 months, however, these periods usually don't align.
    • Regarding Backend "First Party Libraries / Frameworks": A library/framework needs to support only one of the versions allowed in the "Third Party Libraries / Frameworks" list. This should be clarified in the description.
    • Adding folio-s3-client to first party libraries and the AWS and MinIO clients to third party libraries. folio-s3-client is an abstraction for the AWS and MinIO client.
    • Java 21
      • No objections from developers, the migration from 17 to 21 will be easy, simply bump the folio-java-docker image in the Dockerfile and the jenkins-slave-docker image in the Jenkinsfile.
      • Need to contact DevOps whether they can provide a Java 21 version of folio-java-docker and jenkins-slave-docker before the Sunflower development starts; recent commits (folio-java-docker history, jenkins-slave-docker history) have been made by David Crossley, John Malconian, and Julian Ladisch.
        • Craig McNally is reaching out to Mike Gorrell to see if there is capacity in time for Sunflower.
        • If yes, next step will be to create a ticket for DevOps
    • Vertx 5
      • Major release, some changes to how callback, futures, etc.; may affect how we chain things.
      • Julian has tried and found little difficulty with RMB
      • Arguments for holding off until Trillium, not want to upgrade both Vertx and Java, but signal this is coming
      • Note: Trillium expected in Fall '25, assuming no delays; Sunflower needs support for two releases, gets very close to end of Vertx 4 support if there are delays.
      • Current inclination is towards Sunflower, but not solid decision
    • For Java 21 & Vertx 5: will wait for Java 21 capacity conversation, make decision next week.
    • S3/MinIO - S3 API version: still need version, see Action Item below

Previous Notes:

  • Java 21
    • Recap:
      • No objections from developers, the migration from 17 to 21 will be easy. (See above.)
      • Need to contact DevOps whether they can provide Java 21 environment as above.
    • Actions
      • Craig McNally is reaching out to Mike Gorrell to see if there is capacity in time for Sunflower.
      • If yes, next step will be to create a ticket for DevOps
      • Note: KeyCloak has already deprecated support for Java 17 and will remove in next version. Support KeyCloak through a Bouncy Castle library. Bouncy Castle support for Java 21 does not mesh well. KeyCloak is aware, could possibly adjust.
  • Vertx 5
    • Major release, some changes to how callback, futures, etc.; may affect how we chain things.
    • Julian has tried and found little difficulty with RMB
    • Arguments for holding off until Trillium, not want to upgrade both Vertx and Java, but signal this is coming
    • Note: Trillium expected in Fall '25, assuming no delays; Sunflower needs support for two releases, gets very close to end of Vertx 4 support if there are delays.
    • Current inclination is towards Sunflower, but not solid decision
  • For Java 21 & Vertx 5: will wait for Java 21 capacity conversation, make decision next week.
  • S3/MinIO - S3 API version: still need version, see Action Item below
  • OpenSearch 2 & Elasticsearch 8:
    • Some differences have crept in.
    • FOLIO currently uses only the basic features thought the OpenSearch client that are available in both.
    • Marc Johnson has run into some more basic issues with Elasticsearch client in another project, more strict.


NAZoom Chat


No chant

Topic Backlog

Decision Log ReviewAll

Review decisions that are in progress.  Can any of them be accepted?  rejected?

Translation SubgroupAllSince we're having trouble finding volunteers for a subgroup, maybe we can make progress during a dedicated discussion session?
Communicating Breaking ChangesAll

Currently there is a PoC, developed by Maccabee Levine, of a utility to catalog Github PRs that have been labeled with the "breaking change" label. We would like to get developer feedback on the feasibility of this label being used more often, and the usefulness of this utility. 

Officially Supported Technologies - UpkeepAll

Previous Notes:

  • A workflow for these pages. When do they transition from one state to another. Do we even need statuses at all ?

Stripes architecture group has some questions about the Poppy release.

Zak: A handshake between developers, dev ops and the TC. Who makes that decision and how do we pass along that knowledge ? E.g. changes in Nodes and in the UI boxes. How to communicate this ? We have a large number of teams, all have to be aware of it.  TC should be alerted that changes are happening. We have a couple of dedicated channels for that. Most dev ops have subscribed to these channels. How can dev ops folk raise issues to the next level of community awareness ? There hasn't been a specific piece of TC to move that along.

Craig: There is a fourth group, "Capacity Planning" or "Release Planning". Slack is the de facto communication channel.  There are no objections to using Slack. An example is the Java 17 RFC. 

Craig: The TC gets it on the agenda and we will discuss it. The TC gets the final say.

Marc Johnson: We shouldn’t use the DevOps Channel. The dev ops folks have made it clear that it should only be used for support requests made to them.

Jakub: Our responsibility is to avoid piling up technical debt.

Marc: Some set of people have to actually make the call. Who lowers the chequered flag ?

Craig: It needs to ultimately come to the TC at least for awareness. There is a missing piece. Capacity Planning needs to provide input here. 

Marc: Stakeholders / Capacity Planning could make that decision. Who makes the decision ? Is it the government or is it some parts of the body ?

Marc: the developers community, the dev ops community and sys ops are involved. For example the Spring Framework discussion or the Java 17 discussion. But it was completely separate to the TC decision. It is a coordination and communication effort.

Marc: Maybe the TC needs to let go that they are the decision makers so that they be a moderating group.

Jakub: I agree with Marc. But we are not a system operating group. Dependency management should be in the responsibility of Release management. There are structures in the project for that.

Jason Root: I agree with Jakub and with Marc also. Policies should drive operational/release/support aspects of Folio.

Jason Root: If the idea of “support” is that frameworks are supported, then of course the project should meet that.

Marc Johnson
Some group needs to inform OleksAii when a relevant policy event occurs.
These documents effectively ARE the manifestation of the policy.

Craig: This is a topic for the next Monday session.

Craig to see if Oleksii Petrenko could join us to discuss the process for updating the officially supported technologies lists.


Dev Documentation VisibilityAll

Possible topic/activity for a Wednesday session:

Discuss/brainstorm:

  • Ideas for the type of developer-facing documentation we think would be most helpful for new developers
  • How we might bring existing documentation up to date and ensure it's consistent 
  • etc.
API linting within our backend modulesAll

https://folio-project.slack.com/archives/CAQ7L02PP/p1713343461518409


Hello team, I would like to discuss API linting within our backend modules. Some time ago, we transitioned our linting process from Jenkins to GitHub Actions as outlined in https://folio-org.atlassian.net/browse/FOLIO-3678. I am assuming that this move was done via some technical council decision. Please correct me if I'm wrong.
In my observations, I've found two problems:
  1. Schema linting does not occur if the schemas are in YAML format.
  2. There are issues with resolving some deeper references during API linting.
Although I'm unsure about how to improve the existing linting implementations within Folio, I propose to consider an open-source solution that handles OpenAPI linting effectively and allows us to define custom rules. For your reference: https://stoplight.io/open-source/spectral A test of this solution can be found in this PR: https://github.com/folio-org/mod-search/pull/567. The same PR also provides an example of custom rule definition: https://github.com/folio-org/mod-search/pull/567/files#diff-d5da7cb43c444434994b76f3b04aa6e702c09e938de09dbc09d72569d611d9ab.Also, by employing 'Spectral', I discovered AsyncAPI (https://www.asyncapi.com/en), an API design tool similar to OpenAPI but for asynchronous interactions. I suggest that we consider using AsyncAPI in FOLIO to generate documentation for Kafka interactions.


PR TemplatesAll

https://folio-project.slack.com/archives/CAQ7L02PP/p1713445649504769

Hello team, Small request to consider.
Regarding pr templates.
  1. From my perspective, pr template is not good idea. Even the biggest open source projects that are contributed by many people don't have any pr template. Currently what we have for acq modules https://github.com/folio-org/mod-orders-storage/blob/master/PULL_REQUEST_TEMPLATE.md
  2. These pr template is inconsistent in different teams.
What I suggest is that, pr template shouldn't be any instructions, because most developer who are creating pr have already understand the rules. If we put just two section into template, it will encourage developers to write more about their work and that lead to knowledge  sharing among developers.
Proposed Mod KafkaAll

https://folio-project.slack.com/archives/CAQ7L02PP/p1714471592534689

Mike Taylor

Proposal. If and only if a FOLIO instance is running Kafka, it should insert and enable a module called mod-kafka, which consists entirely of a module descriptor that says it provides the interface kafka. The purpose is so that other modules can use the standard <IfInterface> and similar tools to determine whether they should attempt Kafka operations. Rationale: the FOLIO ILS depends absolutely on Kafka, but other uses of the platform will not. One such example: a dev platform that includes only mod-users, used as a source of change events for Metadb.

Action Items