Skip to end of banner
Go to start of banner

2024-10-14 Meeting notes

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Current »

Date

Attendees 

Discussion items

TimeItemWhoNotes
1 minScribeAll

Maccabee Levine is next, followed by Craig McNally

Reminder:  Please copy/paste the Zoom chat into the notes.  If you miss it, this is saved along with the meeting recording, but having it here has benefits.

5-10 minLiaison Updates
  • CC: Maccabee Levine
    • Developer Advocate: CC wants to continue position, but Patrick Pace has decided not to continue for personal reasons. PP will write up refections, CC will look to fill the position.
    • Next Tricouncil meeting 2025-01-13 10am ET
    • AWS: CC decided dev teams to fund their own environments, currently looks like AWS costs are under target of $30K, but not clear how will be affected by any decisions around Eureka. Expect a statement from CC about this.
    • Suggestion about TC role in advocating for developers, and potential for self-hosting institutions to host dev environments at lower cost then AWS.
  • PC: Marc Johnson (covering for Tod Olson)
    • Looking at how to standardize prioritization across SIGS and teams, and bring that back to PC
  • RMS Group:
    • Jakub not here
  • Security Team: 
    • Business as usual (triaging automated vulnerability reports), nothing relevant to report
  • Tri-council Application Formalization:  
    • Want community affirmation of Eureka at next Tri-council meeting, working out what to bring to that meeting for the affirmation to happen.
    • Want to sunset current group (working for > 1 year) and hand off to whatever next group is needed.
1 minUpcoming MeetingsAll
  • - Regular TC meeting - Include Developer Advocate Update
  • - Dedicated discussion: Sunflower OST moving to Accepted.  Issue is Jenkins java version.
  • -  Regular TC meeting
  • - Dedicated discussion: ?
  •   - Regular TC meeting
  • - Dedicated discussion: TCR 44
15 minTCR Board ReviewAll

Today:

TCR-43 mod-marc-migrations

Some failures

  • API criteria failed in self-eval, passing that forward. This is a very soft requirement, Jeremy Huff has found this to be a stumbling block in every review. Craig McNally thinks trend is towards Cypress end-to-end testing and away from Karate API tests. This seems to be managed by Yogesh and QA team and not by the FOLIO governance.
    • Discussion of whether to remove integration testing from criteria; if there are tests wish to use consistent technology to reduce load on dev ops and CI/CD process.
    • Future discussion: whether to remove API test criteria
  • Schema access:
    • design uses views into other schemata, seems to violate spirit of the constraint.
  • Dependencies other than in module descriptor declared in the README.md:
    • The schema references seem to be a dependency, though not an API dependency.
    • Assertion that schema refs not covered by this requirement, TC deferred that decision.
    • Still, there are hard dependencies between this module and other modules that are not expressed in any way. This is a serious consideration for operations staff trying to run the system.
    • Do not want to mix two things together: (1) whether the dependencies are documented, and (2) whether direct schema access is an acceptable integration technique.
    • Decision accepting mod-fqm-manager: https://folio-org.atlassian.net/browse/TCR-29?focusedCommentId=88302
    • Original discussion: https://folio-org.atlassian.net/wiki/spaces/TC/pages/5057321/2023-09-11+-+mod-fqm-manager
    • Approval (conditional) of mod-fqm-manager in Sept. 2023 was predicated on an RFC which has not been submitted.
    • VBar suggests this is different from FQM in that this is very closely aligned to SRS and has a legitimate interest in access to access the db schema; could be an example of a place to bind modules together in an application and consider the application to be the microservice boundary. There are legitimate questions about applicability.
    • This approach would be dependent upon applications and Eureka, though RFCs for these have not been submitted or accepted.
    • Exception was made for FQM as a product imperative and was a limited exception.
  • To be continued: due to time constraints will continue this discussion Wednesday


____


TCR-43: mod-marc-migrations

TCR-45 mod-record-specifications

  • Ingolf Kuss Finished the review, all criteria fulfilled except snapshot version dependence.  Dev lead said they will change that dependency before flower release.
  • There is a pull request #76 to change that TCR criteria.   
    • Marc Johnson We currently have a disagreement on whether/what to change in the criteria.  Ingolf recommends we approve the TCR regardless.  Marc recalls TC preferring that we decide on the PR first.
      • Maccabee Levine Agree we should decide the PR before the TCR.  Started the conversation several times, there may be disagreement, but need to make a decision.
    • Julian Ladisch disagrees with softening the criterion, prefers to remove it entirely.
    • Marc Johnson it's there because it was originally desired.
    • Jenn Colt Criteria that requires the reviewer to make a subjective judgement are difficult, would rather remove the criterion.
    • Marc Johnson Given that using snapshot dependencies is common practice, does anyone feel strongly enough to oppose ditching the criteria entirely?
      • No objections raised.
    • Change accepted without disagreement.  Ingolf Kuss merged the PR and will change the other related file as well.
  • Back to TCR-45.  Ingolf recommends acceptance.  No objections.
    • Approved by lazy consensus.

TCR-44

  • Craig McNally First part is deciding about Go lang.  Second is approval of mod-reporting.
  • Maccabee Levine Jakub suggested "I think there should be a presentation to the TC from DevOps regarding issues (or lack thereof) related to supporting Go in CI/CD and FOLIO deployments.", and then ok to to the TCR in parallel.
    • No disagreement on process.
    • Marc Johnson Should review this decision next time we review processes.
  • Julian Ladisch will do the TCR review of TCR-44.
  • Need presentation from DevOps.
    • Jakub Skoczen Maybe we should structure discussion about the criteria.
    • Craig McNally Agree.  Ask Julian to go through criteria, identify which items require discussion.
    • Marc Johnson Agree but suggest we say beforehand things that we know will fail.  Some gaps already known.
  • Jenn Colt Let Julian get started on this, then note in Slack when we want to use a Wednesday discussion to talk about it.  Or schedule now.
    • Craig McNally We should schedule now, within timebox for initial review.  Wed 10/23 for review.

Last time:

mod-reporting, a Go rewrite of mod-ldp, has been submitted. As Go is not yet supported it requires work regarding static code checks, code coverage automation, etc.

We should first complete the existing TCR requests.


Should we merge the PR that removes the snapshot criterium from our module acceptance criterium? This will solve the issue with the TCR.

Depending on a SNAPSHOT version is bad practice. Maven rejects a release when trying to release a module with SNAPSHOT dependency. It is also bad practice during development because the build is not reproducible. However, as FOLIO libraries are provided as SNAPSHOT version only the modules are forced to use SNAPSHOT versions during development.

When merging the PR we can appove the submitted module without explaining why we overwrite the SNAPSHOT criterium.

Instead of removing the SNAPSHOT criterium it should be a requirement that can be ignored if reasonably justified.

A warning not to use a SNAPSHOT dependency is in the module release documentation for maven based modules: https://dev.folio.org/guidelines/release-procedures/#prepare-and-perform-the-source-release

We don't know how non-maven modules prevent SNAPSHOT dependencies.

It's up to the development teams how to prevent SNAPSHOT dependencies.


Go modul mod-reporting:

When accepted as a module we indirectly adopt Go as a language.

Should we first accept Go as a language including specific criteria regarding static code analysis, security, etc.?

In the past PoCs have been used to add new stuff to FOLIO without evaluation/approval.

We had been discussing the Go RfC for months.

Regarding adding the Go stack we need more feedback from the community.


5 min

Technical Council Sub-Groups Updates



1 minRFCs

All

Reminder(s)

  • Review of open items, such as the need for decision log records and the application of the "Go" RFC.
    • Still need merge and decision log entry
1 minDecision LogAll

Need to log decisions for the following:  (see above)

15 min

Officially Supported Technologies (OST)

All

Last week:

Check Recurring Calendar...

We still need to transition the Sunflower page from DRAFT → ACCEPTED... ASAP.  What is preventing this from happening? 

  • Jenn Colt Delay is related to the devops discussion about Java.  Checked with Mike G. just before the meeting about whether it will cause an issue for Jenkins.
  • Marc Johnson For some period of time we will need Jenkins agents on both versions of java, running concurrently.
  • Jenn Colt Mike's question was about compatibility.  Whether a Java 21 build image will be compatible with Jenkins (on Java 8).
    • Marc Johnson Thought Jenkins was on Java 11.  But how long do we wait?
    • Jenn Colt OST discussion this Wednesday?
    • Sunflower release scope composition deadline was September 27, so that's our deadline.  Jenn Colt will let Mike know that.

This week:

  • Java version:
    • Dev Ops has implemented tooling to declare either Java 17 or Java 21
    • Question: how much time to we allow for dev teams to upgrade to Java 21.  
    • Lift for this upgrade seems to be very small, removing Java 17 and force teams to upgrade seems appropriate.
    • Future topics: When do we remove Java 17, how do we communicate this out?
    • Decision: Accepted move to Java 21 in Sunflower.
NAZoom Chat


10:06:24 From Marc Johnson to Everyone:
https://folio-project.slack.com/archives/C04DV88K1GS/p1728611251622039
10:11:40 From Craig McNally to Everyone:
Sorry guys! I've been held up in meetings all morning and just couldn't get away.
10:12:00 From Jenn Colt to Everyone:
All good
10:18:13 From Maccabee Levine to Everyone:
Yes, EBSCO and TAMU are the two institutions affected, per CC minutes.
10:18:19 From Maccabee Levine to Everyone:
Replying to "Yes, EBSCO and TAMU ..."

We will implement the above principles by following up with the funders of development teams that currently have FOLIO community supported dev environments (EBSCO and TAMU) to implement a transition.
10:29:43 From Jenn Colt to Everyone:
brb
10:30:31 From Julian Ladisch to Everyone:
mod-marc-migrations has an integration test using JUnit: https://github.com/folio-org/mod-marc-migrations/blob/master/src/test/java/org/folio/marc/migrations/MarcMigrationsApplicationIT.java
10:32:05 From Jenn Colt to Everyone:
back
10:32:49 From Marc Johnson to Everyone:
These tests are a mono-repo for the entire product

That’s why the tooling needed to be consistent
10:34:07 From Marc Johnson to Everyone:
Replying to "mod-marc-migrations …"
Those aren’t the style of tests this criteria refer to

This criteria refers to the external to the module, API integration tests written in Karate
10:35:16 From Ingolf Kuss to Everyone:
a mono-repo ?
10:35:57 From Marc Johnson to Everyone:
Replying to "a mono-repo ?"
A single repository that has tests for all modules
10:36:14 From Ingolf Kuss to Everyone:
Reacted to "A single repository ..." with 👌
10:40:05 From Maccabee Levine to Everyone:
Decision accepting mod-fqm-manager: https://folio-org.atlassian.net/browse/TCR-29?focusedCommentId=88302 Original discussion: https://folio-org.atlassian.net/wiki/spaces/TC/pages/5057321/2023-09-11+-+mod-fqm-manager
10:52:21 From Maccabee Levine to Everyone:
Noting for Wednesday, the "conditional" approval we did of mod-fqm-manager was ad-hoc, and we followed it up with procedures on how we could handle this in the future. https://github.com/folio-org/tech-council/blob/master/NEW_MODULE_TECH_EVAL.MD#review
10:52:42 From Maccabee Levine to Everyone:
Replying to "Noting for Wednesday..."

The key part (IMHO) is: "If some of the failures are due to proposed architectural (or other cross-module) changes, the TC may request that Submitter first propose those changes via the RFC process to get sufficient community input. In that situation the TC may defer its decision pending the resolution of the RFC. (See Before Development.)"
10:54:09 From Huff, Jeremy T to Everyone:
I have to run folks, thanks!

Topic Backlog

Decision Log ReviewAll

Review decisions that are in progress.  Can any of them be accepted?  rejected?

Translation SubgroupAllSince we're having trouble finding volunteers for a subgroup, maybe we can make progress during a dedicated discussion session?
Communicating Breaking ChangesAll

Currently there is a PoC, developed by Maccabee Levine, of a utility to catalog Github PRs that have been labeled with the "breaking change" label. We would like to get developer feedback on the feasibility of this label being used more often, and the usefulness of this utility. 

Officially Supported Technologies - UpkeepAll

Previous Notes:

  • A workflow for these pages. When do they transition from one state to another. Do we even need statuses at all ?

Stripes architecture group has some questions about the Poppy release.

Zak: A handshake between developers, dev ops and the TC. Who makes that decision and how do we pass along that knowledge ? E.g. changes in Nodes and in the UI boxes. How to communicate this ? We have a large number of teams, all have to be aware of it.  TC should be alerted that changes are happening. We have a couple of dedicated channels for that. Most dev ops have subscribed to these channels. How can dev ops folk raise issues to the next level of community awareness ? There hasn't been a specific piece of TC to move that along.

Craig: There is a fourth group, "Capacity Planning" or "Release Planning". Slack is the de facto communication channel.  There are no objections to using Slack. An example is the Java 17 RFC. 

Craig: The TC gets it on the agenda and we will discuss it. The TC gets the final say.

Marc Johnson: We shouldn’t use the DevOps Channel. The dev ops folks have made it clear that it should only be used for support requests made to them.

Jakub: Our responsibility is to avoid piling up technical debt.

Marc: Some set of people have to actually make the call. Who lowers the chequered flag ?

Craig: It needs to ultimately come to the TC at least for awareness. There is a missing piece. Capacity Planning needs to provide input here. 

Marc: Stakeholders / Capacity Planning could make that decision. Who makes the decision ? Is it the government or is it some parts of the body ?

Marc: the developers community, the dev ops community and sys ops are involved. For example the Spring Framework discussion or the Java 17 discussion. But it was completely separate to the TC decision. It is a coordination and communication effort.

Marc: Maybe the TC needs to let go that they are the decision makers so that they be a moderating group.

Jakub: I agree with Marc. But we are not a system operating group. Dependency management should be in the responsibility of Release management. There are structures in the project for that.

Jason Root: I agree with Jakub and with Marc also. Policies should drive operational/release/support aspects of Folio.

Jason Root: If the idea of “support” is that frameworks are supported, then of course the project should meet that.

Marc Johnson
Some group needs to inform OleksAii when a relevant policy event occurs.
These documents effectively ARE the manifestation of the policy.

Craig: This is a topic for the next Monday session.

Craig to see if Oleksii Petrenko could join us to discuss the process for updating the officially supported technologies lists.


Dev Documentation VisibilityAll

Possible topic/activity for a Wednesday session:

Discuss/brainstorm:

  • Ideas for the type of developer-facing documentation we think would be most helpful for new developers
  • How we might bring existing documentation up to date and ensure it's consistent 
  • etc.
API linting within our backend modulesAll

https://folio-project.slack.com/archives/CAQ7L02PP/p1713343461518409


Hello team, I would like to discuss API linting within our backend modules. Some time ago, we transitioned our linting process from Jenkins to GitHub Actions as outlined in https://folio-org.atlassian.net/browse/FOLIO-3678. I am assuming that this move was done via some technical council decision. Please correct me if I'm wrong.
In my observations, I've found two problems:
  1. Schema linting does not occur if the schemas are in YAML format.
  2. There are issues with resolving some deeper references during API linting.
Although I'm unsure about how to improve the existing linting implementations within Folio, I propose to consider an open-source solution that handles OpenAPI linting effectively and allows us to define custom rules. For your reference: https://stoplight.io/open-source/spectral A test of this solution can be found in this PR: https://github.com/folio-org/mod-search/pull/567. The same PR also provides an example of custom rule definition: https://github.com/folio-org/mod-search/pull/567/files#diff-d5da7cb43c444434994b76f3b04aa6e702c09e938de09dbc09d72569d611d9ab.Also, by employing 'Spectral', I discovered AsyncAPI (https://www.asyncapi.com/en), an API design tool similar to OpenAPI but for asynchronous interactions. I suggest that we consider using AsyncAPI in FOLIO to generate documentation for Kafka interactions.


PR TemplatesAll

https://folio-project.slack.com/archives/CAQ7L02PP/p1713445649504769

Hello team, Small request to consider.
Regarding pr templates.
  1. From my perspective, pr template is not good idea. Even the biggest open source projects that are contributed by many people don't have any pr template. Currently what we have for acq modules https://github.com/folio-org/mod-orders-storage/blob/master/PULL_REQUEST_TEMPLATE.md
  2. These pr template is inconsistent in different teams.
What I suggest is that, pr template shouldn't be any instructions, because most developer who are creating pr have already understand the rules. If we put just two section into template, it will encourage developers to write more about their work and that lead to knowledge  sharing among developers.
Proposed Mod KafkaAll

https://folio-project.slack.com/archives/CAQ7L02PP/p1714471592534689

Mike Taylor

Proposal. If and only if a FOLIO instance is running Kafka, it should insert and enable a module called mod-kafka, which consists entirely of a module descriptor that says it provides the interface kafka. The purpose is so that other modules can use the standard <IfInterface> and similar tools to determine whether they should attempt Kafka operations. Rationale: the FOLIO ILS depends absolutely on Kafka, but other uses of the platform will not. One such example: a dev platform that includes only mod-users, used as a source of change events for Metadb.

  • No labels