2024-02-21 - Communicating Breaking Changes

Ankita Sen is next, followed by Taras Spashchenko.  It might make sense for Taras to take this on today since I think Ankita might be helping drive the conversation today.

Craig McNally will take notes since Taras can't make it today.

Reminder:  Please copy/paste the Zoom chat into the notes.  If you miss it, this is saved along with the meeting recording, but having it here has benefits. 

Background/Context:  Since we're having trouble finding volunteers for a subgroup, the thought was to try and make progress during a dedicated discussion session...

From the Subgroups page:

• Defining the mechanisms for communicating breaking changes when they occur.
• Educating/communicating to developers what constitutes a breaking change (beyond sharing the DR)
• Meet with representatives from dev teams to review what a breaking change is, and what to do when they happen. (or video, etc.) 

See also:  DR-000036 - Breaking Changes

Goal:  Define the mechanisms for communicating breaking changes when they occur

Notes:

Two parts to this:

• How do we communicate what the TC considers breaking changes out to dev teams?
  • This is covered by the RFC:  https://github.com/folio-org/rfcs/blob/master/text/0003-folio-breaking-changes.md
• How do dev teams raise awareness when they make breaking changes?
  • When / how frequently do we need to communicate these changes?  At release time?  when the change is actually made?  etc.
    • Some ideas discussed include:  release notes, wiki pages, slack messages, etc.
    • Probably needs to be communicated sooner than release time so teams have time to react and make corresponding changes
    • This should be considered in the context of application formalization and more frequent application releases.
    • It may be difficult to get developers to participate in processes which are separate from the flower release cycle
    • At one point the SOP was for teams making breaking changes to also create corresponding PRs in other team's repos when this happens.
      • This may be easier in some cases than others
      • I don't think this is followed very closely these days.
    • Does tying this to the module release process make sense?
      • It's discouraged for teams to arbitrarily make module releases outside of the flower release cycle, but there are explicit policies in place ATM
        • It's unclear why this is the case.
    • At what point in the release cycle do we need to get these things surfaced?
      • We need to give folks sufficient time to make changes/adaptations.
    • Presently, we often don't find these issue until a hosted reference env build fails.
    • Suggestion:  Communicate breaking changes when the PR is created?  merged?  approved
      • At TAMU, when an issue is created, the expectation is set as to whether it will result in a patch release, minor release, major release, etc.
        • Some teams (used to?) do this, it's not consistently done across the project ATM.
    • The audience needs to be considered when communicating breaking changes...
      • JIRAs/PRs may be acceptable for dev teams, but probably not for system operators/hosting providers/implementers/etc.
      • Maybe different audiences have different requirements for timing and level of detail for these things.
        • Release notes may be acceptable for some audiences (SysOps/ICs/etc.), but that's way too late for other dev teams.
    • Some teams already leverage PR labels for indicating breaking changes:  Examples:
      • https://github.com/folio-org/mod-circulation/pulls?q=is%3Apr+is%3Aclosed+label%3Abreaking
      • https://github.com/folio-org/mod-inventory-storage/pulls?q=is%3Apr+is%3Aclosed+label%3Abreaking
      • Suggestion: Maybe automation could be leveraged to send notifications based on the presence of this label?
        • Noting that there are currently capacity issues amongst the project's devops members
      • This doesn't necessarily require changes to existing build/CI pipelines, it could be something independent of those things.
        • Action: Maccabee Levine would be onboard with doing a quick PoC of this.
      • Maybe there are other things we standardize for PRs - e.g. a description of the breaking change in the PR description.
    • How are breaking changes communicated?
      • A slack message to a certain list of channels (TBD), linking to the automated report.
      • As a daily digest?  Immediately?
    • The timing of when breaking changes are made varies greatly from team to team.  Some do it "as they go" throughout the release cycle.  Others make the breaking changes but don't bump the interface version until the end of the release cycle.
      • Standardizing on this and branching strategies would likely be very helpful, but also probably not an easy task.
  • What is the reason for putting these processes/policies in place?  
    • In the past we've seen cases where breaking changes were made when many feel they shouldn't have (e.g. to fix a potential spelling mistake)
    • It isn't necessarily the developers making the decision... Could be POs, or other stakeholders (member orgs / hosting providers / security team / etc.)
  • We should consider discussing the topic of getting feedback prior to making breaking changes separately from this discussion

Still needing to be discussed:

• The suggestions so far are focused on dev-facing comms.  What about the SysOps/ICs/etc.
• We should consider discussing the topic of getting feedback prior to making breaking changes separately from this discussion

Previous Notes:

• A workflow for these pages. When do they transition from one state to another. Do we even need statuses at all ?
• Stripes architecture group has some questions about the Poppy release.
• Zak: A handshake between developers, dev ops and the TC. Who makes that decision and how do we pass along that knowledge ? E.g. changes in Nodes and in the UI boxes. How to communicate this ? We have a large number of teams, all have to be aware of it.  TC should be alerted that changes are happening. We have a couple of dedicated channels for that. Most dev ops have subscribed to these channels. How can dev ops folk raise issues to the next level of community awareness ? There hasn't been a specific piece of TC to move that along.
• Craig: There is a fourth group, "Capacity Planning" or "Release Planning". Slack is the de facto communication channel.  There are no objections to using Slack. An example is the Java 17 RFC. 
• Craig: The TC gets it on the agenda and we will discuss it. The TC gets the final say.
• Marc Johnson: We shouldn’t use the DevOps Channel. The dev ops folks have made it clear that it should only be used for support requests made to them.
• Jakub: Our responsibility is to avoid piling up technical debt.
• Marc: Some set of people have to actually make the call. Who lowers the chequered flag ?
• Craig: It needs to ultimately come to the TC at least for awareness. There is a missing piece. Capacity Planning needs to provide input here. 
• Marc: Stakeholders / Capacity Planning could make that decision. Who makes the decision ? Is it the government or is it some parts of the body ?
• Marc: the developers community, the dev ops community and sys ops are involved. For example the Spring Framework discussion or the Java 17 discussion. But it was completely separate to the TC decision. It is a coordination and communication effort.
• Marc: Maybe the TC needs to let go that they are the decision makers so that they be a moderating group.
• Jakub: I agree with Marc. But we are not a system operating group. Dependency management should be in the responsibility of Release management. There are structures in the project for that.
• Jason Root: I agree with Jakub and with Marc also. Policies should drive operational/release/support aspects of Folio.
• Jason Root: If the idea of “support” is that frameworks are supported, then of course the project should meet that.
• Marc Johnson
  Some group needs to inform OleksAii when a relevant policy event occurs.
  These documents effectively ARE the manifestation of the policy.
• Craig: This is a topic for the next Monday session.

• Craig to see if Oleksii Petrenko could join us to discuss the process for updating the officially supported technologies lists.

• Do common libraries used to build in approved frameworks need to be on this list? Such as spring-way and spring-module-core.

Possible topic/activity for a Wednesday session:

• Discuss/brainstorm:
  • Ideas for the type of developer-facing documentation we think would be most helpful for new developers
  • How we might bring existing documentation up to date and ensure it's consistent 
  • etc.