/
Managing Tools/Frameworks/Dependencies Version - v0.2 DRAFT

Managing Tools/Frameworks/Dependencies Version - v0.2 DRAFT

Feb 08, 2023

What is in scope?

Terminology

  • FOLIO distribution - All deployable artifacts produced by the FOLIO community that are included in a flower releases, e.g. jar files, docker images, JS bundles, vagrant boxes

  • Infrastructure Components - Components that are required to build and run FOLIO but are not packaged with the FOLIO distribution. E.g. Kafka, Postgres

  • Non Infrastructure Components - Components that are a part of FOLIO distribution

  • LTS - Long Term Support

  • Component - Refers to one of the Officially Supported Technologies

  • Component Owner - Could be a team, an individual or a TC member owning one of the Officially Supported Technologies

  • Upgrade -Covers all major versions (For e.g 1.n.n to 2.n.n or any change that has significant impact on support, security)

What is out of scope?

  • Process for Introducing new components to the platform. DR/RFC process is in place for such cases.

  • Deciding what makes is part of Officially Supported Technologies and what is not

  • Resource planning for upgrades

Objectives

  • Ensure third-party component usage is consistent across teams

  • Ensure that the FOLIO platform consistently uses component versions that are supported until the FOLIO flower release support period ends

  • Ensure that fixes to known CVEs are incorporated in a timely manner

  • Ensure that the development/operational aspect of making the upgrade is well planned

Assumptions

  • TC can delegate any of the work related to upgrades to the security group or a sub-group as it sees fit.

  • If the decision to upgrade has significant implications, document the decision as a DR.

  • While it is good to have some time-bound expectations, it is not very clear that it is something that can be enforced practically. That said, this process is expected to work based on an honor system. Teams should try and adopt these guidelines and realize that this will result in the community building a robust platform

Process Overview

Roles and Responsibilities

Component Owner

  • Proactively monitors developments related to the assigned component

  • Works with the release coordinator to understand the release milestones

  • Proactively informs TC/Development teams about ending of LTS

  • Submits a proposal for upgrading/deprecating a component for the TC to review

Development Teams

  • Documents (could be a gating factor for each flower release) current state of Officially Supported Technologies once a year. 

  • Uses the template below to document their usage of Officially Supported Technologies

  • Information collected is expected to be submitted to the TC

  • If the data reveals a certain module/app is using unsupported versions, the team owning the module/app is expected to address the issue within 3-6 months

Common

Infrastructure

Version

Infrastructure

Version

Kafka

 

OpenSearch

 

Postgres

 

Maven

 

Frontend Apps

App Name

React

Yarn

npm

App Name

React

Yarn

npm

Stripes Platform

 

 

 

Backend Modules

Module Name

Java

React

Yarn

npm

Junit

Spring Boot

Vertx

Module Name

Java

React

Yarn

npm

Junit

Spring Boot

Vertx

okapi

 

 

 

 

 

 

 

mod-inventory

 

 

 

 

 

 

 

Technical Council

Upgrade Scenarios

Component Owner-driven 

  • Applies to the situation where the component owner recommends an upgrade to keep up with changing versions 

  • Component owner monitors the release of new LTS version as they become available and recommends teams to schedule an upgrade within 3-6 months of the LTS release

  • Component owner establishes a general upgrade plan and submits to the TC for review

  • Development teams will have the flexibility to alter the plan within reason

Team-driven 

  • Applies to the situation where the team is in need of a particular version to take advantage of some new feature, performance enhancements, or a bug fix

  • The team will collaborate with the component owner to prepare a proposal (includes the upgrade plan) and present it to the TC for review

  • Other development teams will have the flexibility to alter the plan and timeline within reason

Security-driven 

  • Applies to the scenario where libraries needed to be upgraded to address a major critical security vulnerability

  • TC/Security Group will work with all affected teams to establish a timeline for rolling out the fix to the CVE

  • Security driven update could override other development priorities based on the criticality of the security issue

Upgrading Infrastructure vs. Non Infrastructure Components

Infrastructure Components 

  • Given the constraints the community has on testing the platform on multiple versions of supported infrastructure components, support will be limited to just one version of the infrastructure component, at any given time.

  • Service providers can proceed to use unsupported versions at their own risk

  • Intent for upgrading an infrastructure component MUST be submitted (by the component owner) as a proposal (MUST include justification) to the TC at least one release cycle before the intended upgrade

  • TC deliberates the proposal with all relevant stakeholders to arrive at a consensus. The proposal can ONLY move forward when consensus is reached among the stakeholders (e.g.hosting providers, implementors, systems operators, DevOps)

  • In the event the proposal to upgrade moves forward, plans should be put in place to execute it in the next release cycle. Teams should consider making this upgrade very early in the release cycle to give teams enough time to execute all available regression test suites

Note :

In the case when discussions related to the upgrade is not resulting in a consensus, steps should be taken to resolve the situation as soon as possible. Everyone should realize that protracted discussions around upgrades will certainly hold back the community in terms of technological advancements to the platform  

Non Infrastructure Components 

  • Intent for upgrading an non infrastructure component MUST be submitted (by the component owner or the teams submitting via the component owner) as a proposal (MUST include justification) to the TC

  • Teams will decide their own timeline for upgrading non infrastructure components.  However, teams SHOULD not vary from the supported versions for major components.

  • Timeline for deprecating usage of older versions should definitely be considered while making plans for the upgrade

  • In the event of any upgrade affecting build/infrastructure pipeline, the DevOps (as they might the first to see) team will communicate the impact to the developer community. The development team can also proactively communicate the potential changes to the build infrastructure to allow the DevOps team to plan accordingly.

Upgrade Plan

An upgrade plan, at a minimum should include

  • Upgrade Instructions

  • Breaking Changes/ Known Issues/Incompatibilities/Risks

  • Other Components, FOLIO modules affected

  • Rollback Instructions

  • Regression & Performance Test Plans

Next Steps

Questions?

  • Component Owner Assignment

    • TC took a stab at assigning owners. 

  • How is this whole process triggered?

    • Covered by the 3 documented scenarios

  • What happens when the component owner does not agree to the proposal to upgrade ? 

  • How to extract information about outdated components ? Manual vs. Automatic. https://github.com/folio-org/platform-complete/actions/workflows/spring-cve-2022-22965.yml

  • Who are the stakeholders for the upgrade plan and how do we communicate it?

    • Should be part of the upgrade plan 

  • Who will work on the resource plan when the upgrade initiative is TC driven?

  • Do we have reserved capacity to address tech debt items in each release?

    • No

  • How tight the constraints needs to be? TC has to be involved more. Is it worth ?

    • TC to get involved ONLY for major infrastructure components