Kiwi (R3 2021) PO Bugfix and Hotfix Release Process:

Owned by Ann-Marie Breaux (Deactivated)
Last updated: Feb 10, 2022
6 min read

Bugfix = a fix release that happens after the module release deadline, but before the actual release

Hotfix = a fix release that happens after the final flower release, and heads straight to production

NOTE: We are currently in the R3 2021 Kiwi Hotfix period. Hotfix instructions are here; Bugfix instructions are below these.

(Prior Bugfix/Hotfix procedures are saved with the appropriate release.)

This describes the R3 2021 Kiwi Hotfix release process that Product Owners should follow. This may be adjusted for future quarterly releases.

  1. Hotfix release = one that must be added to a release after the final release has been announced. Hotfixes occur after bugfixes.
  2. Generally, hotfix releases are not made except for P1 functional, P2 security, and Implementer Showstopper issues.  Once a hotfix release has been scheduled, lower priority issues (e.g. P2 and P3 functional issues) may also be included with permission.
  3. For R3 2021 hotfixes, the PO should ensure the following are adjusted in the bug Jira:
    1. Make sure the priority is aligned with the definitions here: /wiki/spaces/DQA/pages/2657909
    2. Set the Release field of the bug Jira issues to Kiwi R3 2021 Hot Fix<Number> and make sure the developers doing the fix know that it is needed for a hotfix
    3. Fill in the RCA (root cause analysis) field of the Jira
    4. Include a comment with the TestRail test case(s) that were executed during Bugfest that should have caught the problem, but didn't. If there was no TestRail previously, create it and link to the current hotfix bug Jira.
  4. Get the issue fixed and test on https://folio-snapshot.dev.folio.org/
  5. Any hot fix issue must be approved by the Capacity Planning Team:
    1. Once the fix has passed testing in snapshot, post the Jira on the Slack #release_bug_triage channel and request approval for the fix. POs may also be asked to present the hotfix Jiras at the next available Capacity Planning meeting, usually held on Mondays at 9 am US ET. Hotfix Approvers are Khalilah Gambrell, Mike Gorrell, Harry Kaplanian, Jakub Skoczen and Mark Veksler.  Please tag the approvers with your request (can be considered approved if you get the thumbs up from at least two technical approvers (Mike, Mark or Jakub).  
  6. Once approved, on the issue's Jira, either a member of the Capacity Planning Team or the PO should set the field for Hot Fix Approved by Cap Planning? to Yes. Also include details about when the hot fix was approved in the Hot Fix Approval Comments field.
  7. If a hot fix is not approved, change the Release field to R1 2022 or later, and add comments either in the Hot Fix Approval Comments or in a regular Comment on the Jira.
  8. If approved, ask the module maintainer to create a hotfix release and set the JIRA status to Awaiting release, and Resolution to Done.
    1. You can get the module maintainer’s name from the Team Module Responsibility Matrix.
    2. The release needs to go to the appropriate Release branch and the Main branch. 
    3. The release Jira (not the bug Jira) should be linked to the appropriate R3 2021 Hotfix epic, currently FOLREL-495
    4. When the release is made, the module maintainer will announce it on the Slack #releases channel. 
    1. The module maintainer will change the bug issue's status to Awaiting deployment. There is no need for the PO or module maintainer to request that the release be deployed. 
    2. Jenkins robot automatically generates pull request for the "R3 2021" branch of Platform Complete repository in Github for every released module if version incremented by "patch" number. (For example from 1.1.1 to 1.1.2)
    3. Manual intervention required if release version requires change of major ( 1.1.1 to  2.0.0) or minor (1.1.1 to 1.2.0) number. In this case PO or Dev lead must notify DevOps and Anton Emelianov (Deactivated) about this change.
    4. FOLIO hosting team checks for updates of Platform Complete one time per day and updates Bug Fest system with modules that have been released since yesterday's update.
    5. FOLIO hosting team will post notification in #bug-fest Slack channel when update starts and ends. 
    6. Once daily update has been deployed to Bug Fest, Anton Emelianov (Deactivated) will change the bug issue's status to In bugfix review which is the PO's trigger to do the final round of testing in the Bug Fest environment
    7. Finally, when the issue has passed test in the Bug Fest environment, the PO should change the issue's status to Closed. 
  9. Add information about the hotfix to the R3 2021 Kiwi Release Notes page on the wiki, in the Post-release Hotfixes table.
  10. Once the hotfix has passed testing on BugFest, it needs to be deployed to libraries using the Kiwi release in production or in sandbox.
    1. For EBSCO-hosted libraries, ask an EBSCO PO or Anton to deploy the fix.
    2. For Index Data-hosted libraries, Wayne Schneider is managing the deployments. 
    3. Self-hosted libraries will need to decide whether to implement the hotfix or not, based on the announcement on the Slack releases channel or the release notes.
  11. If hotfixes are deployed by hosting providers, they should notify the libraries using the current release in production or sandbox that a hotfix has been deployed

This describes the R3 2021 Kiwi Bugfix release process (fixes being released after the initial release, but before the final Kiwi deadline). This will be adjusted for future releases

  1. Bugfix = fix that happens after the module release deadline, but before the actual release. Please refer to the Kiwi release schedule Most of these issues will be identified during R3 2021 Kiwi BugFest
  2. Set Release field of the bug Jira issues to R3 2021 Bug Fix and let the developers doing the fix know that it is needed ASAP for a bugfix, definitely before the Kiwi release. Do not confuse with the Fix version field, which should reflect the release version the fix will be included in.
  3. Label the bug Jira issues with bugfest_r3.2021 if the bug is found by a community tester during BugFest. This allows us to keep statistics on the bugs produced during that process. Don't forget to link Jira issue to TestRail test case.
  4. In the Jira Release field, select R3 2021 Bugfix
  5. Select regression or other options in the RCA Group field when applicable; also add label regression
  6. Get the issue fixed and test on https://folio-snapshot.dev.folio.org/ (if possible - some performance issues that don't repro on snapshot will have to skip snapshot testing and be tested on the Bug Fest environment, instead)
  7. No permission is required for bugfixes before the release deadline. Once the fix has passed testing in snapshot, change the status to Awaiting release and the Resolution to Done, and ask the module maintainer to create a bugfix release.
    1. You can get the module maintainer’s name from the Team Module Responsibility Matrix.
    2. The release needs to go to the appropriate Release branch and the Main branch. 
    3. When the release is made, the module maintainer will announce it on the Slack #releases channel. 
  8. Create a release ticket for the module and assign R3 2021 Bugfix Release Epic. Be sure that each bug in the release has the correct release version assigned to it.
  9. When the release has been created
  10. The module maintainer will change the bug issue's status to Awaiting deployment. There is no need for the PO or module maintainer to request that the release be deployed
  11. Jenkins robot automatically generate pull request for the "R3 2021" branch of Platform Complete repository in Github for every released module if version incremented by "patch" number. (For example from 1.1.1 to 1.1.2)
  12. Manual intervention required if release version requires change of major (1.1.1 to  2.0.0) or minor (1.1.1 to 1.2.0) number. In this case PO or Dev lead  have to notify DevOps and Anton Emelianov (Deactivated) about this change.
  13. FOLIO hosting team checks for updates of Platform Complete one time per day and updates Kiwi Bug Fest system with modules that have been released since yesterday's update.
  14. FOLIO hosting team will post notification in #bug-fest Slack channel when update starts and ends. 
  15. Once daily update has been deployed to Bug Fest, Anton Emelianov (Deactivated) will change the bug issue's status to In bugfix review which is the PO's trigger to do the final round of testing in the Bug Fest environment
  16. Finally, when the issue has passed test in the Bug Fest environment, the PO should change the Status to Closed