Nolana (R3 2022) PO Bugfix and Hotfix Release Process:

Owned by Anton Emelianov (Deactivated)
Oct 19, 2022
6 min read

Bugfix = a fix release that happens after the module release deadline, but before the actual release

Hotfix = a fix release that happens after the final flower release, and heads straight to production

NOTE: We are in the Morning Glory R2 2022 Bugfix period, which precedes the Hotfix period. Bugfix instructions are here; Hotfix instructions are below these.

(Prior Bugfix/Hotfix procedures are saved with the appropriate release.)

This describes the Morning Glory R2 2022 Bugfix release process (fixes being released after the initial release, but before the final Morning Glory deadline). This may be adjusted for future releases

  1. Bugfix = fix that happens after the module release deadline, but before the actual release. Please refer to the Morning Glory Release Schedule Most of these issues will be identified during Morning Glory R2 2022 BugFest Prep
  2. Update several fields in the Jira bug issue
    1. Make sure the priority is aligned with the definitions here: /wiki/spaces/DQA/pages/2657909
    2. Release field = Morning Glory R2 2022 Bug Fix
    3. Assign label bugfest_r2.2022 if the bug is found by a community tester during BugFest. This allows us to keep statistics on the bugs produced during that process. 
    4. Fill in the RCA (root cause analysis) field of the Jira, or else be sure that the Developer fills it in. 
    5. Link the Jira issue to the appropriate TestRail case(s). If there is no TestRail, create one, and then link to the bug.
  3. Let the developers doing the fix know that it is needed ASAP for a bugfix, definitely before the Morning Glory official release. Do not confuse with the Fix version field, which should reflect the release version the fix will be included in.
  4. Get the issue fixed and test on https://folio-snapshot.dev.folio.org/ (if possible - some performance issues that don't repro on snapshot will have to skip snapshot testing and be tested on the Bug Fest environment, instead)
  5. No permission is required for bugfixes before the release deadline. Once the fix has passed testing in snapshot, change the status to Awaiting release and the Resolution to Done, and ask the module maintainer to create a bugfix release.
    1. You can get the module maintainer’s name from the Team Module Responsibility Matrix.
    2. The release needs to go to the appropriate Release branch and the Main branch. 
    3. When the release is made, the module maintainer will announce it on the Slack #releases channel. 
  6. Create a release ticket for the module and assign  FOLREL-535 - Getting issue details... STATUS . Be sure that each bug in the release has the correct release version assigned to it.
  7. When the release has been made, the module maintainer will change the bug issue's status to Awaiting deployment. There is no need for the PO or module maintainer to request that the release be deployed
  8. Jenkins robot automatically generate pull request for the "R1 2022" branch of Platform Complete repository in Github for every released module if version incremented by "patch" number. (For example from 1.1.1 to 1.1.2)
  9. Manual intervention required if release version requires change of major (1.1.1 to  2.0.0) or minor (1.1.1 to 1.2.0) number. In this case PO or Dev lead  have to notify DevOps and Anton Emelianov (Deactivated) about this change.
  10. FOLIO hosting team checks for updates of Platform Complete one time per day and updates Morning Glory Bug Fest with modules that have been released since yesterday's update.
  11. FOLIO hosting team will post notification in #bug-fest Slack channel when update starts and ends. 
  12. Once daily update has been deployed to Bug Fest, Anton Emelianov (Deactivated) will change the bug issue's status to In bugfix review 
  13. When an issue is moved to In Bugfix Review, the PO and/or community should do a final round of testing in the Bug Fest environment
  14. Finally, when the issue has passed test in the Bug Fest environment, the PO should
    1. add a new test result to the TestRail case, which will hopefully move the test from Blocked or Failed to Passed.
    2. change the Jira Status to Closed


This describes the Morning Glory R2 2022 Hotfix release process that Product Owners should follow. This may be adjusted for future quarterly releases.

  1. Hotfix release = one that must be added to a release after the final release has been announced. Hotfixes occur after bugfixes.
  2. Generally, hotfix releases are not made except for P1 functional, P2 security, and Implementer Showstopper issues.  Once a hotfix release has been scheduled, lower priority issues (e.g. P2 and P3 functional issues) may also be included with permission.
  3. For R1 2022 hotfixes, the PO should ensure the following are adjusted in the bug Jira:
    1. Make sure the priority is aligned with the definitions here: /wiki/spaces/DQA/pages/2657909
    2. Set the Release field of the bug Jira issues to Morning Glory R2 2022 Hot Fix<Number> and make sure the developers doing the fix know that it is needed for a hotfix
    3. Fill in the RCA (root cause analysis) field of the Jira
    4. Include a comment with the TestRail test case(s) that were executed during Bugfest that should have caught the problem, but didn't. If there was no TestRail previously, create it and link to the current hotfix bug Jira.
  4. Get the issue fixed and test on https://folio-snapshot.dev.folio.org/
  5. Any hot fix issue must be approved by the Capacity Planning Team:
    1. Once the fix has passed testing in snapshot, post the Jira on the Slack #release_bug_triage channel and request approval for the fix. POs may also be asked to present the hotfix Jiras at the next available Capacity Planning meeting, usually held on Mondays at 9 am US ET. Hotfix Approvers are Khalilah Gambrell, Mike Gorrell, Harry Kaplanian, Jakub Skoczen and Mark Veksler.  Please tag the approvers with your request (can be considered approved if you get the thumbs up from at least two technical approvers (Mike, Mark or Jakub).  
  6. Once approved, on the issue's Jira, either a member of the Capacity Planning Team or the PO should set the field for Hot Fix Approved by Cap Planning? to Yes. Also include details about when the hot fix was approved in the Hot Fix Approval Comments field.
  7. If a hot fix is not approved, change the Release field to R2 2022 or later, and add comments either in the Hot Fix Approval Comments or in a regular Comment on the Jira.
  8. If approved, ask the module maintainer to create a hotfix release and set the JIRA status to Awaiting release, and Resolution to Done.
    1. You can get the module maintainer’s name from the Team Module Responsibility Matrix.
    2. The release needs to go to the appropriate Release branch and the Main branch. 
    3. The release Jira (not the bug Jira) should be linked to the appropriate R2 2022 Hotfix epic, currently FOLREL-536 - Getting issue details... STATUS
    4. When the release is made, the module maintainer will announce it on the Slack #releases channel. 
    1. The module maintainer will change the bug issue's status to Awaiting deployment. There is no need for the PO or module maintainer to request that the release be deployed. 
    2. Jenkins robot automatically generates pull request for the "R2 2022" branch of Platform Complete repository in Github for every released module if version incremented by "patch" number. (For example from 1.1.1 to 1.1.2)
    3. Manual intervention required if release version requires change of major ( 1.1.1 to  2.0.0) or minor (1.1.1 to 1.2.0) number. In this case PO or Dev lead must notify DevOps and Anton Emelianov (Deactivated) about this change.
    4. FOLIO hosting team checks for updates of Platform Complete one time per day and updates Bug Fest system with modules that have been released since yesterday's update.
    5. FOLIO hosting team will post notification in #bug-fest Slack channel when update starts and ends. 
    6. Once daily update has been deployed to Bug Fest, Anton Emelianov (Deactivated) will change the bug issue's status to In bugfix review which is the PO's trigger to do the final round of testing in the Bug Fest environment
    7. When an issue is moved to In Bugfix Review, the PO and/or community should do a final round of testing in the Bug Fest environment
    8. Finally, when the issue has passed test in the Bug Fest environment, the PO should change the Jira Status to Closed
  9. Add information about the hotfix to the Morning Glory R2 2022 Release Notes page on the wiki, in the Post-release Hotfixes table.
  10. Once the hotfix has passed testing on BugFest, it needs to be deployed to libraries using the Morning Glory release in production or in sandbox.
    1. For EBSCO-hosted libraries, ask an EBSCO PO or Anton to deploy the fix.
    2. For Index Data-hosted libraries, Wayne Schneider is managing the deployments. 
    3. Self-hosted libraries will need to decide whether to implement the hotfix or not, based on the announcement on the Slack releases channel or the release notes.
  11. If hotfixes are deployed by hosting providers, they should notify the libraries using the current release in production or sandbox that a hotfix has been deployed