Platform, DevOps and Release Management (UXPROD-1814)

[UXPROD-2120] support for upgrading schemas without complete reload of data (DB migrations) - rollout + devops Created: 11/Oct/19  Updated: 31/May/23  Resolved: 31/May/23

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: None
Parent: Platform, DevOps and Release Management

Type: New Feature Priority: P3
Reporter: Tod Olson Assignee: Taras Spashchenko
Resolution: Won't Do Votes: 0
Labels: back-end, cap-mvp, platform-backlog, po-mvp
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
is blocked by FOLIO-2169 SPIKE investigate schema migrations i... Closed
Cloners
clones UXPROD-1815 support for upgrading schemas without... Closed
Duplicate
is duplicated by UXPROD-753 live db upgrades (db migration) Closed
Relates
relates to DEBT-2 No automatic schema/data migrations Closed
relates to FOLIO-464 Design and implement module upgrade p... Closed
relates to FOLIO-1519 Automatic loading of sample and refer... Closed
relates to FOLIO-406 Design module life cycle Closed
relates to FOLIO-838 performance test and bulk import test... Closed
relates to MODVEND-81 Load sample data via tenant API Closed
Epic Link: Platform, DevOps and Release Management
Back End Estimate: XXL < 30 days
Back End Estimator: Jakub Skoczen
Estimation Notes and Assumptions: XXL is the largest available. This item is likely larger.
Development Team: Core: Platform
Kiwi Planning Points (DO NOT CHANGE): 20
PO Rank: 9
Rank: BNCF (MVP Feb 2020): R1
Rank: Chicago (MVP Sum 2020): R1
Rank: Cornell (Full Sum 2021): R1
Rank: Duke (Full Sum 2021): R1
Rank: 5Colleges (Full Jul 2021): R1
Rank: GBV (MVP Sum 2020): R1
Rank: hbz (TBD): R1
Rank: Lehigh (MVP Summer 2020): R1
Rank: Leipzig (ERM Aut 2019): R2
Rank: MO State (MVP June 2020): R1
Rank: TAMU (MVP Jan 2021): R1
Rank: U of AL (MVP Oct 2020): R2

 Description   

Currently in FOLIO, if there is a change to the schema used by a storage module, there is no upgrade path for the data already in the module. As we now have test deployments and are trying to move towards production deployments, this will be critical for ongoing testing and operations.

This lack has been identified as Technical Debt by the Technical Council.

Some solution for schema upgrades and migrating data will be needed for all storage modules. The solution could include multiple components. For example, maybe one part of the solution is to include data updates the the Definition of Done for each development team, and maybe there's something that could be added to the core to make it easier to create these migrations. That's just for illustration. The important thing is that FOLIO needs a way to address address schema updates and corresponding data changes that can be consistent across modules.

SysOps identified these sub-issues:

  • Ability to roll back an upgrade
  • Consider the wider scope of how upgrades will occur in FOLIO systems
  • Downtime considerations, can we limit or have no downtime for an application?
  • May need additional work in modules/Okapi/RMB
  • Would want schema upgrade included in Definition of Done

FOLIO June meeting TC tech debt slides:
https://docs.google.com/presentation/d/1Cz5-xhvMdCdm7SXYLhIQE8KV9YIsQW1faNUIWmMANx8



 Comments   
Comment by Julian Ladisch [ 29/Jan/20 ]

Can someone lift the embargo from the slides https://docs.google.com/presentation/d/1Cz5-xhvMdCdm7SXYLhIQE8KV9YIsQW1faNUIWmMANx8 ? Thanks!

Generated at Fri Feb 09 00:21:18 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.