| Table of Contents |
|---|
Brief context
...
Analyze
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
...
To make the process of preparing and testing data migration scripts more transparent, understandable and effective, and to reduce the risks of problems occurring directly during the release / migration on the production environment, 2 main steps are proposed:
| Info | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
|
...
More examples of data migrations:
Jira Legacy server System JiraJIRA serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key CIRCSTORE-295
...
| #1 Lazy migration | #2 Expand-contract pattern | #3 Optimization | #4 Separate Applications approach | #5 Blue-green deployment | #6 Aggregate several data migration in one | |
|---|---|---|---|---|---|---|
| Explanation | Update FOLIO platform without immediate data migration, and start a background task(-s) for gradual data migration Applications are to be ready to work with both migrated and not-yet-migrated data while migration is in progress | Update FOLIO platform without immediate data migration but build the migration into the code, and do a "modify upon get". So whenever a record is accessed, perform the necessary changes and write them back to the DB. This way the migration happens over time, and only one schema is in play at a given time. More to read https://www.prisma.io/dataguide/types/relational/expand-and-contract-pattern | Follow existing way of data migration as a part of release but review and analyze data migration scripts, and optimize them | Follow an approach of FOLIO deployment as not a monolith but rather as a set of independent applications. As the result release of individual applications will be possible, and data migration will also be distributed among applications | Before migration a) create a clone of main database, b) execute data migration on the clone, c) roll over all new changes that happened during the migration, d) during release - switch main database from the previous one to this clone, and consider the clone as a new main db | |
| Benefits | No data migration as a part of release | No data migration as a part of release | No significant changes in the FOLIO platform logic. Potential improvements with minimal costs | Even long migration will affect a particular application only rather than full platform | Safe non-blocking migration | |
| Concerns | The data migration process can be less transparent to engineers, while leaving the risks of partially completed migrations; application logic gets more complex as well | Not suitable for cases with breaking changes or when fully migrated data is required for proper functioning (e.g. for filtering, or sorting, etc.) | There might be a case of still unsatisfactory performance while no place for further optimization | Data migration is distributed across applications, and the approach to migration itself does not change. Therefore, a lengthy migration process is still possible, although this will only affect one application and not the entire platform | Step c) from the list above - accurate synchronization of all changes that occurred in the main database during the period of time between cloning and switching is required | |
| Efforts | Totally new approach, so that one need implement background task runner, make sure applications can work with both versions of data | Totally new approach | There is a chance that small point optimizations can be efficient enough | This is just a side-effect of Separate Applications approach | Mainly devops efforts to automate the process |
...