Rancher Improvement: Implement Data Migration Functionality
Description
Priority
TBDLabels
Fix versions
Development Team
Assignee
Hanna HulevichSolution Architect
Parent
Parent Field Value
Parent Status
is blocked by
is defined by
Checklist
hideTestRail: Results
Activity
Khalilah Gambrell March 3, 2022 at 1:02 PM
will this feature be included in the Lotus release?
Hanna Hulevich February 1, 2022 at 8:40 AM
Hi ,
I think this requirement should be covered. Only one thing we should do to support this is to prepare specific data set for the release version. So we can use this any time later. please what is your view?
We can discuss on demo this week
Thank you,
Hanna
Ann-Marie Breaux December 2, 2021 at 6:21 PM
Hi I happened to see your message in the tech leads channel. I'm not sure exactly how this fits, but I wanted to mention it.
In the hosted ref envs (folio-testing, snapshot, snapshot-load), we have default reference data for things like locations, acquisitions unit, all kinds of Inventory settings, etc.
In Bugfest, we have completely different, and much more diverse, reference data. Is it decided what the reference data for this Rancher migration env will be? I would advocate that if you plan to use a large set of ref data that the hosted ref env's ref data also be included, so that tests written for that specific set of ref data don't break.
I'm not sure if that's covered by your 4th bullet point in the requirements, but I wanted to mention it.
Hanna Hulevich November 23, 2021 at 1:56 PM
What does the third one do that is different to those two?
it will be a wrapper for first and second job. Will do both initial install and migration so you should not wait until first job is completed to run the second one.
What do you mean by
the performance rancher will be down for 8 hours?
For the cost saving purpose Perf Rancher env implemented in the way to become down in 8 hours (by default)
Are all of these jobs going to run against a single shared environment?
please help me to answer this question.
Thank you,
Hanna
Marc Johnson November 23, 2021 at 1:01 PM
one can install fresh FOLIO on specified data set, the second one just run a migration, and the third one do initial install and run migration.
I think the first one creates an initial environment, the second one runs a specified module upgrade.
What does the third one do that is different to those two?
As far as I know performer rancher will be down in 8 hours by default but this behavior can be changed and we can keep it longer if needed
What do you mean by the performance rancher will be down for 8 hours?
Are all of these jobs going to run against a single shared environment?
Details
Details
Reporter
Hanna Hulevich
Current situation or problem:
Currently there is no possibility to run migration on performance rancher. That's why we can't test migration from old release version to new one before bug fest. It is not optimal situation because teams need to have possibility test data migration (including data migration performance) during development to identify and solve possible issues on early stage.
Probably this functionality also can be used to the schema migration testing. Which help us eliminate a lot of efforts from the development teams.
(currently schema migration testing is done by the teams locally using vagrant box and a set of data generated by a team themselves)
This functionality should allow us to identify the problematic modules from the data migration performance point of view so SA can focus on improving them and this improvement can be measured after it is implemented
In scope
Jenkins job need to be implemented to run migration from the specified release version to the specified release version on the rancher performance environment on the specified data set.
Modules versions need to be taken from platform-complete by specified tag or from master branch. It should be possibility to override modules version/branch if needed (from configuration file. version/branch from the file will be finally deployed)
Logging should be added to log time took for data migration per module
(added by Raman Auramau) Logged duration should be aggregated in report and available via Jenkins job
(added by Raman Auramau) Notification is required if upgrade job executes too long (longer than some threshold)
We need to be sure that we are installing initial release version with compatible data set.
We can want to use Cornell data set (but sensitive data need to be disclosed) . We need to talk to Cornell before. But implementation should not be blocked by this. We can start implementation on Bugfest snapshot.
We want to be able to update not all modules but only some specified in the list.
We want to be able to disable functionality to check modules compatibility so we can test upgrade from any version to any version in any time.
We need to be able to restore the database to some state if something happened (ability to restore the database).
Out of scope
implementing this functionality for the development teams rancher envs
Preparing Cornell data for using with this env
Use case(s)
Proposed solution/stories
Links to additional info
Questions
Should we implement this functionality for the PTF env?
Kitfox and PTF should discuss this
Should Perf env should be supported for multiple teams in the same time?
Currently it supports 3 teams at a time