[RANCHER-6] Collect requirements for platform testing Created: 19/Feb/21  Updated: 14/Mar/22

Status: Draft
Project: rancher
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P3
Reporter: Craig McNally Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Relates
relates to UXPROD-3387 Rancher Improvement: Implement Data M... Closed
Sprint:
Development Team: Kitfox

 Description   

Overview

There's an increasing need for a test environment which allows for platform-level operations to be easily tested. This serves as a place to capture the requirements/use-cases which will help drive the effort to create such an environment.

Use Cases

  • Start from a well-defined platform release (e.g. Honeysuckle GA). Deploy the latest snapshot versions of the platform (a la folio-testing/snapshot), and perform an upgrade.
    • Doing this on a module-by-module basis is too tedious and time consuming, we need the ability to specify something like a branch of platform-complete, or provide an install.json file, etc.
    • Having an "upgrade to latest" feature would be a huge time-saver (for example using the module list published at https://folio-testing.dev.folio.org/okapi-install.json ).
  • Given a branch of RMB or a specific backend module, we want to be able to perform a complete system upgrade and do some basic smoke testing

Other requirements

  • Logs must be accessible
  • We should be able to shut down the environment to save on costs - no need for it to be up 24/7.
  • The version and number of okapi instances must be adjustable.


 Comments   
Comment by Craig McNally [ 19/Feb/21 ]

cc Jakub Skoczen Hanna Hulevich Anton Emelianov Adam Dickmeiss

Guys, this is just a start - if you can think of other use cases or requirements please add them here Thanks!

Comment by Hongwei Ji [ 04/Mar/21 ]

I assume this env is based on current scratch env. Is it possible to specify the type of infrastructure to use? For example, Aurora for db, MSK for Kafka, AWS Elasticsearch managed service for Elasticsearch, and etc? If cannot be done, that's OK. We can test it separately in our hosting env as needed.

Comment by John Malconian [ 04/Mar/21 ]

I've contemplated the idea of running folio-snapshot on a single-server MicroK8s platform and giving backend FOLIO developers access to the K8s API in order do things like access logs, restart modules, and view resource utilization. The system would be comprised of backend and edge modules from platform-complete's snapshot branch (install.json). Instead of rebuilding the system each day from scratch, the backend modules would be upgraded daily (or on some other defined, regular basis) based on changes written to platform-complete's snapshot branch. For such a system to work, the scope or purpose needs to be limited. It cannot be co-opted and used as a system to preview UI features or development branches or to do demos. Its primary focus needs to backend integrations and tenant upgrades. I would even make the argument that it doesn't need a Stripes frontend. Other than ec2, I wouldn't rely too much on other AWS services that require setting up and managing IAM account and permissions.

Comment by Hongwei Ji [ 04/Mar/21 ]

John Malconian, just to be on same page, the reason I mentioned some AWS managed services in my comments is there was a time RMB worked fine with PostgreSQL but did not with AWS Aurora. Jakub Skoczen mentioned we should try to capture this kind of issues. I fully understand this might not be feasible or not easy to achieve, that's fine with me.

Comment by Marc Johnson [ 15/Apr/21 ]

Are these general requirements for changes to all of the rancher / scratch environments or requirements for a Core Platform team specific environment?

I'm asking because these requirements may be relevant for other teams as well.

Comment by Marc Johnson [ 15/Apr/21 ]

John Malconian

For such a system to work, the scope or purpose needs to be limited. It cannot be co-opted and used as a system to preview UI features or development branches or to do demos

Why is that the case?

Comment by Hanna Hulevich [ 15/Apr/21 ]

Marc Johnson we created this ticket to collect requirements for Core Platform env only. But probably other team has similar issues and some of requirements can be relevant for other teams as well. 

Comment by Marc Johnson [ 15/Apr/21 ]

Hanna Hulevich

we created this ticket to collect requirements for Core Platform env only. But probably other team has similar issues and some of requirements can be relevant for other teams as well.

Thanks.

Jakub Skoczen

which will help drive the effort to create such an environment.

Is the intention that this will be a separate environment solely for the Core Platform team, rather than expanded requirements for the existing scratch / rancher environments?

Comment by John Malconian [ 15/Apr/21 ]

Marc Johnson

For such a system to work, the scope or purpose needs to be limited. It cannot be co-opted and used as a system to preview UI features or development branches or to do demos

This comment really only applies to the potential dedicated environment option I described. I think if the existing scratch environments are suited to this team's needs, then I suppose you can use that environment for whatever you need. The idea behind the dedicated environment was specifically for testing module upgrades. I think if this is something we want to do as part of a regular CI process, then I wouldn't necessarily think it's a good idea to pollute it with other use cases.

Comment by Stanislav Miroshnichenko [ 16/Apr/21 ]

Craig McNally, what data do you need in DB to perform testing? Sample, Reference, 7M from Bugfest?

Comment by Craig McNally [ 19/Apr/21 ]

Sample/Reference is probably good to start - I don't think we identified any need for a larger dataset

Comment by Marc Johnson [ 19/Apr/21 ]

Former user Jakub Skoczen This has been moved to the Rancher project, does this mean that it has been decided that these requirements will be used to enhance the scratch environments?

Comment by Marc Johnson [ 19/Apr/21 ]

John Malconian

I think if the existing scratch environments are suited to this team's needs, then I suppose you can use that environment for whatever you need.

My understanding is that this issue exists because it has been decided that the scratch environments do not meet the Core Platform team's needs.

The idea behind the dedicated environment was specifically for testing module upgrades. I think if this is something we want to do as part of a regular CI process, then I wouldn't necessarily think it's a good idea to pollute it with other use cases.

Would it be reasonable to reword that as automated testing of module upgrades within a Jenkins build process should use a dedicated environment not shared by anything else (unlike the API integration tests)?

Comment by Vitaly Demchenko [ 08/Feb/22 ]

Craig McNally Do you have any progress on the requirements for this ticket? Who is responsible for collecting such type of requirements?

Comment by Vitaly Demchenko [ 14/Mar/22 ]

Craig McNally any updates on that one?

Generated at Thu Feb 08 23:25:05 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.