[PERF-173] Create AWS instances for Elasticsearch perf testing Created: 23/Dec/20 Updated: 29/Apr/21 Resolved: 29/Apr/21 |
|
| Status: | Closed |
| Project: | perf-testing |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Story | Priority: | P2 |
| Reporter: | Mikhail Fokanov | Assignee: | Mykola Borchuk |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||||||||||||||
| Sprint: | DevOps Sprint 110, DevOps Sprint 111, DevOps Sprint 112, DevOps Sprint 113 | ||||||||||||||||||||
| Development Team: | FOLIO DevOps | ||||||||||||||||||||
| Description |
|
Purpose/Overview: To make appropriate testing of required features on real-like data volume in cluster mode , there should be 2 AWS instances for Elasticsearch with the following configuration:
There should be (temporary for testing purposes):
Note: these instances can be turned off from 18:00 to 06:00 UTC. |
| Comments |
| Comment by Marc Johnson [ 26/Jan/21 ] |
|
Are 2 instances needed for each environment e.g. folio-snapshot, folio-testing, folio-snapshot-core, folio-testing-core? Is there an issue for also adding elastic search to the scratch environments? Is elastic search also going to be added to the Vagrant images? |
| Comment by Jakub Skoczen [ 02/Feb/21 ] |
|
Mikhail Fokanov we've have discussed this during the DevOps planning meeting and there are some open questions: 1. does this ticket request an ES instance for the purpose of performance testing, if so this is not the same as providing instances for the sole purpose of running reference environments (which have very little data) and there should be a separate ticket created as a blocker for
Mikhail Fokanov it might make sense for you to join the DevOps meeting at 15:30 CET to clarify? |
| Comment by Mikhail Fokanov [ 02/Feb/21 ] |
|
Jakub Skoczen, Unfortunetly, I wasn't available at 15:30 CET today and I also will have another meeting at this time tommorrow. First of all, these instances are needed for testing on the real data volume before the production. Also they could be used for: folio-snapshot, folio-testing, folio-snapshot-core, folio-testing-core . But if there is a small ammount of data on the mentioned envs, and you can provide another solution for the envs I would vote not to use these AWS instances for these envs. |
| Comment by Marc Johnson [ 02/Feb/21 ] |
This issue blocks
|
| Comment by Jakub Skoczen [ 03/Feb/21 ] |
|
Mikhail Fokanov John Malconian In this case we will create a new ticket for creating ES instances to support reference envs and Vagrant boxes and block
|
| Comment by John Malconian [ 16/Mar/21 ] |
|
It's not entirely clear what resources are needed here, but If you are using the us-east-1 region, create the instance(s) in 'FOLIO VPC' in subnet 'ID Public East 1a' and/or 'ID Public East 1b'. If you are providing developers SSH access to the system, please create a unique ssh key pair. Store the private key in Secrets Manager so we can access if needed. |
| Comment by Mykola Borchuk [ 19/Apr/21 ] |
|
For this task we use AWS ElasticSearch Service in us-west-2 region with Rancher. Amazon Elasticsearch Service (ES) uses private subnet and has own SG. Access to ES is from Rancher and containers. Snapshots for ES will be implemented (if possible) in other task. Also we changed DB to RDS for using DB snapshots with actual data. |
| Comment by Jakub Skoczen [ 27/Apr/21 ] |
|
Stanislav Miroshnichenko have you reviewed this ticket? If so, can it be closed? |
| Comment by Stanislav Miroshnichenko [ 27/Apr/21 ] |
|
Jakub Skoczen Yes, we can close it |