[FOLIO-1245] create automated performance testing environment Created: 15/May/18 Updated: 12/Nov/18 Resolved: 09/Jul/18 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Umbrella | Priority: | P3 |
| Reporter: | Jakub Skoczen | Assignee: | Wayne Schneider |
| Resolution: | Done | Votes: | 0 |
| Labels: | ci, performance, sprint38, sprint39, sprint40, sprint41, sprint42, sprint44, sprint45, sprint48 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||||||||||||||||||||||||||||||
| Issue links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | |||||||||||||||||||||||||||||||||||||||||||||||||
| Development Team: | Prokopovych | ||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
A living document describing the approach has been created during a session in Durham and is located here: https://docs.google.com/document/d/1MiF4N3ot3ewT8j9_jer6_br5ekOpXMrMEJR93WWPALA/edit |
| Comments |
| Comment by Jakub Skoczen [ 17/May/18 ] |
|
Guys, we have discussed that Hongwei Ji and Varun Javalkar from EBCSO will be helping out in the set up of the performance test environment, perfomance test coverage with jMeter and automation of the performance test suite. The work-in-progress document linked in the description captures the rough approach and initial requirements and goals for the environment and the test suite. Let's extend this document with more details and organize the work into smaller and more tangible deliverables. Hongwei Ji Mark Veksler if you'd like to take a first stab at scoping the deliverables before the actual work commences Wayne Schneider and I will be happy to review them and provide comments. I know you guys have some initial ideas already so any graphical or textual descriptions are welcome – either put them directly here or in the FOLIO Wiki or Gdoc/Drive and link them here. Thanks. |
| Comment by Hongwei Ji [ 17/May/18 ] |
|
Hi Jakub Skoczen and Wayne Schneider, I see in FOLIO-1066, Wayne Schneider already has a way to create a performance testing environment. Is it automated or can it be automated? More importantly, is it AWS independent?We do have a way to create FOLIO but it is coupled with AWS native technologies like ECS and ALB, so might not be a neutral candidate to benchmark FOLIO performance. Either way, we are happy to work together to automate the performance environment creation, come up with standard test data, and contribute to JMeter test scripts. More importantly, we should/will make all those available to community so they have more confidence on FOLIO and also benefit from these as well. |
| Comment by Jakub Skoczen [ 29/May/18 ] |
|
Hongwei Ji Wayne Schneider Guys, did you get a chance to talk about bootstrapping the env? |
| Comment by Hongwei Ji [ 04/Jun/18 ] |
|
Hi Jakub Skoczen and all, this is what I have so far. Thanks for the help from Wayne Schneider and John Malconian
|
| Comment by Hongwei Ji [ 06/Jun/18 ] |
|
BTW, I stood up a temp Jenkins to show the pipeline. http://jenkins.int.aws.folio.org:9200/ |
| Comment by John Malconian [ 06/Jul/18 ] |
|
After some modifications to FOLIO Jenkins and the configuration in the repository, I'm able to get a successful run in FOLIO Jenkins. https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-test/ I've merged branch
Outstanding:
|
| Comment by Hongwei Ji [ 06/Jul/18 ] |
|
John Malconian, I believe the Performance graphs need at least two successful builds to show the trend. As for the job frequencies, I have been running it only once a day in our environment. |
| Comment by John Malconian [ 09/Jul/18 ] |
|
Thanks, Hongwei Ji. You are correct about the graphs. I've scheduled the job to be run daily. |
| Comment by Jakub Skoczen [ 25/Jul/18 ] |
|
What is left here to make sure we can use the environment to track performance regressions and address them. We need a clear upgrade process, is it implemented? |
| Comment by Hongwei Ji [ 25/Jul/18 ] |
|
Jakub Skoczen it pulls the latest code from snapshot-stable automatically every time. Can you elaborate the "clear upgrade process"? |