2020-10-06 UI Testing Team Meeting Notes

60 min

Discussion

Everyone 

E2E Testing capabilities

• Unsure about capabilities for removing a tenant via API

• Testrail tests are run once per release as a crowd-sourced event. Anton posts the link to the test rails instance. One website is running previous release, and other is running next release

• Environments are “bugfest-goldenrod.folio.ebsco.com” and bugfest-honeysuckle.folio.ebsco.com”

  • Data is migrated from golden rod to honeysuckle

  • In Q1 system is built from UChicago data set

  • In Q2 (golden rod) we migrated the original dataset, then decommissioned 

  • In Q3 we will migrate Q2 over to to the Q3 release

• Next year, we will have three releases instead of four, but in both cases this test rail is the final QA event.

• Data is not refreshed. Main objective was to have an enormous data set.

• You cannot have real user data in the system at all, and so there may be a different class of bugs that arise based on an accumulation of usage data in the production environment, but we cannot lift the dataset from a library.

• Tenant that being used for TestRail test is nowhere close to real tenant, so more like a unit test.

Nightmare testing

• platform-core/test/ui-testing/*.js contains most tests, 

• Most is navigating around from one place to the next making sure the UI is ready

• Helpers from stripes testing library 

  • Not a great way to track whether something is successful well

  • If you click on a button that triggers a drop-down, the drop-down not blurring, therefore not validating. Difficult to simulate subtle UI interactions and sequences https://github.com/folio-org/stripes-testing/blob/master/helpers.js#L722

• Hard to know when you’re in nightmare and when you’re in the browser

• Some cleanup happens, but not a whole lot

• Run as part of the nightly provisioning script, only run against nightly deploy (unless developing)

Questions / Solutions

• One of the challenges that QA world has is that it is difficult to improve the tools

• Symbiotic relationship between development teams and QA.

  • Test Rail tests are very declarative and share a form with BigTest tests

  • Maybe generate the human driven tests off of the UI tests

• Develop workflows that center around tests that allow collaboration between QA and ui testers

• What are the requirements for the backend for automating the QA setup. 

  • The deployment aspect is not core to folio, so there’s very little we can assume

  • We know that it’ll be a DOM app

  • Want to assume the capabilities for deployment that we have now (EBSCO and AWS)

  • Let’s look at 10 tests that are very valuable, and can we implement them with what we have today

• Take Aways

  • Anton to give Frontside an account to Test Rails so that we can see the test plan

  • Take section by section of tests to automate 

  • Implement the nightmare tests and see how many are covered in Test Rail

  • How much can be handled with a single set of base data. E.g. if all of that circulation data is effectively read-only, we can run much more of our tests on that data.