Smoke testing - Only include test cases that are critical to the operation of your apps/modules
Brainstorm - How to minimize hotfixes
30 minutes
All
Questions to consider
What resources are needed?
Is there a step(s) that is missing?
Environment?
Observations
Analysis of hotfixes to see patterns
Reported via bugfest or production environments?
Regression
Unhappy path scenarios not covered
Edge cases BUT it is a primary use case for one library
E2E testing especially cross app interaction & dependencies
Automated testing to help focus testers on real life cases
Environment (setup inconsistencies)
Bugfest > Is it really a Gold Build environment?
Bugfest data is difficult to test and do not represent real-life workflow
Do we have an environment that represents a real-life environment?
Test data vs. Real data
Improving testing environments
More thorough testing and often
More time as a Public beta
UAT?
More exploratory testing?
Real life scenarios
Try actual workflow during bugfest period -
The production data is being populated in a different way that the test environments (including bugfest) and sometimes the issues are only manifesting themselves in the production. We should advocate for the consistency in how the data is populated.
Bring over a production libraries' environment
Maybe the support sig could spawn a QA working group. In DI there are technically oriented people who test pretty hard and who follow development specifically to understand what they should test. [Jenn Colt]
Set right expectations
Ask for QA resource. Or ask the community to allocate a QA resource.
Next steps:
Environment and data issues
Anton
Automated testing Capabilities
Capture Real-life scenarios
QA resource need - and ask of the community support