FEEDBACK:
It would be very helpful to collect your feedback prior to running the retrospective meeting.
We will be using https://funretro.io/ tool to collect your feedback. It is being used by multiple FOLIO development teams and we though it would be a good idea to follow their steps.
...
- Meeting will be scheduled between Thursday 3/14 and Friday 3/22.
- Go to LettuceMeet and post your availability
- Please be aware that 9 to 5 time on the schedule is in the Eastern Daylight Savings time zone.
- Follow this link to the Bug Fest Retrospective board
...
- and post your Feedback.
- Please add your comments, thoughts, ideas to "Went well" and "To improve" columns.
- If you like or agree with comments of the other contributors please don't hesitate to give them your vote of approval by clicking on the "thumbs up" button.
Also, we would appreciate if could provide your feedback on the following questions:
- Was it easy to follow BugFest instructions?
- Are there any gaps in the test case coverage?
- Should tasks provide more details and steps?
- If you have any questions, please post them into Questions column. The answers will be provided during Retrospective meeting and subsequently on this page.
MEETING INFO:
...
SLACK CALL on bug-fest channel 9 to 10 AM, Wednesday 3/20; New York Time. (check your local time)
If you have time conflicts please use bug-fest channel to let everyone know. We'll move the meeting if we get a lot of conflicts.
ACTION ITEMS:
...
BFQ4-18 ACTION ITEMS STATUS :
| Action Item | Status | Add to Playbook |
|---|---|---|
Consider to use TestRail for subsequent Bug Fest runs | Will be used in BF Q2 2019 | |
Not all the areas of FOLIO were covered. Do next BF soon to cover them. | DONE Coverage over 99% in BF Q1 2019 | |
| Better scope definition is needed to avoid testing incomplete features. | DONE |
|
| Always use real customer data. | DONE |
|
| Original reference data must be migrated to the BF environment. | DONE |
|
| Must have a planning meeting for every BF run: Define scope; Assign tasks to make sure that participants have time to complete them. | DONE |
|
Test cases for each module need to be reviewed with respective POs | DONE |
|
| Discussion is needed about how long BF environment is staying up after test run is over. | TBD | |
| Discuss what should be bug workflow for the future BF runs. |
Q&A:
Q: What happens next with the bugs?
A: Bugs were triaged. We assigned them proper priority. Several defects were closed after fixing performance problem. Another batch of bug was closed because they were filed against incomplete features. After that defects were moved from Bug Fest project to the respective component projects. It will be up to PO and the team to prioritize and assign them to a sprint.
Q: Unclear about proper way to handle several bugs filed which are covered by a more general issue. See, for example, STCOM-408, which is apparently the root cause of several Bug Fest bugs. Should those bugs be closed as duplicates of STCOM-408 or should they be linked up as blocked (as is currently the case)
A: It has been don properly. PO doesn't need to add these bugs to a team backlog but needs to check if the issue has been resolved after the main bug has been fixed. Linked bugs may require some specific work like fixing message test or modal window name.
Q: Not clear which browsers we should be testing in and what should be done with bugs specific to non-supported browsers (e.g. anything but Chrome). For example: UIIN-429
A: FireFox is not officially supported. Therefore this issue should be ignored at the moment.
Q: What to do with bugs that repro on BugFest environment but not in snapshot stable (e.g. UIREQ-193)?
A: It could be due to data size. In this case a developer should take a look at the BugFest system.
Q: How do we determine which team gets what bugs? Is it the team that originally implemented the feature? If this is the case, I assume the Core Functional team will get everything in Inventory, Codex, Users (except password management and fees/fines), Check in, Check out, Loan policies, Requests, most of Settings etc... That's a lot.
...
| TBD |
ACTION ITEMS
Action items were aggregated from the retrospective board:
Make sure that performance issues are reported separately
Use TestRail instead of the Google Spreadsheet
Transfer test cases into TestRail
Provide TestRail training
PO to improve test case descriptions
Post module/PO mapping before BF start for testers to post direct questions to the right person.
Call separate meeting on the bug triage and handling workflow.
PO should NOT test a module that they are working on
Start BF on Tuesday
Q&A:
Q: When planning future areas/features to test, are we planning to continue to test old features or to shift focus?
A: Yes, all existing features need to be retested (both old and new). Bug Fest is a regression test by its nature.
Q: When a feature has unresolved bugs from last BugFest, do we test it anyway (to see if there are new bugs) or leave it untested (because we know it will fail)?
A: PO should list known unresolved bugs prior to BF. A feature should be retested anyway. If bug is still there tester should add current bug fest label. If new bug has been found they should be logged as well.