| Note | ||
|---|---|---|
| ||
We run full regression test during Q4 Q1 2020 Bug Fest.89% 91% of the test cases has been marked as PASSED. It is 12% higher compare to Bug Fest 3.2. Also, we had a record test case count of 689. Q4 release was delayed due to P1 and P2 defects that were found during Bug Fest. That being said, the Testing Community did a great job of finding blocking bugs and prevented them slipping into the release. We expect Q4 release to be finalized in the fist half of January. |
|
Bug Fest notes:
Please see table below that compares results for the pas 3 Bug Fest Cycles.
We had fewer testers to participate in this Bug Fest: 36 vs 39
vs 43- Testing community has been a great help again! 31 testers from different institutions helped us to complete this test cycle.
- It was much better executed test cycle compare to Q3.2:
- Not executed 1 vs 33
- Blocked 4 vs 12
- Retest 8 vs 40
- Failed 66 vs 50
- Passed 610 vs 464
- Total 689 vs 599
- Prior to the test run, small team of testers worked to create descriptions and steps for 425 test cases. Many thanks to everyone who contributed:
- Ann-Marie Breaux
- Anton Emelianov
- Cate Boerema
- Catherine Smith
- Charlotte Whitt
- Cheryl Malmborg
- Colin Van Alstine
- Cornelia Awenius
- Darcy Branchini
- Dennis Bridges
- Emma Boettcher
- Holly Mistlebauer
- Jana Freytag
- Jennifer Eustis
- Khalilah Gambrell
- Martin Spenger
- Patricia Wanninger
- Peter Duschner
- Stefanie Sußmann
- There is a positive trend in terms of balancing test case between Testing Community Members and Folio Staff:
- Community members executed more test cases in Q4 vs Q3.2: 545 vs 395
- Folio staff executed less test cases in Q4 vs Q3.2: 137 vs 180
- We had several environment issues related to data migration and configuration of the reference data.
Product Owners picked up more test cases this time. A lot of test cases were added the last minute and cannot be delegated to community testers.
It was the first time we were using large inventory dataset (almost 9M items). It allowed us to uncover a lot of issues that didn't manifest themselves with a smaller dataset.
Most of schema updates issues were relatively easy to resolve. However, we had to delay Bug Fest start by 1 day.
Overall, system performance was good.
However it suffered due to the fact that some table indexes were missing during data migration.
Individual Participation:
Participation by Institution:
Highlights of the results:
- 70
117 defects has been logged.
- 13
19 defects has been closed as "duplicate", "cannot reproduce", or "won't do". It is
much lower percentageabout the same compare to previous BugFest
Q3.2. - Defects were created by 24 different participants.
- Holly Mistlebauer is the Bug Fest Champion with 18 submitted defects.
- Testers logged the most number for defects against ui-users (25).
- ui-inventory and mod-inventory-storage did much better this time. Only 4 defects has been filed.
- ui-orders (5), ui-finance (6), and ui-organizations (5) received the highest defect count.
- All 7
.
Testers didn't create defects in Jira this time. This task was assigned to product owners. Therefore we cannot provide number of defects per tester report
ui-users module receive the highest number of defects(31) again.
Notably, mod-circulation (14), ui-inventory(18), ui-requests(10) did not as good as the other modules.
All 21 P1 defects were addressed before release.
There were
2146 out of 54 P2 defects
. 18 of them has been fixed. The rest has been assigned to the future releases.- Link to the live JIRA dashboard is here.
- Q4 release was delayed due to time needed to fix defects that were found during Bug Fest Q4.
were fixed before release or included into Hot Fixes.
Q1 release was delivered as planned on time.
