Bug Fest Q4 2019 Report

Owned by Anton Emelianov (Deactivated)
Last updated: Jan 08, 2020
3 min read

Summary

We run full regression test during Q4 Bug Fest. 89% of the test cases has been marked as PASSED. It is 12% higher compare to Bug Fest 3.2. Also, we had a record test case count of 689. Q4 release was delayed due to P1 and P2 defects that were found during Bug Fest. That being said, the Testing Community did a great job of finding blocking bugs and prevented them slipping into the release. We expect Q4 release to be finalized in the fist half of January.

Bug Fest notes:

  • We had fewer testers to participate in this  Bug Fest: 39 vs 43
  • Testing community has been a great help again! 31 testers from different institutions helped us to complete this test cycle.
  • It was much better executed test cycle compare to Q3.2:
    • Not executed 1 vs 33 
    • Blocked 4 vs 12
    • Retest 8 vs 40
    • Failed 66 vs 50
    • Passed 610 vs 464
    • Total 689 vs 599
  • Prior to the test run, small team of testers worked to create descriptions and steps for 425 test cases. Many thanks to everyone who contributed:
    • Ann-Marie Breaux
    • Anton Emelianov
    • Cate Boerema
    • Catherine Smith
    • Charlotte Whitt
    • Cheryl Malmborg
    • Colin Van Alstine
    • Cornelia Awenius
    • Darcy Branchini
    • Dennis Bridges
    • Emma Boettcher
    • Holly Mistlebauer
    • Jana Freytag
    • Jennifer Eustis
    • Khalilah Gambrell
    • Martin Spenger
    • Patricia Wanninger
    • Peter Duschner
    • Stefanie Sußmann
  • There is a positive trend in terms of balancing test case between Testing Community Members and Folio Staff:
    • Community members executed more test cases in Q4 vs Q3.2: 545 vs 395
    • Folio staff executed less test cases in Q4 vs Q3.2: 137 vs 180
  • We had several environment issues related to data migration and configuration of the reference data.
  • Overall, system performance was good. However it suffered due to the fact that some table indexes were missing during data migration.

Individual Participation:

Participation by Institution:



Highlights of the results:

  • 70 defects has been logged.
  • 13 defects has been closed as "duplicate", "cannot reproduce", or "won't do". It is much lower percentage compare to previous BugFest Q3.2.
  • Defects were created by 24 different participants.
  • Holly Mistlebauer is the Bug Fest Champion with 18 submitted defects.
  • Testers logged the most number for defects against ui-users (25).
  • ui-inventory and  mod-inventory-storage did much better this time. Only 4 defects has been filed.
  • ui-orders (5), ui-finance (6), and  ui-organizations (5) received the highest defect count. 
  • All 7 P1 defects were addressed before release. 
  • There were 21 P2 defects. 18 of them has been fixed. The rest has been assigned to the future releases.
  • Link to the live JIRA dashboard is here.
  • Q4 release was delayed due to time needed to fix defects that were found during Bug Fest Q4.