Bug Fest Q3.1 2019 Report

Owned by Anton Emelianov (Deactivated)
Last updated: Aug 12, 2019

Summary

We run full regression test during Q3.1 Bug Fest. 84% of the test cases has been marked as PASSED. It is the highest percentage out of all Bug Fest cycles that we had. Also, the team of professional testers from Test.io has been involved during Q3.1 Bug Fest (see details below). Happy to report that they didn't find any P1 or P2 defects. Overall, FOLIO did really well during this test run.

Bug Fest notes:

  • We didn't have Bug Fest planning (kick-off) meetings. All coordination was done via Slack.
  • Only 5% of test cases were marked as failed. 
  • All P1 and P2 defects required for Q3.1 release has been fixed before deadline.
  • Inventory search by barcode performance issue has been identified and fixed before the  release deadline.
  • We had several environment issues that were addressed by deploying hot-fixes.
  • Overall, system performance was good.

Highlights of the results:

  • 46 defects has been logged. 14 of them were found by Test.io (see notes below)
  • 8 defects has been closed as "duplicate", "cannot reproduce", or "won't do". 
  • Defects were submitted by 13 different participants. 
  • Testers logged the most number for defects against ui-users and ui-inventory modules. 8 defects each.
  • ui-eholdings, ui-requests, ui-data-import received 5 defects each.
  • There were 11 P2 defects. 5 of them has been fixed. The rest has been assigned to the future releases.
  • Link to the live JIRA dashboard is here


Test.io Pilot Test Run

  • Test.io is a crowd-testing company that provides on-demand functional manual testing as a service.
  • We conducted a pilot test run during Bug Fest 3.1
  • Testing scope was limited to ui-users, ui-checkin, ui-checkout, ui-inventory, and ui-eholdings
  • Testing was performed in the "exploratory" style
  • Test.io deployed 35 testers who completed defined work scope in 1 day.
  • 25 defects has been created and 14 defects has been accepted by PO's to the FOLIO backlog.
  • No P1 and only 1 P2 defect has been found by Test.io 
  • Overall, testers produces well-documented defects.
  • Every defect included a video capture.
  • Evaluation team was completely satisfied with the results and recommended to use Test.io in the future.