Thunderjet - Quality Metrics

The question is - how will we know that the overall quality of the software in Acquisition & Finance domain is gradually improving as we implement our ideas? This requires metrics that characterize the state of A&F, and of particular importance is the understanding of these metrics in dynamics, over several release cycles. So, what metric can be used?

This Thunderjet - Bugs metrics Jira dashboard provides a lot of interesting statistics in terms of bugs and bugs reported in the Jira tracking system.

So, the following metrics are proposed:

Metric	Sense / Expectations	Source	Frequency of collection / analysis
Number of bugs including their priority	expectation is zero P1	Jira dashboard	once per sprint
Number of overall detected bugs by priorities and releases	the expected trend is to reduce the total number of bugs from release to release while reducing the priority of identified bugs	Jira dashboard	once per release
Number of bugs escaped into Production	the expected trend is to minimize and even avoid P1/P2 bugs in Production (P3 is acceptable); Production bugs are those which have RRT or Support label	Jira dashboard	once per release
Number of occurrences (similar bugs reported by different customers)		FSE / Rally log, as a part of RCA	once per release
Average time of bug fixing	there does not seem to be any SLA in this area yet, so the expected trend is to decrease this time as a result of improved tools for troubleshooting and analysis	manually, by the special script	once per release
No performance degradation from release to release	no Baseline metrics in this area; there are only measurements for exporting orders in Edifact format (Morning Glory -> Nolana - it got better) Action item: contact PTF, write performance tests, organize and load test against a set of scenarios in Orchid to get baseline values	performance testing	once per release