Calculate the standard deviation of CICO response times
Description
Environment
None
Potential Workaround
None
Checklist
hideTestRail: Results
Activity
Show:
Artem Vronik
updated the Linked IssuesMarch 13, 2023 at 2:00 PMThis issue is cloned by PERF-486
None
Artem Vronik
updated the Linked IssuesMarch 13, 2023 at 1:53 PMNone
This issue is cloned by PERF-486
Denis Anischenko
updated the Story PointsFebruary 27, 2023 at 3:19 PMNone
2
Hanna Tuhuzbayeva
changed the StatusFebruary 27, 2023 at 2:42 PMIn Review
Closed
Hanna Tuhuzbayeva
updated the ResolutionFebruary 27, 2023 at 2:42 PMNone
Done
Artem Vronik
changed the AssigneeFebruary 24, 2023 at 2:19 PMArtem Vronik
Hanna Tuhuzbayeva
Artem Vronik
changed the StatusFebruary 24, 2023 at 2:19 PMIn progress
In Review
Artem Vronik
updated the Linked IssuesFebruary 23, 2023 at 1:00 PMNone
This issue links to "Page (FOLIO Wiki)"
Artem Vronik
changed the StatusFebruary 21, 2023 at 6:29 AMDraft
In progress
Artem Vronik
changed the StatusFebruary 20, 2023 at 8:55 AMOpen
Draft
Artem Vronik
changed the AssigneeFebruary 20, 2023 at 8:55 AMUnassigned
Artem Vronik
Martin Tran
updated the RankFebruary 20, 2023 at 1:37 AMNone
Ranked higher
Martin Tran
updated the SprintFebruary 20, 2023 at 12:35 AMNone
PTF Sprint 159
Martin Tran
changed the PriorityFebruary 20, 2023 at 12:34 AMTBD
Martin Tran
updated the DescriptionFebruary 20, 2023 at 12:34 AMThe margin of errors is the small percentage of the distribution that falls outside the average range but is still relevant that we'd want to keep track of. Often time when running CICO tests we don't know what the true response times are because we perform only a few tests and the response times don't tell us anything conclusively. We need to find the margin of errors so that iwe can confidently say that the results are within the margin of errors or not.
Research the exact mathematical formula/algorithm to calculate the margin of errors. Run 10 CICO tests of 8 and 20 users separately (20 in total) and record 95%th response times. Use these data to feed into the formula.
If the margin of errors are difficult to obtain, still run the suggested number of tests, but this time calculate the poor man's version of margin of errors by:
* Find the average response times of all 8-users tests
* Group the ones that have similar response times of within 10ms of the average
* Group the ones that are 5-10% more or less of the average response times.
* Group the ones that are 11-20% more or less of the average response times.
* Count how many samples are in each of these groups.
* Repeat the above tests for the 20 users tests.
Acceptance criteria:
* Create a Confluence doc that defines the margin of errors, how to derive and calculate it.
* Report shows the distribution of the response times. and a graph (or chart) showing the bell curve and distributions.
*
The standard deviation can help to show how the response times of check in/out transactions are distributed, which is useful for analysis as we need to know which are the outlier response times to discard and which ones to keep.
Calculate the standard deviation of the overall check in and check out response times ("overall response time = check in or check out JMeter controller time). Run 10 CICO tests of 8 and 20 users separately (20 in total) and record 95%th response times. Use these data to feed into the standard deviation formula.
*Acceptance criteria:*
* Create a Confluence doc that defines the mathematical formula of std.
* Report shows the distribution of the response times. and a graph (or chart) showing the bell curve and distributions.
Martin Tran
updated the SummaryFebruary 20, 2023 at 12:29 AMCLONE - Calculate the margin of errors of CICO tests
Calculate the standard deviation of CICO response times
Martin Tran
updated the Linked IssuesFebruary 20, 2023 at 12:28 AMNone
This issue clones PERF-447
Martin Tran
created the IssueFebruary 20, 2023 at 12:28 AMDone
Assignee
Reporter
Martin Tran
Martin TranPriority
Story Points
2
Sprint
None
Development Team
None
TestRail: Cases
Open TestRail: Cases
TestRail: Runs
Open TestRail: Runs
Created February 20, 2023 at 12:28 AM
Updated March 13, 2023 at 2:00 PM
Resolved February 27, 2023 at 2:42 PM
TestRail: Cases
TestRail: Runs
The standard deviation can help to show how the response times of check in/out transactions are distributed, which is useful for analysis as we need to know which are the outlier response times to discard and which ones to keep.
Calculate the standard deviation of the overall check in and check out response times ("overall response time = check in or check out JMeter controller time). Run 10 CICO tests of 8 and 20 users separately (20 in total) and record 95%th response times. Use these data to feed into the standard deviation formula.
Acceptance criteria:
Create a Confluence doc that defines the mathematical formula of std.
Report shows the distribution of the response times. and a graph (or chart) showing the bell curve and distributions.