Calculate the standard deviation of CICO response times

Description

The standard deviation can help to show how the response times of check in/out transactions are distributed, which is useful for analysis as we need to know which are the outlier response times to discard and which ones to keep.  

Calculate the standard deviation of the overall check in and check out response times  ("overall response time = check in or check out JMeter controller time).  Run 10 CICO tests of 8 and 20 users separately (20 in total) and record 95%th response times. Use these data to feed into the standard deviation formula. 

Acceptance criteria:

  • Create a Confluence doc that defines the mathematical formula of std. 

  • Report shows the distribution of the response times. and a graph (or chart) showing the bell curve and distributions.

Environment

None

Potential Workaround

None

Confluence content

mentioned on

Checklist

hide

TestRail: Results

Activity

Show:
Artem Vronik
updated the Linked IssuesMarch 13, 2023 at 2:00 PM
This issue is cloned by PERF-486
None
Artem Vronik
updated the Linked IssuesMarch 13, 2023 at 1:53 PM
None
This issue is cloned by PERF-486
Denis Anischenko
updated the Story PointsFebruary 27, 2023 at 3:19 PM
None
2
Hanna Tuhuzbayeva
changed the StatusFebruary 27, 2023 at 2:42 PM
In Review
Closed
Hanna Tuhuzbayeva
updated the ResolutionFebruary 27, 2023 at 2:42 PM
None
Done
Artem Vronik
changed the AssigneeFebruary 24, 2023 at 2:19 PM
Artem Vronik
Hanna Tuhuzbayeva
Artem Vronik
changed the StatusFebruary 24, 2023 at 2:19 PM
In progress
In Review
Artem Vronik
updated the Linked IssuesFebruary 23, 2023 at 1:00 PM
None
This issue links to "Page (FOLIO Wiki)"
Artem Vronik
changed the StatusFebruary 21, 2023 at 6:29 AM
Draft
In progress
Artem Vronik
changed the StatusFebruary 20, 2023 at 8:55 AM
Open
Draft
Artem Vronik
changed the AssigneeFebruary 20, 2023 at 8:55 AM
Unassigned
Artem Vronik
Martin Tran
updated the RankFebruary 20, 2023 at 1:37 AM
None
Ranked higher
Martin Tran
updated the SprintFebruary 20, 2023 at 12:35 AM
None
PTF Sprint 159
Martin Tran
changed the PriorityFebruary 20, 2023 at 12:34 AM
TBD
P2
Martin Tran
updated the DescriptionFebruary 20, 2023 at 12:34 AM
The margin of errors is the small percentage of the distribution that falls outside the average range but is still relevant that we'd want to keep track of.  Often time when running CICO tests we don't know what the true response times are because we perform only a few tests and the response times don't tell us anything conclusively.  We need to find the margin of errors so that iwe can confidently say that the results are within the margin of errors or not.  Research the exact mathematical formula/algorithm to calculate the margin of errors.  Run 10 CICO tests of 8 and 20 users separately (20 in total) and record 95%th response times. Use these data to feed into the formula.  If the margin of errors are difficult to obtain, still run the suggested number of tests, but this time calculate the poor man's version of margin of errors by: * Find the average response times of all 8-users tests * Group the ones that have similar response times of within 10ms of the average * Group the ones that are 5-10% more or less of the average response times.   * Group the ones that are 11-20% more or less of the average response times. * Count how many samples are in each of these groups.  * Repeat the above tests for the 20 users tests. Acceptance criteria: * Create a Confluence doc that defines the margin of errors, how to derive and calculate it. * Report shows the distribution of the response times. and a graph (or chart) showing the bell curve and distributions. *  
The standard deviation can help to show how the response times of check in/out transactions are distributed, which is useful for analysis as we need to know which are the outlier response times to discard and which ones to keep.   Calculate the standard deviation of the overall check in and check out response times  ("overall response time = check in or check out JMeter controller time).  Run 10 CICO tests of 8 and 20 users separately (20 in total) and record 95%th response times. Use these data to feed into the standard deviation formula.  *Acceptance criteria:* * Create a Confluence doc that defines the mathematical formula of std.  * Report shows the distribution of the response times. and a graph (or chart) showing the bell curve and distributions.
Martin Tran
updated the SummaryFebruary 20, 2023 at 12:29 AM
CLONE - Calculate the margin of errors of CICO tests
Calculate the standard deviation of CICO response times
Martin Tran
updated the Linked IssuesFebruary 20, 2023 at 12:28 AM
None
This issue clones PERF-447
Martin Tran
created the IssueFebruary 20, 2023 at 12:28 AM
Done

Assignee

Hanna Tuhuzbayeva

Reporter

Priority

Story Points

Sprint

Development Team

None

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs
Created February 20, 2023 at 12:28 AM
Updated March 13, 2023 at 2:00 PM
Resolved February 27, 2023 at 2:42 PM
TestRail: Cases
TestRail: Runs

Flag notifications