<!-- 
RSS generated by JIRA (1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d) at Fri Feb 09 00:18:46 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary add field=key&field=summary to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>FOLIO Jira</title>
    <link>https://folio-org.atlassian.net</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>1001.0.0-SNAPSHOT</version>
        <build-number>100246</build-number>
        <build-date>07-02-2024</build-date>
    </build-info>

<item>
            <title>[UXPROD-1816] Real-world Load and Performance Testing Methodology</title>
                <link>https://folio-org.atlassian.net/browse/UXPROD-1816</link>
                <project id="10000" key="UXPROD">UX Product</project>
                    <description>&lt;p&gt;We need an environment, set of tests and an approach to load and performance testing that exercise the system based on realistic usage/behavior. Things like waits, concurrency, sequence, parameters, etc... all of these things can contribute to realistic and valuable test results, or if not founded in real-world and expected usage, garbage results that force us to chase our tail. There&apos;s no sense in working on optimizing an API so that it can stand up to a pounding if it will never be pounded in real life. &lt;/p&gt;

&lt;p&gt;We need to test front-end modules as well as backend. We&apos;ve all seen times when the frontend is not using the backend correctly and causing performance problems as a result. Backend components&apos; contribution to performance are obvious.&lt;/p&gt;

&lt;p&gt;I think we need a few things:&lt;br/&gt;
1) An environment we can use for end-to-end and API testing. We need to understand limitations and uniqueness of these environments - how applicable are they to a live production environment such that when an issue comes up we can determine it&apos;s importance. Ideally there&apos;d be a nice direct correlation between the environment and a production environment.&lt;br/&gt;
2) A transparent and well understood testing methodology. We throw around terms like &quot;50 user test&quot;. What does that mean? What does each user do? are they identical? If not identical, what&apos;s the distribution of what they do (simple vs complex, etc.)..? Are there sleep times between actions? Do they ramp up and ramp down? How long does each live?&lt;br/&gt;
3) Ability to replicate results - so that we can see regressions and also so that we can reproduce or diagnose problems when needed.&lt;br/&gt;
4) We need to understand what realistic system behavior is. For a given tenant, what % of activity is read vs write, circulation vs acquisitions, etc. Some parts of FOLIO may be isolated such that it doesn&apos;t matter what&apos;s happening in Acquisitions as far as Inventory or Circulation go... but some pieces of the system MAY care. This also may depend on how the system is laid out in terms of deployment and load balancing (not just software architecture).&lt;/p&gt;

&lt;p&gt;This epic/issue will be used to track the various activities that relate to the overall load and performance testing approach for FOLIO.&lt;/p&gt;</description>
                <environment></environment>
        <key id="11762">UXPROD-1816</key>
            <summary>Real-world Load and Performance Testing Methodology</summary>
                <type id="10002" iconUrl="https://folio-org.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10322?size=medium">New Feature</type>
                            <parent id="11697">UXPROD-1814</parent>
                                    <priority id="10001" iconUrl="https://dev.folio.org/assets/jira-priority/jira-p2.svg">P2</priority>
                        <status id="1" iconUrl="https://folio-org.atlassian.net/images/icons/statuses/open.png" description="The issue is open and ready for the assignee to start work on it.">Open</status>
                    <statusCategory id="2" key="new" colorName="blue-gray"/>
                                    <resolution id="-1">Unresolved</resolution>
                                                        <assignee accountid="557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d">Jakub Skoczen</assignee>
                                                                <reporter accountid="5ae08bd47b44642d39d323c4">Mike Gorrell</reporter>
                                    <labels>
                            <label>cap-mvp</label>
                            <label>performance</label>
                    </labels>
                <created>Mon, 3 Jun 2019 10:59:41 +0000</created>
                <updated>Tue, 2 Nov 2021 13:51:56 +0000</updated>
                                                                                <due></due>
                            <votes>0</votes>
                                    <watches>8</watches>
                                                                <comments>
                                                            <comment id="17768" author="557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d" created="Wed, 5 Jun 2019 13:00:25 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5ae08bd47b44642d39d323c4&quot; class=&quot;user-hover&quot; rel=&quot;5ae08bd47b44642d39d323c4&quot; data-account-id=&quot;5ae08bd47b44642d39d323c4&quot; accountid=&quot;5ae08bd47b44642d39d323c4&quot; rel=&quot;noreferrer&quot;&gt;Mike Gorrell&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I think a next step would be to start breaking down this epic into user stories. Initially large and high-level, that&#8217;s okay, we can break them down further and/or elevate some to the epic level. Couple of high level user-stories  come to my mind:&lt;br/&gt;
&#8226; spec out the environment, including:&lt;br/&gt;
        a) how is it hosted and constructed  &#8212; is there any automation e.g built daily based on snapshots, or quaterly based on releases. The frequency of testing will be affected by this (and will also influence this)&lt;br/&gt;
       b) data set &#8212; what data set do we use (choose one), some of it ike bib data could be a real life data set, other likely (like transaction) needs to be artificial but preferably modelled after real life data. How is this data set going to get updated (FOLIO adds new records and extends existing one from version to version) and whose responsibility is it (e.g programmers vs devops)&lt;br/&gt;
&#8226; define the test methodology &#8212; establish main guidelines on things like warm up, cool down between tests, level of concurrency for each tests (number of virtual users, their ramp up and ramp down), level of concurrency between different test cases (if any). Establish some baselines on what we treat as acceptable performance and what is not acceptable (probably a topic in it&#8217;s own right)&lt;br/&gt;
&#8226; define initial set of test scenarios &#8212; there could be many so the first step is to focus on most frequent real-life operations. How do we get this information? Look at logs from existing systems? If so, this is something where sysop sig might be instrumental. A taks for them to analyze their logs. Going forward we need to have a process on extending the initial test cases &#8212; adding more test will involve adding more perf data &#8212; again depending on how we want to do it&#8217;s either a responsibility of the test author, developer building new functionality or the devops guys maintaining the tests.&lt;br/&gt;
&#8226; plan how the test process (and results) feed into the development process &#8212; e.g if we focus on testing quarterly releases I would imagine the perf data sets and scenarios are updated and ready when a new release comes out, the perf testing team tests it and creates appropriate tickets for the dev team(s). Dev teams will need to have some ways to reproduce or analyze the perf problems: e.g access to monitoring tools on the env and ability to rerun test cases, or the instrumentation  data must be available to them for analysis (sometime it&#8217;s probably enough). This is more of a general thing in FOLIO and must be addressed because the people running FOLIO and tracking perf issues are not the same as the people fixing them, so there must be process for how to report perf problems effectively.&lt;/p&gt;</comment>
                                                            <comment id="17769" author="5ae08bd47b44642d39d323c4" created="Wed, 5 Jun 2019 14:51:27 +0000"  >&lt;p&gt;Added several stories.&lt;/p&gt;</comment>
                                                            <comment id="17770" author="557058:b45447b0-4142-447e-9ac7-b2b97eeb90bc" created="Mon, 17 Jun 2019 14:00:33 +0000"  >&lt;blockquote&gt;&lt;p&gt;define initial set of test scenarios &#8212; there could be many so the first step is to focus on most frequent real-life operations. How do we get this information? Look at logs from existing systems?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;For the initial set of test scenarios we could rely on the libraries&apos; existing data, as they would have information on rates of check in/out, requests, renew, and other workflows. Using this data we can assess which ones are more used than others. Going forward we can get this information by adding custom metrics to our modules. Metrics such as rate, count, and timer that are strategically placed in various parts of the module can help to identify which APIs or paths of the code are being called and their frequency of being called. For the first release we can&apos;t rely on these metrics because they won&apos;t be exercised by real customers yet, but once a real customer starts using the system, these metrics will light up and able to tell us the scenarios and workflows that customers often use.  Having metrics can also tell us how the system performs in real time, and aid with analyzing performance issues.&lt;/p&gt;</comment>
                                                            <comment id="17771" author="5af5ed55244bc90a106063c7" created="Mon, 29 Jul 2019 16:41:56 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5ae08bd47b44642d39d323c4&quot; class=&quot;user-hover&quot; rel=&quot;5ae08bd47b44642d39d323c4&quot; data-account-id=&quot;5ae08bd47b44642d39d323c4&quot; accountid=&quot;5ae08bd47b44642d39d323c4&quot; rel=&quot;noreferrer&quot;&gt;Mike Gorrell&lt;/a&gt; and &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3Ab8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; class=&quot;user-hover&quot; rel=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; data-account-id=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; accountid=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; rel=&quot;noreferrer&quot;&gt;Jakub Skoczen&lt;/a&gt; can we assign this to someone and get a PO rank on it.  Thanks!&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10000">
                    <name>Blocks</name>
                                                                <inwardlinks description="is blocked by">
                                        <issuelink>
            <issuekey id="81433">FOLIO-2296</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10003">
                    <name>Relates</name>
                                            <outwardlinks description="relates to">
                                        <issuelink>
            <issuekey id="38401">DEBT-6</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Back End Estimate</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10124"><![CDATA[XXL < 30 days]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10054" key="com.atlassian.jira.plugin.system.customfieldtypes:userpicker">
                        <customfieldname>Back End Estimator</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10000" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummarycf">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10057" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Development Team</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10168"><![CDATA[None]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10014" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue key="$xmlutils.escape($text)">Platform, DevOps and Release Management</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10056" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Estimation Notes and Assumptions</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>XXL is the largest available. This item is likely larger.</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10062" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Kiwi Planning Points (DO NOT CHANGE)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>20.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10019" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0|hzzshb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10071" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: 5Colleges (Full Jul 2021)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10223"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10068" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: Chicago (MVP Sum 2020)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10208"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10069" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: Cornell (Full Sum 2021)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10213"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10070" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: Duke (Full Sum 2021)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10218"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10074" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: GBV (MVP Sum 2020)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10238"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10078" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: Lehigh (MVP Summer 2020)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10259"><![CDATA[R2]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10085" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: MO State (MVP June 2020)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10297"><![CDATA[R4]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10089" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: TAMU (MVP Jan 2021)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10314"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10091" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: U of AL (MVP Oct 2020)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10325"><![CDATA[R2]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10076" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: hbz (TBD)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10248"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10020" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10024" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>[CHART] Date of First Response</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 5 Jun 2019 13:00:25 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                </customfields>
    </item>
</channel>
</rss>