Overview

• In this report, PTF conducts performance testing to identify the appropriate environment scale that will ensure the fulfillment of the NFRs outlined in ASA NFRs. The goal is also to identify potential bottlenecks in the system that may require code changes to optimize performance and meet the specified requirements.

https://folio-org.atlassian.net/browse/PERF-1079 

Summary

• During testing with different configurations, we observed that once OpenSearch exceeded 80% of its SPU capacity, performance was significantly impacted, leading to degraded ASA operations. As a result, we determined that the ASA load with additional traffic on the PTF-LOC environment should not exceed 80% of OpenSearch's SPU capacity. Therefore, we will use the r7g.4xlarge instance type to maintain optimal performance.

• During ASA testing with extra load on the PTF-LOC environment, OpenSearch SPU utilization reached 62%, compared to 52% without the extra load. We expected that OpenSearch CPU utilization would increase to 70% with the additional load, as in Test 1, where 31 sessions were lost due to insufficient capacity.

• Additionally, we found that operations involving the "CONTAINS_ALL" search criterion were significantly slower, with response times 6 to 12 times higher compared to other criteria.

• Despite the extra load on the PTF-LOC environment, overall performance remained stable, with variations of less than 5%.

• NFR_007 - Performance degradation should be less than 10% for each block of active users, which meets the NFR requirements.

• NFR_006 - The 95th percentile of response time is greater than 1 second, which does not match the NFR due to the use of the "CONTAINS_ALL" search criterion. However, the 99th percentile of response time is less than 2 seconds, which meets the NFR, even with "CONTAINS_ALL" included.

• NFR_004 - The 95th percentile of response time exceeds 2 seconds for some "CONTAINS_ALL" operations, which does not match the NFR, but all other ASA operations match the NFR.

• The total number of searching and viewing operations is 31 per second. Since we lost 31 user sessions, we can infer that the operations are close to 33 per second, which meets the NFR requirements.

Test Runs

Test Results

NFR_007

This table contains response times and percentage degradation for API Load Testing with a Gradual Increase of Users Per Block tests.