FameFlower Test Results

• Overview
• Test Runs
• Results
  • 1. High level FameFlower results data
  • 2.  CPU Utilization comparisons
  • 3.  Memory trends
  • 4.  Database CPU trends  
  • 5.  Slow queries
  • 6. Missing indexes
• Summary
  • Issues

Overview

Using the Carrier-io framework for capturing and analyzing performance test results, the following tests for the export instance UUIDs workflows were executed.

The testing was provided with the modules:

Backend:
- mod-inventory-storage-19.1.2
- mod-inventory-14.1.2
- mod-authtoken-2.4.0
- mod-permissions-5.9.0
- okapi-2.35.2
Frontend:
- folio_inventory-2.0.2

Test Runs

• 30-min Runs for export instance UUIDs workflow:

  Test	Virtual Users	Duration	OKAPI log level	Profiled	Rump up	Size of records
  1. FameFlower-Baseline	1	30 min	INFO	No	5	10K~50K instances
  2. FameFlower-Baseline	1	30 min	INFO	No	1	50K~100K instances
  3. FameFlower-Baseline	1	30 min	INFO	Yes	10	10K~50K instances
  4. FameFlower-Baseline	1	30 min	INFO	Yes	10	50K~100K instances
  5. FameFlower-Baseline	5	30 min	INFO	No	50	10K~50K instances
  6. FameFlower-Baseline	5	30 min	INFO	No	10	50K~100K instances
  7. FameFlower-Baseline	5	30 min	INFO	Yes	50	10K~50K instances
  8. FameFlower-Baseline	5	30 min	INFO	Yes	50	50K~100K instances

  
  

  Results

  *All numbers are in milliseconds except for those in the Delta % column, which indicates the difference in percentage going from 1 to 5 users, 10K~50K instances

  1. High level FameFlower results data

  Image RemovedImage RemovedImage AddedImage Added

• The chart shows the overall high-level API stats obtained by JMeter calling various APIs in the save instance UUIDs worfklow. It breaks down average response times for 1 and 5 users tests per API call. 

A few things to note:

• GET_/inventory/instances and GET_/instance-bulk/ids have the slowest response time. Have failed responses even for 1 user 

• The workflow with more than 100K records become unresponsive even with 1 user

• The workflow with more than 5 users become unresponsive

The charts below offer a clearer side-by-side comparison for the 1 and 5 users tests runs 

...

Summary

        See Attached FameFlower Performance Test Runs.xlsx for details 

• FOLIO performs better without being profiled when the tests are running 

...

• Most of failed requests were related to GET_/inventory/instances and GET_/instance-bulk/ids that use mod-inventory-storage service
• GET_/inventory/instances and GET_/instance-bulk/ids average response times are still over 300ms in the 8-users tests have failed responses even for 1 user

• The workflow with more than 100K records become unresponsive even with 1 user

• The workflow with more than 5 users become unresponsive
• Memory Issues: 
  • mod-inventory-storage has noticeable to significant gains in memory used.  
• fasterxml.jackson.databind.ObjectMapper.readValue method of mod-inventory-storage service overuses CPU resources as there are a lot of JSON decoding, this implementation could be reviewed and improved to reduce operations with JSON

...