The Karate Test Framework(Karate) has an performance testing feature that FOLIO could employ to determine some performance benchmarks earlier in the software development lifecycle. Karate integrates with Gatling, another open source project to execute performance tests. Basically, a Gatling process kicks off the testing process & a Karate feature file can be used to exercise endpoints.

Benefits

• Drive load in developer environments to perform profiling. This will help with troubleshooting memory leaks, concurrency issues, inefficient algorithms & processes in a dev environment.

• Performance testing becomes portable for use by other segments of the FOLIO community e.g. hosting providers.

• When Karate tests are branched by release, performance tests would carry over as well.

• Unified testing language to drive both functional and performance testing.

• Leverage existing functional test scenarios for performance testing.

• Early performance feedback can be obtained by incorporating performance testing early in the development lifecycle.

Project Setup

The steps described are with the folio-integration-tests repository in mind. Review this PR for how to setup a module in folio-integration-tests.

Scenario Guidelines

Guidelines are similar to what is described here: Karate Coding Standards & Best Practices . A major characteristic required for a scenario is its repeatability within the same tenant & multiple virtual users executing the same scenario at the same time.

Continuous Integration

A jenkins plugin is available to show performance trends of Gatling runs. Performance tests can be executed on a schedule and appropriate reports are displayed in jenkins, just like regular cucumber reports from Karate.