BF-970 Sidecar Performance Testing
- Pavel Filippov
- 1 Purpose
- 2 Approach
- 2.1 Testing Framework
- 2.2 folio-module-sidecar
- 2.3 keycloak
- 2.4 mock-server
- 3 Performance test configuration
- 4 Performance test results
- 4.1 50 ms delay / 0.1 vCPU / 1 Event Loop / 5 Thread workers
- 4.2 50 ms delay / 0.25 vCPU / 1 Event Loop / 5 Thread workers
- 4.3 50 ms delay / 0.5 vCPU / 1 Event Loop / 5 Thread workers
- 4.4 50 ms delay / 1 vCPU / 1 Event Loop / 5 Thread workers
- 4.5 200 ms delay / 1 Event Loop / 5 Thread workers
- 4.6 1000 ms delay / 1 Event Loop / 5 Thread workers
- 4.7 50 ms delay / 0.1 vCPU 1 Event Loop / 5 Thread workers / Native Image
Purpose
Evaluate sidecar performance under different types of loads:
Ingress requests with user JWT token
Ingress requests with system JWT token
Ingress requests with user and system JWT tokens
Unauthorized egress requests
Approach
Testing Framework
Load testing designed for DevOps and CI/CD | Gatling
Automatic token issuing with specified delay for each pair of username + password in ingress-requests.file
Ingress and egress requests file is formed based on available user capabilities and has the following format
{"username":"nENjhUV6_vDJptu3f30DB","password":"yAW0AUyvFOZuUGOdICel","method":"POST","url":"/inventory-hierarchy/items-and-holdings"}
{"username":"giLofMi8_QmkHFrjEFKXd","password":"iYJWfBVJ5bit3v6MZCd1","method":"DELETE","url":"/identifier-types/{id}"}
{"username":"N1lgC0Th_XL6DhYuVJqJA","password":"W9vpGUOwddz2oE3nwtIk","method":"GET","url":"/item-storage/items"}Each test can be configured using a profile in .conf
ingress-10u-300s-50s {
baseUrl = "http://localhost:19021"
tracingEnabled = true
rampUpUsers = 10
rampUpDuration = 1s
testDuration = 300s
tokenRefreshInterval = 60s
config {
type = "ingress"
includeUserJwt = true
includeSystemJwt = true
requestsFile = "igress-requests.jsonl"
}
}Before each test to warm-up sidecar, a load is given for 1 minute for 10 simultaneous users
folio-module-sidecar
A sidecar image with maximum isolation from other system services, all outside requests are mocked using Wiremock, including:
mgr-applications
mgr-tenant-entitlements
mgr-tenants
AWS SSM service calls
keycloak
It contains a tenant and 100 auth users with 10 random permissions per user based on mod-inventory-storage capabilities.
mock-server
A native image based on Quarkus can respond to any request quickly with a specified delay.
mock-server should not be a bottleneck for sidecar performance testing
Performance test configuration
Property | Value |
|---|---|
Environment | Local (Windows 11, Docker Desktop) Client:
Version: 27.4.0
Context: desktop-linux
Debug Mode: false
Server Version: 27.4.0
Storage Driver: overlayfs
driver-type: io.containerd.snapshotter.v1
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
CDI spec directories:
/etc/cdi
/var/run/cdi
Swarm: inactive
Runtimes: io.containerd.runc.v2 nvidia runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 472731909fa34bd7bc9c087e4c27943f9835f111
runc version: v1.1.13-0-g58aa920
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
Kernel Version: 5.15.153.1-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 30.93GiB
Name: docker-desktop
ID: 626567ea-afe5-4475-889f-b23fe4dcdf36
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Labels:
com.docker.desktop.address=npipe://\\.\pipe\docker_cli
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: false |
sidecar | folio-module-sidecar |
folio-module | A native image based on Quarkus that accepts any request and returns it with all requested data back, including the request body |
keycloak | |
Performance Test duration | 360 sec |
V_USERS | 500 |
RAMP_UP | 320 sec |
Performance test results
50 ms delay / 0.1 vCPU / 1 Event Loop / 5 Thread workers
50 ms delay / 0.25 vCPU / 1 Event Loop / 5 Thread workers
50 ms delay / 0.5 vCPU / 1 Event Loop / 5 Thread workers
50 ms delay / 1 vCPU / 1 Event Loop / 5 Thread workers
200 ms delay / 1 Event Loop / 5 Thread workers
1000 ms delay / 1 Event Loop / 5 Thread workers
50 ms delay / 0.1 vCPU 1 Event Loop / 5 Thread workers / Native Image