PTF-Test Hybrid DBDeployment on multiple tenants[Galileo enviroment].

PTF-Test Hybrid DBDeployment on multiple tenants[Galileo enviroment].

 

Overview

This performance testing was conducted in three distinct phases to validate the stability, scalability, and cost-efficiency. The testing scope progressed from establishing single-tenant benchmarks to validating high-concurrency loads, and finally, evaluating a proposed hybrid database architecture.

The specific objectives for each phase were as follows:

 

Phase 1: Single Tenant Baseline

 

The initial phase established the performance baseline for the singe tenant(cs00000001_0002) operations on the segcon environment (DB r7g.4xlarge).

  • Scope: Execution of standard user workflows—including Authentication, Circulation (50 checkouts/min), Search (200/min), and Inventory browsing—for a single tenant.

  • Goal: Define baseline metrics with a target API p95 response time of <1s and Database CPU utilization under 20% to serve as a reference for scaling tests.

 

Phase 2: High-Concurrency Scalability (30 Tenants)

 

Upon establishing the baseline, the load was scaled to simulate a realistic multi-tenant environment.

  • Scope: Testing on 30 concurrent tenants, simulating typical usage by multiplying the single-tenant baseline workload by 30 tenants.

  • Goal: Validate system stability and resource isolation under load 30 tenants load. The success criteria focused on maintaining API p95 <2s and ensuring Database CPU remains below 50% while keeping average system resource usage (CPU/Memory) within the optimal 40–60% range.

 

Phase 3: Hybrid Architecture Validation (Cost Optimization)

 

The final phase tested an experimental Hybrid Database Deployment to evaluate potential cost reductions without compromising performance.

  • Scope: Integration of the provisioned db.r7g.4xlarge instance with Aurora Serverless V2 (1–4 ACU). The workload involved 30 concurrent tenants with intentionally scheduled "burst traffic" (synchronized large loads/DIs) to trigger serverless auto-scaling.

  • Goal: Prove that the hybrid model can deliver performance parity to an all-provisioned setup while achieving ≥40% cost savings (compared to a db.r7g.8xlarge). Success relies on the Serverless component scaling within <30 seconds with zero connection errors.

 

 

Summary

Test Runs 

Duraion of each test = 30 minutes

The performance validation was executed in three targeted runs:

  • Test_1_STB: Single Tenant Baseline
    On a single tenant(cs00000001_0002) and the provisioned db.r7g.4xlarge DB instance.

    Authentication - 100 Users,
    Circulation = ( Check In -50 operation/min, Check Out -50 operation/min),
    Search operation - 200 searches per minute,
    Inventory- 100 requests per minutes)

  • Test_2 _MTC: Multi-Tenant Concurrency (30 Tenants)
    Scaled the baseline workload to 30 concurrent tenants on the segcon environment. This run the same test as on Test_1_STB, but on 30 tenants with 10 different users on each tenant and the provisioned db.r7g.4xlarge DB instance.
    Authentication - 270 Users,
    Circulation = 32300 CICO Transactions per 30 minmutes test. 36 transaction per minute
    Search operation - 237554 searches request per test. 264 searches request per test
    Inventory- 80 requests per minutes
    Data-import 25K, Create profile - 10 imports in parralel on 10 first tenants

 

  • Test_3_HDv1: Hybrid Deployment 4xl +1 xl Executed the 30-tenant workload on the Hybrid architecture (Provisioned db.r7g.4xlarge + Aurora Serverless db.r7g.1xlarge).
    Authentication - 270 Users,
    Circulation = 32400 CICO Transactions per 30 minmutes test. 37 transaction per minute
    Search operation - 241904.00 searches request per test. 264 searches request per test
    Inventory- 80 requests per minutes
    Data-import 25K, Create profile - 10 imports in parralel on 10 first tenants

Non- functional requirements:

  • API p95 <2s, p99 <5s

  • Average CPU 40-60%

  • Average Memory 40-60%

  • Database CPU <50%

  • All tenant schemas isolated correctly

 

 

Results

 

Test statuses

Test

API
p95 <2s

API
p99 <5s

Modules CPU 40-60%

Average Memory 40-60%

Database
CPU <50%

All tenant schemas isolated correctly

Test

API
p95 <2s

API
p99 <5s

Modules CPU 40-60%

Average Memory 40-60%

Database
CPU <50%

All tenant schemas isolated correctly

Test_1_STB

Passed

Passed

Passed

Passed

Passed (9%)

Passed

Test_2 _MTC

Passed

Passed

Failed

Failed

Failed

Passed

Test_3_HD

 

 

 

 

 

 



 

T2

T3.1

NumberOfSamples

NumberOfSamples

Average

pct90

pct95

NumberOfSamples

Average

pct90

pct95

TC_PARENT_CICO

32288.00

9.29 s

16.55 s

1.02 min

32454.00

7.94 s

4.50 s

23.09 s

TC_PARENT_INVENTORyBrowsing

71375.00

240.47 ms

764.50 ms

1.53 s

71917.00

222.43 ms

208.59 ms

629.80 ms

TC_PARENT_SEARH

8603.00

5.60 s

7.25 s

18.19 s

8647.00

4.92 s

3.72 s

8.03 s





Test_1_STB

transaction

Test_1_STB

NumberOfSamples

Average

pct90

pct95

IS_GET_/inventory/instances

5

540

566

566

IS_GET_/inventory/instances/{id}

2487

48

52

58

IS_GET_/inventory/items

5

604

653

653

IS_GET_/inventory/items-by-holdings-id

2491

12

14

17

IS_GET_/inventory/items/{itemID}

2522

29

33

38

MSF_GET /search/authorities keyword <> random sentence

176

56

64

75

MSF_GET /search/authorities keyword <> randomword

176

61

73

86

MSF_GET /search/authorities keyword = *random sentence*

176

202

217

221

MSF_GET /search/authorities keyword = random sentence

176

16

23

23

MSF_GET /search/authorities keyword = randomword

177

34

49

58

MSF_GET /search/authorities keyword == random sentence

176

16

23

26

MSF_GET /search/authorities keyword == randomword

176

33

50

52

MSF_GET /search/authorities keyword all *randomword*

176

222

247

253

MSF_GET /search/authorities keyword all random sentence

176

17

27

33

MSF_GET /search/authorities keyword all randomword

177

83

270

296

MSF_GET /search/authorities keyword any random sentence

176

46

64

69

MSF_GET /search/authorities keyword any randomword

177

43

84

125

MSF_GET /search/instances: contributors <> 2random words

176

115

119

126

MSF_GET /search/instances: contributors = *randomword

176

107

148

166

MSF_GET /search/instances: contributors = randomword*

176

38

89

93

MSF_GET /search/instances: contributors == 2random words

176

17

25

30

MSF_GET /search/instances: contributors all 2random words

176

17

23

26

MSF_GET /search/instances: contributors all randomword

176

28

38

47

MSF_GET /search/instances: contributors any 2random words

176

30

39

46

MSF_GET /search/instances: keyword all random sentence AND languages == randomlang

352

21

26

34

MSF_GET /search/instances: languages == (lang) AND items.status.name == ("Available" OR "In transit")

352

98

203

209

MSF_GET /search/instances: languages == lang

352

101

209

221

MSF_GET /search/instances: languages == lang AND items.status.name == "Available"

352

97

198

204

MSF_GET /search/instances: subjects <> 1randomword

176

29

39

45

MSF_GET /search/instances: subjects = (randomword OR randomword)

176

37

46

63

MSF_GET /search/instances: subjects = *1randomword

176

195

253

259

MSF_GET /search/instances: subjects = 1randomword*

176

81

130

137

MSF_GET /search/instances: subjects == 1randomword

176

29

38

50

MSF_GET /search/instances: subjects all 1randomword

176

29

38

48

MSF_GET /search/instances: subjects all randomword AND title all randomword

176

33

44

46

MSF_GET /search/instances: subjects all randomword NOT title all randomword

176

33

48

50

MSF_GET /search/instances: subjects all randomword OR title all randomword

176

48

66

77