Table of Contents outline true
...
- Data Export tests finished successfully on qcon environment using the profiles Default instances export job profile and srs - holdings and items job profile.
- Data Export test were run on College and Central tenants, but results for comparing between environment releases were taken from College tenant.
- Comparing with previous testing results Poppy and Quesnelia releases
- Data Export processed all files including file with 500k records without errors for Quesnelia releases.
- Data Export durations improved - 80% in Average for Quesnelia releases.
- During testing, we noticed spikes in the mod-data-export up to 452% 593% CPU.No memory leak is suspected for Data Export modules.
- For Test №5 Data Export started on College tenant(cs00000int_0001), Central tenant(cs00000int) and Professional tenant(cs00000int_0002) concurrently using the Default instances export job profile, we observed that the CPU usage of the mod-data-export module was initially at 44% before the test began then it spiked to 109% during the test and remained elevated without returning to the initial state.
Test Results
This table contains durations for Data Export with 2 job profiles.
Profile | CSV File | Tenant College (cs00000int_0001) | Central Tenant (cs00000int) | ||
|---|---|---|---|---|---|
| Result | Status | Result | Status | ||
| DE MARC Bib (Default instances export job profile) | 1k.csv | 0:00:02 | COMPLETED | 0:00:05 | COMPLETED |
| 100k.csv | 0:02:39 | COMPLETED | 0:04:24 | COMPLETED | |
| 500k.csv | 0:05:21 | COMPLETED | 0:06:17 | COMPLETED | |
| DE MARC Bib (srs - holdings and items) | 1k.csv | 0:00:05 | COMPLETED | 0:00:05 | COMPLETED |
| 100k.csv | 0:08:15 | COMPLETED | 0:05:58 | COMPLETED | |
| 500k.csv | 0:09:22 | COMPLETED | 0:08:28 | COMPLETED | |
Comparison
This table contains durations
...
for Test №5 Data Export for 3 tenants concurrently.
| Tenant | CSV File | Result | Status |
|---|---|---|---|
Tenant College (cs00000int_0001) | 500k.csv | 0:10:24 | COMPLETED |
Tenant Professional (cs00000int_0002) | 500k.csv | 0:06:47 | COMPLETED |
Central Tenant (cs00000int) | 500k.csv | 0:07:56 | COMPLETED |
Comparison
This table contains durations comparison between Poppy and Quesnelia releases.
| Profile | CSV File | DE Duration/Status Orchid | DE Duration/Status Poppy 1 set | DE Duration/Status Quesnelia Tenant College (cs00000int_0001) | DE Duration, DELTA Poppy/Quesnelia | |||
| Result | Status | Result | Status | Result | Status | hh:mm:ss / percent | ||
| DE MARC Bib (Default instances export job profile) | 1k.csv | 00:00:08 | COMPLETED | 0:00:02 | COMPLETED | -00:00:06 | ||
| 100k.csv | 00:15:36 | COMPLETED | 0:02:39 | COMPLETED | -00:12:57 | |||
| 500k.csv | 00:57:25 | FAIL | 0:05:21 | COMPLETED | -00:52:04 | |||
| DE MARC Bib (srs - holdings and items) | 1k.csv | 00:00:27 | COMPLETED | 00:00:29 | COMPLETED | 0:00:05 | COMPLETED | -00:00:24 |
| 100k.csv | 00:47:51 | COMPLETED | 00:47:23 | COMPLETED | 0:08:15 | COMPLETED | -00:39:08 | |
| 500k.csv | 04:00:26 | COMPLETED | 04:11:09 | FAIL | 0:09:22 | COMPLETED | -04:01:47 | |
...
| Expand | ||
|---|---|---|
|
...
| ||
...
| ||||||||||||||||||||||||||||||||||||||||
Service CPU Utilization
Here we can see that mod-data-export used 336% 593% CPU in spike.
Service Memory Utilization
Here we can see that all modules show a stable trend.
We observed that the CPU usage of the mod-data-export module was initially at 44% before the test began. It spiked to 109% during the test and remained elevated without returning to the initial state.
DB CPU Utilization
DB CPU was 35%50%.
DB Connections
DB connections was 13771368.
DB load
Top SQL-queries
Appendix
Infrastructure
PTF - environment Quesnelia (qcon)
...
- Test 1: Manually tested 1k, 100k and 500k records files Data Export started on College tenant(cs00000int_0001) only using Default instances export job profile.
- Test 2: Manually tested 1k, 100k and 500k records files Data Export started on College tenant(cs00000int_0001) only using srs - holdings and items job profile.
- Test 3: Manually tested 1k, 100k and 500k records files Data Export started on Central tenant(cs00000int) only using Default instances export job profile.
- Test 4: Manually tested 1k, 100k and 500k records files Data Export started on Central tenant(cs00000int) only using srs - holdings and items job profile.
- Test 5: Manually tested 500k records file Data Export started on College tenant(cs00000int_0001), Central tenant(cs00000int) and Professional tenant(cs00000int_0002) concurrently using Default instances export job profile.
To get status and time range for export jobs the query used:
...