...
Ticket:
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
Recommendations & Jiras
Add more CPU for
Summary
The total duration:
...
Data export | Instances | Holdings | Authority | ||||||
500, sec | 1000 , sec | 2000 , sec | 500 , sec | 1000 , sec | 2000, sec | 500 , sec | 1000 , sec | 2000, sec | |
Average duration | 83 | 426 | 1184 | 10 | 87 | 405 | 5,9 | 6,2 | 10,6 |
Minimal duration | 12 | 32 | 1372127 | 5 | 16 | 32 | 5,8 | 5,7 | 1,15 |
Maximal duration | 131 | 571 | 1271372 | 29 | 127 | 504 | 6,6 | 11,7 | 15,8 |
75t percentile | 114 | 539 | 1350 | 12 | 113 | 490 | 5,8 | 6,1 | 14,4 |
90t percentile | 126 | 553 | 1362 | 19 | 119 | 500 | 5,9 | 6,5 | 15,5 |
...
The duration of the longest data-export for holdings is 7 hours and 40 minutes, and 12 hours and 30 minutes for instances on 42nd tenants.
Recommendations & Jiras
Restart the tests DE_instances T2(Max) and DE_holdings T4(Max) with different POOL_SIZE, and BATCH_SIZE parameters to check Data-export duration improvement. [MDEXP-688] Make configurable params for Data-Export(POOL_SIZE, BATCH_SIZE) - FOLIO Jira (atlassian.net)
Restart Data-Import(Authority) test on mcpt to check how Kafka topics affect performance(found in the scope of data-preparation) [MODELINKS-213] Data-import process creating incorrect topics in Kafka - FOLIO Jira (atlassian.net)
...
Transaction name | DE 500, sec | DE 1K, sec | DE 2K, sec | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0001 | 12 | 32 | 127 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0002 | 16 | 51 | 207 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0003 | 23 | 73 | 330 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0004 | 22 | 100 | 494 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0005 | 27 | 122 | 606 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0006 | 29 | 131 | 704 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0007 | 28 | 154 | 782 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0008 | 34 | 170 | 875 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0009 | 39 | 191 | 936 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0010 | 38 | 216 | 965 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0011 | 40 | 228 | 1022 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0012 | 45 | 261 | 1054 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0013 | 45 | 277 | 1082 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0014 | 45 | 293 | 1108 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0015 | 52 | 308 | 1163 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0016 | 51 | 331 | 1161 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0017 | 56 | 372 | 1183 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0018 | 56 | 374 | 1209 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0019 | 63 | 399 | 1232 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0020 | 63 | 419 | 1254 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0021 | 68 | 432 | 1275 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0022 | 69 | 457 | 1271 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0023 | 72 | 479 | 1290 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0024 | 74 | 475 | 1306 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0025 | 74 | 489 | 1300 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0026 | 75 | 506 | 1319 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0027 | 85 | 503 | 1336 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0028 | 80 | 502 | 1331 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0029 | 91 | 535 | 1347 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0030 | 87 | 532 | 1344 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0031 | 87 | 530 | 1357 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0032 | 93 | 539 | 1349 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0033 | 97 | 535 | 1363 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0034 | 97 | 547 | 1357 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0035 | 97 | 560 | 1373 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0036 | 107 | 554 | 1349 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0037 | 102 | 557 | 1359 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0038 | 103 | 563 | 1370 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0039 | 109 | 571 | 1363 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0040 | 104 | 548 | 1364 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0041 | 109 | 552 | 1365 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0042 | 115 | 553 | 1362 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0043 | 119 | 497 | 1363 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0044 | 126 | 555 | 1363 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0045 | 121 | 550 | 1356 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0046 | 126 | 553 | 1352 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0047 | 120 | 544 | 1352 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0048 | 131 | 551 | 1347 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0049 | 126 | 546 | 1340 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0050 | 130 | 538 | 1336 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0051 | 127 | 533 | 1341 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0052 | 128 | 533 | 1337 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0053 | 127 | 529 | 1334 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0054 | 123 | 527 | 1320 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0055 | 122 | 516 | 1320 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0056 | 119 | 516 | 1316 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0057 | 114 | 508 | 1315 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0058 | 110 | 503 | 1306 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0059 | 108 | 501 | 1302 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0060 | 101 | 493 | 1298 | |||
DE_Exporting MARC Bib records workflow Transaction cs00000001_0061 | 98 | 487 | 1289 DE_Exporting MARC Bib records workflow Transaction cs00000001_0062 | 16 | 31 | 36 |
...
DE_instances T2(Max). The table below contains durations for Data-export for jobs with “Default instances export profile” on 61 tenants with 1 set of data(all of the instances on each tenant).
...
Tenant | Number of exported instances | Duration | Shift of the start time | |
cs00000001_0001 | 65359 | 08:34:11.924 | 0:00:00 | |
cs00000001_0002 | 22370 | 02:01:17.113 | 00:00:04.576 | |
cs00000001_0003 | 136291 | 15:38:26.969 | 00:20:55.783 | |
cs00000001_0004 | 77785 | 10:42:11.942 | 00:02:52.106 | |
cs00000001_0005 | 110093 | 13:57:20.399 | 00:12:33.693 | |
cs00000001_0006 | 293580 | 17:24:21.58 | 02:58:03.642 | |
cs00000001_0007 | 279493 | 17:15:17.802 | 02:55:41.421 | |
cs00000001_0008 | 160024 | 16:21:18.866 | 00:56:26.929 | |
cs00000001_0009 | 110918 | 14:01:42.358 | 00:25:51.269 | |
cs00000001_0010 | 83254 | 11:36:56.235 | 00:14:09.228 | |
cs00000001_0011 | 94171 | 12:43:36.487 | 00:20:24.292 | |
cs00000001_0012 | 227489 | 16:52:02.623 | 02:24:44.754 | |
cs00000001_0013 | 19131 | 01:58:23.547 | 00:01:37.648 | |
cs00000001_0014 | 20746 | 02:16:27.599 | 00:01:52.916 | |
cs00000001_0015 | 26899 | 03:24:08.545 | 00:03:01.837 | |
cs00000001_0016 | 153333 | 15:46:35.492 | 01:22:21.469 | |
cs00000001_0017 | 16492 | 01:41:43.743 | 00:02:40.122 | |
cs00000001_0018 | 22263 | 02:43:54.315 | 00:04:28.863 | |
cs00000001_0019 | 5312 | 00:14:46.338 | 00:02:05.943 | |
cs00000001_0020 | 44259 | 06:30:26.536 | 00:12:04.781 | |
cs00000001_0021 | 83988 | 11:46:23.477 | 00:33:39.156 | |
cs00000001_0022 | 86900 | 12:03:06.865 | 00:36:22.655 | |
cs00000001_0023 | 91517 | 12:27:03.606 | 00:40:56.469 | |
cs00000001_0024 | 385123 | 15:35:51.105 | 05:43:10.805 | |
cs00000001_0025 | 84215 | 11:47:31.81 | 00:36:33.173 | |
cs00000001_0026 | 153912 | 15:32:54.171 | 01:42:36.675 | |
cs00000001_0027 | 48176 | 07:11:08.729 | 00:16:44.682 | |
cs00000001_0028 | 64556 | 09:30:37.567 | 00:25:12.347 | |
cs00000001_0029 | 5140 | 00:16:41.394 | 00:03:41.85 | |
cs00000001_0030 | 97589 | 12:52:29.07 | 00:50:52.289 | |
cs00000001_0031 | 71895 | 10:26:16.656 | 00:31:10.105 | |
cs00000001_0032 | 80074 | 11:22:00.789 | 00:37:31.626 | |
cs00000001_0033 | 45886 | 06:52:06.447 | 00:17:36.951 | |
cs00000001_0034 | 18240 | 02:14:18.252 | 00:07:47.49 | |
cs00000001_0035 | 135704 | 14:55:28.661 | 01:29:32.543 | |
cs00000001_0036 | 114170 | 13:56:14.427 | 01:08:43.67 | |
cs00000001_0037 | 24226 | 03:18:25.02 | 00:09:57.63 | |
cs00000001_0038 | 225563 | 16:13:32.965 | 03:06:16.602 | |
cs00000001_0039 | 128523 | 14:37:45.055 | 01:24:25.093 | |
cs00000001_0040 | 111992 | 13:47:45.898 | 01:08:36.465 | |
cs00000001_0041 | 61091 | 09:03:29.08 | 00:27:37.435 | |
cs00000001_0042 | 634675 | 12:24:24.995 | 09:48:38.838 | |
cs00000001_0043 | 2831 | 00:02:12.583 | 00:04:22.427 | |
cs00000001_0044 | 70871 | 10:19:26.438 | 00:35:14.19 | |
cs00000001_0045 | 78320 | 11:10:12.277 | 00:41:17.857 | |
cs00000001_0046 | 109677 | 13:38:14.993 | 01:08:58.069 | |
cs00000001_0047 | 48531 | 07:19:11.872 | 00:22:04.318 | |
cs00000001_0048 | 53346 | 08:01:00.977 | 00:24:42.919 | |
cs00000001_0049 | 6952 | 00:31:27.638 | 00:06:36.51 | |
cs00000001_0050 | 190736 | 15:54:19.153 | 02:34:40.768 | |
cs00000001_0051 | 351325 | 15:41:17.61 | 05:23:26.089 | |
cs00000001_0052 | 137940 | 14:54:13.405 | 01:39:56.924 | |
cs00000001_0053 | 61834 | 09:11:33.182 | 00:31:14.104 | |
cs00000001_0054 | 16297 | 02:00:46.997 | 00:09:40.899 | |
cs00000001_0055 | 176009 | 15:42:52.69 | 02:21:13.276 | |
cs00000001_0056 | 160976 | 15:28:32.362 | 02:06:05.583 | |
cs00000001_0057 | 22596 | 03:06:34.087 | 00:12:13.641 | |
cs00000001_0058 | 123819 | 14:19:45.503 | 01:27:29.477 | |
cs00000001_0059 | 37715 | 05:38:43.984 | 00:18:51.151 | |
cs00000001_0060 | 38395 | 05:45:32.811 | 00:19:17.503 | |
cs00000001_0061 | 87117 | 12:00:00.92 | 00:53:34.119 cs00000001_0062 | 1 |
DE_holdings T3(500-1K-2k). The table below contains durations for Data-export for jobs with “Default holdings export profile” on 61 tenants with 1 set of data(all of the instances on each tenant).
...
Tenant | Number of exported instances | Duration | Shift of the start time | |
cs00000001_0001 | 66697 | 04:47:40.032 | 00:07:07.001 | |
cs00000001_0002 | 22477 | 01:46:16.222 | 00:00:00.895 | |
cs00000001_0003 | 136679 | 07:11:34.247 | 00:07:06.999 | |
cs00000001_0004 | 78347 | 05:21:32.462 | 00:06:56.961 | |
cs00000001_0005 | 110669 | 06:32:25.987 | 00:07:06.999 | |
cs00000001_0006 | 354865 | 09:08:03.315 | 00:07:14.627 | |
cs00000001_0007 | 281879 | 08:44:44.905 | 00:07:14.627 | |
cs00000001_0008 | 183455 | 07:56:22.925 | 00:07:07.001 | |
cs00000001_0009 | 113451 | 06:37:35.114 | 00:06:58.234 | |
cs00000001_0010 | 85930 | 05:41:41.91 | 00:07:06.989 | |
cs00000001_0011 | 96350 | 06:04:40.493 | 00:07:07.001 | |
cs00000001_0012 | 234350 | 08:25:25.455 | 00:07:14.626 | |
cs00000001_0013 | 20385 | 01:35:23.336 | 00:00:01.088 | |
cs00000001_0014 | 20971 | 01:47:21.388 | 00:01:56.992 | |
cs00000001_0015 | 32559 | 02:42:56.589 | 00:03:28.627 | |
cs00000001_0016 | 153429 | 07:30:21.621 | 00:07:06.999 | |
cs00000001_0017 | 17052 | 01:17:15.902 | 00:00:00.652 | |
cs00000001_0018 | 22781 | 01:56:54.12 | 00:02:09.495 | |
cs00000001_0019 | 5959 | 00:11:46.132 | 00:00:01.087 | |
cs00000001_0020 | 45940 | 03:37:46.61 | 00:05:30.563 | |
cs00000001_0021 | 90206 | 05:52:00.395 | 00:06:58.233 | |
cs00000001_0022 | 87358 | 05:45:21.752 | 00:07:07.001 | |
cs00000001_0023 | 92097 | 05:55:59.4 | 00:07:07.001 | |
cs00000001_0024 | 393084 | 09:14:09.688 | 00:07:14.627 | |
cs00000001_0025 | 85689 | 05:41:10.838 | 00:06:58.234 | |
cs00000001_0026 | 155147 | 07:32:07.59 | 00:07:07.002 | |
cs00000001_0027 | 49392 | 03:50:53.104 | 00:06:08.168 | |
cs00000001_0028 | 66640 | 04:47:37.357 | 00:06:58.234 | |
cs00000001_0029 | 7958 | 00:25:20.682 | 00:00:00.653 | |
cs00000001_0030 | 102042 | 06:16:01.035 | 00:06:58.234 | |
cs00000001_0031 | 76243 | 05:15:13.512 | 00:06:58.234 | |
cs00000001_0032 | 86817 | 05:43:58.799 | 00:07:07.001 | |
cs00000001_0033 | 47651 | 03:44:36.908 | 00:05:48.33 | |
cs00000001_0034 | 19994 | 01:32:52.325 | 00:00:00.737 | |
cs00000001_0035 | 136563 | 07:11:27.804 | 00:07:07.002 | |
cs00000001_0036 | 115912 | 06:41:36.881 | 00:07:07.001 | |
cs00000001_0037 | 25969 | 02:13:07.978 | 00:02:36.213 | |
cs00000001_0038 | 244420 | 08:30:13.075 | 00:07:14.627 | |
cs00000001_0039 | 130262 | 07:03:14.774 | 00:07:07.001 | |
cs00000001_0040 | 116735 | 06:42:50.786 | 00:07:06.988 | |
cs00000001_0041 | 61472 | 04:31:30.09 | 00:07:07.001 | |
cs00000001_0042 | 650197 | 09:32:34.768 | 00:07:14.627 | |
cs00000001_0043 | 830 | 00:00:50.612 | 00:00:00.814 | |
cs00000001_0044 | 71477 | 05:01:47.812 | 00:06:58.234 | |
cs00000001_0045 | 92823 | 05:57:44.596 | 00:06:54.272 | |
cs00000001_0046 | 126251 | 06:57:56.793 | 00:06:58.233 | |
cs00000001_0047 | 49518 | 03:51:26.841 | 00:06:08.004 | |
cs00000001_0048 | 53596 | 04:05:29.871 | 00:07:06.989 | |
cs00000001_0049 | 7458 | 00:26:40.751 | 00:00:47.596 | |
cs00000001_0050 | 208836 | 08:11:54.75 | 0:07:07 | |
cs00000001_0051 | 363363 | 09:10:02.246 | 0:07:07 | |
cs00000001_0052 | 148889 | 07:25:35.755 | 00:07:07.001 | |
cs00000001_0053 | 62321 | 04:34:11.182 | 0:07:07 | |
cs00000001_0054 | 17155 | 01:26:12.309 | 00:01:39.154 | |
cs00000001_0055 | 182363 | 07:55:36.678 | 0:07:07 | |
cs00000001_0056 | 167863 | 07:43:34.721 | 00:07:14.627 | |
cs00000001_0057 | 23743 | 02:01:53.176 | 00:02:20.403 | |
cs00000001_0058 | 124395 | 06:55:00.179 | 00:07:06.988 | |
cs00000001_0059 | 38685 | 03:08:57.995 | 00:04:22.604 | |
cs00000001_0060 | 38775 | 03:09:17.833 | 00:04:27.063 | |
cs00000001_0061 | 88457 | 05:47:58.582 | 00:07:07.002 | cs00000001_0062 |
DE_authority T5(500-1K-2k). The table below contains durations for Data-export for jobs with “Default authority export profile” on 61 tenants with 1 set of data(all of the instances on each tenant).
...
| Expand | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Methodology/Approach
All tests were run using the JMeter script. Data has been pre-generated for all tenants
Infrastructure
PTF -environment mcpt
...