...
DE_instances T2(Max) and DE_holdings T4(Max) Both tests show a delay in the start time, but for DE_holdings T4(Max) maximal delay is 8 minutes for the files with the largest number of id (Tenant 42, export 640K records). The delay for the instances data-export for the same tenant is 9 hours and 50 minutes.
The duration of the longest data-export for holdings is 7 hours and 40 minutes, and 12 hours and 30 minutes for instances on 42nd tenants.
Recommendations & Jiras
Restart the tests DE_instances T2(Max) and DE_holdings T4(Max) with different POOL_SIZE, and BATCH_SIZE parameters to check Data-export duration improvement. [MDEXP-688] Make configurable params for Data-Export(POOL_SIZE, BATCH_SIZE) - FOLIO Jira (atlassian.net)
Restart Data-Import(Authority) test on mcpt to check how Kafka topics affect performance(found in the scope of data-preparation) [MODELINKS-213] Data-import process creating incorrect topics in Kafka - FOLIO Jira (atlassian.net)
...
Transaction name | DE 500, sec | DE 1K, sec | DE 2K, sec |
DE_Exporting AuthorityTransaction cs00000001_0001 | 6796 | 7119 | 11516 |
DE_Exporting AuthorityTransaction cs00000001_0002 | 5962 | 5779 | 16226 |
DE_Exporting AuthorityTransaction cs00000001_0003 | 5836 | 6311 | 21968 |
DE_Exporting AuthorityTransaction cs00000001_0004 | 5865 | 5750 | 26586 |
DE_Exporting AuthorityTransaction cs00000001_0005 | 5886 | 5745 | 28384 |
DE_Exporting AuthorityTransaction cs00000001_0006 | 5877 | 5753 | 28288 |
DE_Exporting AuthorityTransaction cs00000001_0007 | 5884 | 5737 | 33628 |
DE_Exporting AuthorityTransaction cs00000001_0008 | 5868 | 5730 | 39163 |
DE_Exporting AuthorityTransaction cs00000001_0009 | 5902 | 5751 | 44187 |
DE_Exporting AuthorityTransaction cs00000001_0010 | 5956 | 5869 | 44396 |
DE_Exporting AuthorityTransaction cs00000001_0011 | 5893 | 5787 | 49254 |
DE_Exporting AuthorityTransaction cs00000001_0012 | 5902 | 5858 | 55799 |
DE_Exporting AuthorityTransaction cs00000001_0013 | 5881 | 5756 | 54919 |
DE_Exporting AuthorityTransaction cs00000001_0014 | 5891 | 5739 | 61341 |
DE_Exporting AuthorityTransaction cs00000001_0015 | 5866 | 5755 | 66577 |
DE_Exporting AuthorityTransaction cs00000001_0016 | 5902 | 5732 | 66923 |
DE_Exporting AuthorityTransaction cs00000001_0017 | 5863 | 5928 | 70759 |
DE_Exporting AuthorityTransaction cs00000001_0018 | 5854 | 6484 | 77038 |
DE_Exporting AuthorityTransaction cs00000001_0019 | 5876 | 6102 | 83432 |
DE_Exporting AuthorityTransaction cs00000001_0020 | 5868 | 6468 | 82352 |
DE_Exporting AuthorityTransaction cs00000001_0021 | 5879 | 6918 | 90676 |
DE_Exporting AuthorityTransaction cs00000001_0022 | 5857 | 6046 | 94664 |
DE_Exporting AuthorityTransaction cs00000001_0023 | 5943 | 6604 | 85713 |
DE_Exporting AuthorityTransaction cs00000001_0024 | 5860 | 6171 | 105028 |
DE_Exporting AuthorityTransaction cs00000001_0025 | 5899 | 6299 | 105846 |
DE_Exporting AuthorityTransaction cs00000001_0026 | 5868 | 6131 | 115376 |
DE_Exporting AuthorityTransaction cs00000001_0027 | 5874 | 5944 | 111442 |
DE_Exporting AuthorityTransaction cs00000001_0028 | 5879 | 6011 | 117511 |
DE_Exporting AuthorityTransaction cs00000001_0029 | 5878 | 5792 | 127511 |
DE_Exporting AuthorityTransaction cs00000001_0030 | 5863 | 5840 | 128740 |
DE_Exporting AuthorityTransaction cs00000001_0031 | 5879 | 5820 | 128969 |
DE_Exporting AuthorityTransaction cs00000001_0032 | 5866 | 6501 | 132437 |
DE_Exporting AuthorityTransaction cs00000001_0033 | 5867 | 11505 | 138441 |
DE_Exporting AuthorityTransaction cs00000001_0034 | 5859 | 11731 | 144630 |
DE_Exporting AuthorityTransaction cs00000001_0035 | 5877 | 7080 | 144332 |
DE_Exporting AuthorityTransaction cs00000001_0036 | 5849 | 10923 | 145234 |
DE_Exporting AuthorityTransaction cs00000001_0037 | 5897 | 6041 | 151330 |
DE_Exporting AuthorityTransaction cs00000001_0038 | 5894 | 6145 | 150469 |
DE_Exporting AuthorityTransaction cs00000001_0039 | 5877 | 5739 | 155768 |
DE_Exporting AuthorityTransaction cs00000001_0040 | 5904 | 5744 | 156220 |
DE_Exporting AuthorityTransaction cs00000001_0041 | 5874 | 6202 | 156766 |
DE_Exporting AuthorityTransaction cs00000001_0042 | 5879 | 5816 | 157841 |
DE_Exporting AuthorityTransaction cs00000001_0043 | 6608 | 5880 | 158721 |
DE_Exporting AuthorityTransaction cs00000001_0044 | 5866 | 5771 | 156396 |
DE_Exporting AuthorityTransaction cs00000001_0045 | 5892 | 5854 | 157477 |
DE_Exporting AuthorityTransaction cs00000001_0046 | 5876 | 5738 | 152934 |
DE_Exporting AuthorityTransaction cs00000001_0047 | 5902 | 5750 | 153392 |
DE_Exporting AuthorityTransaction cs00000001_0048 | 5838 | 5745 | 149484 |
DE_Exporting AuthorityTransaction cs00000001_0049 | 5872 | 5860 | 149659 |
DE_Exporting AuthorityTransaction cs00000001_0050 | 5852 | 5795 | 149111 |
DE_Exporting AuthorityTransaction cs00000001_0051 | 5866 | 6178 | 143901 |
DE_Exporting AuthorityTransaction cs00000001_0052 | 5916 | 5882 | 144713 |
DE_Exporting AuthorityTransaction cs00000001_0053 | 5858 | 5747 | 139844 |
DE_Exporting AuthorityTransaction cs00000001_0054 | 5840 | 6032 | 135910 |
DE_Exporting AuthorityTransaction cs00000001_0055 | 5917 | 5903 | 136991 |
DE_Exporting AuthorityTransaction cs00000001_0056 | 5893 | 6455 | 132152 |
DE_Exporting AuthorityTransaction cs00000001_0057 | 5843 | 5769 | 126592 |
DE_Exporting AuthorityTransaction cs00000001_0058 | 5864 | 5814 | 125191 |
DE_Exporting AuthorityTransaction cs00000001_0059 | 5860 | 6123 | 120351 |
DE_Exporting AuthorityTransaction cs00000001_0060 | 5840 | 5844 | 118520 |
DE_Exporting AuthorityTransaction cs00000001_0061 | 5840 | 5890 | 114587 |
...
Service CPU Utilization
DE_instances T1(500-1K-2k). Service CPU usage during the test was quite low, only the mod-data-export-b okapi-b mod-permissions-b services had a slight increase up to 13%.
...