| Table of Contents |
|---|
...
- Memory utilization increasing for mod-lists up to 33% due to previous module restarting and everyday cluster shutdown processes - no memory leak is suspected for all of the modules.
- For mod-lists, CPU utilization was up to 384% for the test with 10 concurrent users and R/W split disabled. Average CPU usage did not exceed 13 % for all other modules.
- Approximately DB CPU usage is up to 99.5%.
Recommendations and Jiras
Results
| List name | Search from the number of records | Records found | 1 user | 4 users | 8 users | 10 users | query |
|---|---|---|---|---|---|---|---|
| Holdings all statistical code | 28,016,666 | 500,000 | 00:05:40 | 00:17:41 | 00:18:04 | 00:24:32 | "holdings.statistical_code_names contains all [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)]" |
| Holdings any statistical code | 28,016,666 | 5,347,866 | 02:03:17 | 05:47:55 | 05:01:34 | holdings.statistical_code_names contains any [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)] | |
| Holdings not all statistical code | 28,016,666 | 5,453,772 | 02:24:23 if success
| 05:48:02 | 05:08:24 | holdings.statistical_code_names does not contain all [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)] | |
| Holdings not any statistical code | 28,016,666 | 105,908 | 00:03:21 | 00:18:36 | 00:05:29 | 00:06:38 | holdings.statistical_code_names does not contain any [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)] |
| Items all statistical code | 27,957,839 | 100,000 | 00:06:02 | 00:09:55 |
| 00:07:03 | 00:12:37 | items.statistical_code_names contains all [University of Chicago: books - Books, print (books), University of Chicago: mfilm - Microfilm (mfilm)] | |||||
| Items any statistical code | 27,957,839 | 4,874,982 | 02:36:32 | 02:30:23 | FAILED SQL [select cast(array["item.item".id, 'tenant'] as varchar[]) from... | items.statistical_code_names contains any [University of Chicago: books - Books, print (books), University of Chicago: mfilm - Microfilm (mfilm)] | |
|---|---|---|---|---|---|---|---|
| Items not all statistical code | 27,957,839 | 5,184,433 | 02:01:15 | 06:07:59 | 04:58:38 | items.statistical_code_names does not contain all [University of Chicago: books - Books, print (books), University of Chicago: mfilm - Microfilm (mfilm)] | |
| Items not any statistical code | 27,957,839 | 409,451 | 00:08:39 | 00:17:33 | 00:19:32 | 00:21:39 | items.statistical_code_names does not contain any [University of Chicago: mfilm - Microfilm (mfilm), University of Chicago: books - Books, print (books)] |
| Users all departments | 250,085 | 7,686 | 00:00:13 | 00:00:12 | 00:00:12 | 00:00:13 | users.departments contains all [9af255eb-b87c-4045-9407-d9c324bf0e8c, b1318aeb-0273-42b3-ba57-f1ee5d5cc59e] |
| Users any departments | 250,085 | 29,216 | 00:00:18 | 00:00:19 |
| 00:00:22 | 00:00:22 | users.departments contains any [b1318aeb-0273-42b3-ba57-f1ee5d5cc59e, fa6b405f-311e-493c-be6f-41610552742c, 9af255eb-b87c-4045-9407-d9c324bf0e8c] | |||||||||||||
| Users not all departments | 250,085 | 36,445 | 00:00:24 | 00:00:24 | 00:00:25 | 00:00:27 | users.departments does not contain all [b1318aeb-0273-42b3-ba57-f1ee5d5cc59e, 2eceb91f-6230-4fdb-9d22-8b34216ab800, 9af255eb-b87c-4045-9407-d9c324bf0e8c] | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Users not any departments | 250,085 | 7,229 | 00:00:12 | 00:00:12 | 00:00:12 | 00:00:13 | users.departments does not contain any [b1318aeb-0273-42b3-ba57-f1ee5d5cc59e, fa6b405f-311e-493c-be6f-41610552742c, 9af255eb-b87c-4045-9407-d9c324bf0e8c] | ||||||||
| POL all Acquisition units | any | 0 | not able to find POL
| - | - | - | vendor_organization.accounts[*]->acq_unit_ids contains all [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041] | ||||||||
| POL any Acquisition units | any | 0 | not able to find POL
| - | - | - | vendor_organization.accounts[*]->acq_unit_ids contains any [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041] | ||||||||
| POL not all Acquisition units | any | 0 | not able to find POL
| - | - | - | vendor_organization.accounts[*]->acq_unit_ids does not contain all [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041] | ||||||||
| POL not any Acquisition units | any | 0 | not able to find |
POL
| - | - | - | vendor_organization.accounts[*]->acq_unit_ids does not contain any [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041] | |||||||||||
| Instance all statistical code | 29,105,407 | 20,000 or "FAILED" | Condition with org.folio.list.services.refresh.ListRefreshService was not fulfilled within 1 hours 30 minutes.
| FAILED in 02:35:45 [400 Bad Request] during [GET] to [http://query/b793e898-9a17-4b28-bcd5-38fd89f55ba1] [QueryClient#getQuery(UUID)]: [] | FAILED SQL [select cast(array["instance.inst".id, 'tenant'] as varchar[]) from... | instance.statistical_code_names contains all [ |
|---|
| University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC] | |||||||||||||||||||||||
| Instance any statistical code | 29,105,407 | "FAILED" |
| Not able to create the list
| - | - | instance.statistical_code_names contains any [PTF: PTF1 - PTF1, PTF: PTF3 - PTF3, University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC] | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Instance not all statistical code | 29,105,407 | "FAILED" |
| Not able to create the list
| - | - | instance.statistical_code_names not contain all [PTF: PTF1 - PTF1, PTF: PTF3 - PTF3, University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC] | ||||||||||||||||
| Instance not any statistical code | 29,105,407 | 534,975 or "FAILED" | 02:51:29 | 02:47:14 | FAILED SQL [select cast(array["instance.inst".id, 'tenant'] as varchar[]) from... | instance.statistical_code_names not contain any [PTF: PTF1 - PTF1, PTF: PTF3 - PTF3, University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC] |
Memory Utilization
Memory utilization increasing for mod-lists up to 33% due to previous module restarting and everyday cluster shutdown processes - no memory leak is suspected for all of the modules.
...
For mod-lists, CPU utilization was up to 384% for the test with 10 concurrent users and R/W split disabled. Average CPU usage did not exceed 13 % for all other modules. Each CPU spike corresponds to a separate list refresh.
...
- mod_inventory_storage.item = 22130108
PTF -environment ncp5
- 12 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
2 database instances, one reader, and one writer
Name API Name Memory GIB vCPUs max_connections R6G Extra Large db.r6g.xlarge 32 GiB 4 vCPUs 2731 - MSK ptf-kakfa-3
- 4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
...