[Ramsons] List App test report for Contains any, Contains all, Not contains any, Not contains all Operators
Overview
This document contains the results of testing List App refreshing of records in the Ramsons release for Contains any, Contains all, Not contains any, Not contains all Operators. - PERF-892Getting issue details... STATUS
Summary
- Some lists can’t be created at all, and even if they are, there's a high chance they’ll fail when trying to refresh.
- Memory utilization increasing for mod-fgm-manager up to 80% during lists refresh then goes down in few minutes - OOM issue for mod-fqm-manager - MODFQMMGR-501Getting issue details... STATUS but only for creating new lists while the test query is running.
- For mod-lists, CPU utilization was up to 237% for the test with 10 concurrent users and R/W split enabled. Average CPU usage did not exceed 14 % for all other modules.
- Approximately DB CPU usage is up to 98.5%.
Recommendations and Jiras
To have the possibility of refreshing lists for a large number of records, we need to increase QUERY RETENTION DURATION (mod-fqm-manager) and mod-lists.general.refresh-query-timeout-minutes to at least 10 hours.
- MODFQMMGR-506Getting issue details... STATUS
- MODLISTS-149Getting issue details... STATUS
- MODLISTS-150Getting issue details... STATUS
- MODFQMMGR-522Getting issue details... STATUS
- MODFQMMGR-501Getting issue details... STATUS
- MODFQMMGR-505Getting issue details... STATUS
- MODFQMMGR-510Getting issue details... STATUS
- MODLISTS-147Getting issue details... STATUS
- MODFQMMGR-518Getting issue details... STATUS
Results
List name | Search from the number of records | Records found | 1 user | 4 users | 8 users | 10 users | query |
---|---|---|---|---|---|---|---|
Holdings all statistical code | 28,016,666 | 500,000 | 00:05:40 | 00:17:41 | 00:18:04 | 00:24:32 | "holdings.statistical_code_names contains all [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)]" |
Holdings any statistical code | 28,016,666 | 5,347,866 | 02:03:17 | 04:56:23 | 05:01:34 | 09:04:10 | holdings.statistical_code_names contains any [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)] |
Holdings not all statistical code | 28,016,666 | 5,453,772 | 02:24:23 if success
| 05:08:48 | 05:08:24 | 04:31:33 | holdings.statistical_code_names does not contain all [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)] |
Holdings not any statistical code | 28,016,666 | 105,908 | 00:03:21 | 00:18:36 | 00:05:29 | 00:06:38 | holdings.statistical_code_names does not contain any [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)] |
Items all statistical code | 27,957,839 | 100,000 | 00:06:02 | 00:09:55 | 00:07:03 | 00:12:37 | items.statistical_code_names contains all [University of Chicago: books - Books, print (books), University of Chicago: mfilm - Microfilm (mfilm)] |
Items any statistical code | 27,957,839 | 4,874,982 | 02:36:32 | 03:22:25 | FAILED SQL [select cast(array["item.item".id, 'tenant'] as varchar[]) from... | FAILED in 03:42:33 [500 Internal Server Error] during [GET] to [http://query/73497f47-34fd-4eef-93db-4b858b20c6af/sortedIds?offset=2004000&limit=1000] [QueryClient#getSortedIds(UUID,Integer,Integer)]: [{"timestamp":"2024-10-09T13:55:52.355+00:00","status":500,"error":"Internal Server Error","path":"/query/73497f47-34fd-4eef-93db-4b858b20c6af/sortedIds"}] | items.statistical_code_names contains any [University of Chicago: books - Books, print (books), University of Chicago: mfilm - Microfilm (mfilm)] |
Items not all statistical code | 27,957,839 | 5,184,433 | 02:01:15 | 02:16:12 | 04:58:38 | FAILED in 03:43:02 [500 Internal Server Error] during [GET] to [http://query/22bdc696-558e-47f5-9661-f940369359e2/sortedIds?offset=3989000&limit=1000] [QueryClient#getSortedIds(UUID,Integer,Integer)]: [{"timestamp":"2024-10-09T13:56:23.683+00:00","status":500,"error":"Internal Server Error","path":"/query/22bdc696-558e-47f5-9661-f940369359e2/sortedIds"}] | items.statistical_code_names does not contain all [University of Chicago: books - Books, print (books), University of Chicago: mfilm - Microfilm (mfilm)] |
Items not any statistical code | 27,957,839 | 409,451 | 00:08:39 | 00:17:33 | 00:19:32 | 00:21:39 | items.statistical_code_names does not contain any [University of Chicago: mfilm - Microfilm (mfilm), University of Chicago: books - Books, print (books)] |
Users all departments | 250,085 | 7,686 | 00:00:13 | 00:00:12 | 00:00:12 | 00:00:13 | users.departments contains all [9af255eb-b87c-4045-9407-d9c324bf0e8c, b1318aeb-0273-42b3-ba57-f1ee5d5cc59e] |
Users any departments | 250,085 | 29,216 | 00:00:18 | 00:00:19 | 00:00:22 | 00:00:22 | users.departments contains any [b1318aeb-0273-42b3-ba57-f1ee5d5cc59e, fa6b405f-311e-493c-be6f-41610552742c, 9af255eb-b87c-4045-9407-d9c324bf0e8c] |
Users not all departments | 250,085 | 36,445 | 00:00:24 | 00:00:24 | 00:00:25 | 00:00:27 | users.departments does not contain all [b1318aeb-0273-42b3-ba57-f1ee5d5cc59e, 2eceb91f-6230-4fdb-9d22-8b34216ab800, 9af255eb-b87c-4045-9407-d9c324bf0e8c] |
Users not any departments | 250,085 | 7,229 | 00:00:12 | 00:00:12 | 00:00:12 | 00:00:13 | users.departments does not contain any [b1318aeb-0273-42b3-ba57-f1ee5d5cc59e, fa6b405f-311e-493c-be6f-41610552742c, 9af255eb-b87c-4045-9407-d9c324bf0e8c] |
POL all Acquisition units | any | 0 | not able to find POL - MODFQMMGR-506Getting issue details... STATUS | - | - | - | vendor_organization.accounts[*]->acq_unit_ids contains all [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041] |
POL any Acquisition units | any | 0 | not able to find POL - MODFQMMGR-506Getting issue details... STATUS | - | - | - | vendor_organization.accounts[*]->acq_unit_ids contains any [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041] |
POL not all Acquisition units | any | 0 | not able to find POL - MODFQMMGR-506Getting issue details... STATUS | - | - | - | vendor_organization.accounts[*]->acq_unit_ids does not contain all [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041] |
POL not any Acquisition units | any | 0 | not able to find POL - MODFQMMGR-506Getting issue details... STATUS | - | - | - | vendor_organization.accounts[*]->acq_unit_ids does not contain any [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041] |
Instance all statistical code | 29,105,407 | 20,000 or "FAILED" | Condition with org.folio.list.services.refresh.ListRefreshService was not fulfilled within 1 hours 30 minutes. - MODLISTS-147Getting issue details... STATUS | FAILED in 02:35:45 [400 Bad Request] during [GET] to [http://query/b793e898-9a17-4b28-bcd5-38fd89f55ba1] [QueryClient#getQuery(UUID)]: [] | FAILED SQL [select cast(array["instance.inst".id, 'tenant'] as varchar[]) from... | 03:56:59 | instance.statistical_code_names contains all [University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC] |
Instance any statistical code | 29,105,407 | "FAILED" | - MODFQMMGR-505Getting issue details... STATUS | Not able to create the list | - | - | instance.statistical_code_names contains any [PTF: PTF1 - PTF1, PTF: PTF3 - PTF3, University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC] |
Instance not all statistical code | 29,105,407 | "FAILED" | - MODFQMMGR-505Getting issue details... STATUS | Not able to create the list - MODFQMMGR-501Getting issue details... STATUS | - | - | instance.statistical_code_names not contain all [PTF: PTF1 - PTF1, PTF: PTF3 - PTF3, University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC] |
Instance not any statistical code | 29,105,407 | 534,975 or "FAILED" | 02:51:29 | 02:47:14 | FAILED SQL [select cast(array["instance.inst".id, 'tenant'] as varchar[]) from... | 04:17:27 | instance.statistical_code_names not contain any [PTF: PTF1 - PTF1, PTF: PTF3 - PTF3, University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC] |
Instance CPU Utilization
Service Memory Utilization
mod-lists & mod-fqm-manager
Service CPU Utilization
Service CPU Utilization without mod-fqm-manager