[FOLIO-2143] verify performance of RMB26 versions of mod-inv-s, mod-circ-s and mod-users Created: 03/Jul/19 Updated: 03/Jun/20 Resolved: 18/Jul/19 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Task | Priority: | P2 |
| Reporter: | Jakub Skoczen | Assignee: | Hongwei Ji |
| Resolution: | Done | Votes: | 0 |
| Labels: | platform-backlog | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||||||||||||||||||||||||||||||
| Issue links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | CP: sprint 67, CP: sprint 68 | ||||||||||||||||||||||||||||||||||||||||||||||||
| Story Points: | 5 | ||||||||||||||||||||||||||||||||||||||||||||||||
| Development Team: | Core: Platform | ||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Please check if there are any performance regressions or API errors on folio-perf-platform when running with mod-circulation-storage, mod-inventory-storage and mod-users based on RMB 26. Please report here any particular regressions as compared to previous succesfull runs. Also, please review RMB logfiles for those modules to check for any WARN messages about missing indexes, e.g missing FT, GIN and BTREE indexes for specific queries. Please report the occurrences of those warnings and corresponding CQL and SQL queries here. |
| Comments |
| Comment by Jakub Skoczen [ 04/Jul/19 ] |
|
Hongwei Ji can you please start working on this when you get back next week? Thanks! |
| Comment by Hongwei Ji [ 09/Jul/19 ] |
|
Below are the failed perf test cases. 2, /inventory/items?query=(barcode==itemBarcode) 3, /circulation/check-out-by-barcode |
| Comment by Hongwei Ji [ 10/Jul/19 ] |
Missing index messages in the logSee attached files for the details including corresponding CQL and SQL queries mod-inventory-storage:WARNING: Doing LIKE search without GIN index for instance_holding_view.ho_jsonb->>'permanentLocationId' mod-usersWARNING: Doing FT search without FT index proxyfor.jsonb->>'proxyUserId' mod-circulation-storageWARNING: Doing FT search without FT index loan.jsonb->>'userId' |
| Comment by Hongwei Ji [ 10/Jul/19 ] |
|
Another issue: running concurrent checkout requests see errors like below. Possibly due to race condition to get next id in the audit table.
|
| Comment by Hongwei Ji [ 10/Jul/19 ] |
|
Performance is not ideal. I had to reduce the threads from 50 to 20 to avoid timeout errors. See https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-platform/178/performance/. It is build with the RMB 26 enabled mod-inv-s, mod-circ-s, and mod-users. BTW, RMB 26 mod-circ-s is not available on snapshot site yet. |
| Comment by Hongwei Ji [ 11/Jul/19 ] |
|
Updated all three modules to use the latest RMB 26.2.0. |
| Comment by Jakub Skoczen [ 15/Jul/19 ] |
|
Hongwei Ji How is the performance after those updates? Did it help if any of the cases? What slow queries remain? |
| Comment by Jakub Skoczen [ 15/Jul/19 ] |
|
Hongwei Ji It would be great if those WARNINGS above logged also the CQL query (or at least index relation term for this particular node of the query) |
| Comment by Hongwei Ji [ 15/Jul/19 ] |
|
The fix made by Adam Dickmeiss (match FT index with FT search in RMB 26.2.0) made a big difference. The new array syntax search time is down from a few minutes to under 100ms. As of now, there are three slow tests left. BTW Jakub Skoczen, RMB does log the CQL query and SQL right after the WARNINGS message. For example: WARNING: Doing LIKE search without GIN index for instance_holding_view.ho_jsonb->>'permanentLocationId' 10 Jul 2019 00:15:50:478 INFO CQLWrapper [3439061eqId] CQL >>> SQL: (instance_holding_view.ho_jsonb.permanentLocationId==b241764c-1466-4e1d-a028-1a3684a5da87) >>> WHERE lower(f_unaccent(instance_holding_view.ho_jsonb->>'permanentLocationId')) LIKE lower(f_unaccent('b241764c-1466-4e1d-a028-1a3684a5da87')) LIMIT 10 OFFSET 0 |
| Comment by Julian Ladisch [ 16/Jul/19 ] |
|
The warning is from CQL2PG, the "CQL >>> SQL" message is from RMB CQLWrapper. |
| Comment by Hongwei Ji [ 18/Jul/19 ] |
|
Latest perf result looks good. https://github.com/folio-org/mod-inventory-storage/blob/master/ramls/instance.json. The only really slow ones are searching instances by joining other tables. |