[FOLIO-2087] test modinvstor-285 on the platform-perf-test environment Created: 07/Jun/19  Updated: 03/Jun/20  Resolved: 12/Jun/19

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P2
Reporter: Jakub Skoczen Assignee: Hongwei Ji
Resolution: Done Votes: 0
Labels: platform-backlog
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Cloners
is cloned by FOLIO-2098 test modinvstor-285 on the platform-p... Closed
Relates
relates to RMB-399 CQL2PG: Use "caseSensitive" "removeAc... Open
relates to MODINVSTOR-285 upgrade to RMB 26 and resolve related... Closed
Sprint: CP: sprint 65
Story Points: 1
Development Team: Core: Platform

 Comments   
Comment by Hongwei Ji [ 07/Jun/19 ]

Working with John Malconian to get the PR build pushed to Docker hub and also upload the matching Module Descriptor to folio-registry.

Comment by Hongwei Ji [ 07/Jun/19 ]

Per John Malconian, the PR build id is mod-inventory-storage-15.5.0-SNAPSHOT.1. I just kicked off https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-platform/132/. Let's see how it goes.

Comment by Hongwei Ji [ 07/Jun/19 ]

Got following error. Did we change "_id" to "id"?

copy supertenant_mod_inventory_storage.service_point(_id, jsonb) FROM 'datasets/common/servicePoints.tsv' DELIMITER E'\t'
ERROR:  column "_id" of relation "service_point" does not exist
Comment by Hongwei Ji [ 07/Jun/19 ]

Changed the data to use id over _id, kicked off https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-platform/133/. There are some test failures. Need more investigation. Failed tests can be seen from https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-platform/133/performance/

Comment by Hongwei Ji [ 10/Jun/19 ]

Kicked off another build https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-platform/141/performance/. Only two tests are failing now

Perf - GET inventory_instances?limit=30&query=(identifiers adj "\"value\"_ \"{idVal}*\"") sortby title (100% errors)
Perf - GET inventory_items?query=(barcode=={itemBarcode}) (16% errors, most likely the response takes too long)
Comment by Jakub Skoczen [ 11/Jun/19 ]

John Malconian Hongwei Ji Guys, how do we know which perf builds are based on the PR build of mod-inventory-storage?

Comment by Hongwei Ji [ 11/Jun/19 ]

You can look at the build parameters. When kick off the build, there is a parameter to specify which version of modules to use. Right now only #141 (and #133) but I will focus #141.

Comment by Hongwei Ji [ 11/Jun/19 ]

About the two perf test failures. Here are the reasons.

/inventory/instances?limit=30&query=(identifiers adj "\"value\": \"5335676698*\"") sortby title
CQL: only right truncation supported. Seems * is not the last character of "\"value\": \"5335676698*\"".

CQL >>> SQL: (barcode==1263493901) >>> WHERE lower(f_unaccent(item.jsonb->>'barcode')) LIKE lower(f_unaccent('1263493901'))
It causes sequential scan because the GIN index on barcode is case/accent sensitive. We can adjust barcode GIN index (easy) or tweak the CQL->SQL logic (riskier)

Overall I feel it is a bit rush to push RMB25 into Q2.2.

Comment by Julian Ladisch [ 11/Jun/19 ]

Did we change "_id" to "id"?

Yes: https://github.com/folio-org/raml-module-builder/blob/a950264/doc/upgrading.md#version-25

For the first failure: Using "\"value\": \"5335676698*" (no " after the *) in the perf tests should be fine.

The second failure is a bug in cql2pg. https://github.com/folio-org/mod-inventory-storage/blob/v15.5.1/src/main/resources/templates/db_scripts/schema.json has these entries since March 7, 2019 – https://github.com/folio-org/mod-inventory-storage/pull/254/files :

"index": [
        {
          "fieldName": "barcode",
          "tOps": "ADD",
          "caseSensitive": true,
          "removeAccents": false
        }
      ],
      "ginIndex": [
        {
          "fieldName": "barcode",
          "tOps": "ADD",
          "caseSensitive": true,
          "removeAccents": false
        }
     ]

Therefore CQL2PG should use neither lower() nor f_unaccent() to allow for umlaut barcodes.

Comment by Hongwei Ji [ 11/Jun/19 ]

For the first failure: Using "\"value\": \"5335676698*" (no " after the *) in the perf tests should be fine.

We can change the perf test easily but it will not match the actual APi calls generated by the application.
For the second failure, if the CQL does not specify the case/accent, the generated SQL will lower case and remove accent by default, unless we change the rule to be "if operator is ==, we will NOT modify accent/case by default". I do not think CQL2PG should look at the db index and tweak the SQL in this case.

Comment by Julian Ladisch [ 11/Jun/19 ]

Regarding lower() f_unaccent() I've create https://folio-org.atlassian.net/browse/RMB-399 CQL2PG: Use "caseSensitive" "removeAccents" index options from schema.json

Comment by Julian Ladisch [ 11/Jun/19 ]

The front-end needs to change the "\"value\": \"5335676698*\"" query because we only support right truncation. We dropped supporting * inside a string in https://folio-org.atlassian.net/browse/CQLPG-81 Note 5.

Comment by Jakub Skoczen [ 12/Jun/19 ]

Let's close this but create another ticket in the backlog for trying a PR build of modinvstor when the issues are fixed.

Comment by Hongwei Ji [ 12/Jun/19 ]

Created FOLIO-2098 Closed to retest once all issues are resolved.

Generated at Thu Feb 08 23:18:06 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.