[MSEARCH-74] Outage due to permissions error with mod-search Created: 08/Mar/21 Updated: 09/Mar/21 Resolved: 09/Mar/21 |
|
| Status: | Closed |
| Project: | mod-search |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Task | Priority: | P2 |
| Reporter: | Marc Johnson | Assignee: | Hongwei Ji |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | environments, outage | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||
| Sprint: | |||||||||
| Development Team: | Falcon | ||||||||
| Description |
|
The folio-snapshot build fails with the following error:
fatal: [10.36.1.88]: FAILED! => {"changed": false, "connection": "close", "content": "POST request for mod-search-1.0.0-SNAPSHOT.33 /_/tenant failed with 500: {\"errors\":[{\"message\":\"[422 Unprocessable Entity] during [POST] to [http://perms/users] [PermissionsClient#assignPermissionsToUser(Permissions)]: [{\\n \\\"errors\\\" : [ {\\n \\\"message\\\" : \\\"Unable to update derived fields: Attempting to add non-existent permissions inventory-storage.inventory-view.instances.collection.get to permission user with id 14a02f1f-9fbc-4ccc-9523-4cd9f474abf3\\\",\\n \\\"type\\\" : \\\"1\\\",\\n \\\"code\\\" : \\\"-1\\\",\\n \\\"parameters\\\" : [ {\\n \\\"key\\\" : \\\"id\\\",\\n \\\"value\\\" : \\\"14a02f1f-9fbc-4ccc-9523-4cd9f474abf3\\\"\\n } ]\\n } ]\\n}]\",\"type\":\"UnprocessableEntity\",\"code\":\"unknown_error\"}],\"total_records\":1}", "content_length": "715", "content_type": "text/plain", "elapsed": 157, "msg": "Status code was 400 and not [200]: HTTP Error 400: Bad Request", "redirected": false, "status": 400, "url": "http://10.36.1.88:9130/_/proxy/tenants/diku/install?deploy=true&tenantParameters=loadSample%3Dtrue%2CloadReference%3Dtrue", "vary": "origin"}
|
| Comments |
| Comment by Marc Johnson [ 08/Mar/21 ] |
|
I thought that mod-permissions created dummy permissions for nested permissions that didn't exist? |
| Comment by Ian Hardy [ 08/Mar/21 ] |
|
looks like mod-search is trying to build a system user of some type https://github.com/folio-org/mod-search/commit/095cb98e462e89cf5468aeb4a3f54725cf4f6ff9 and perhaps failing there. Looks like the error okapi is throwing is saying this perm doesn't exist yet https://github.com/folio-org/mod-search/blob/095cb98e462e89cf5468aeb4a3f54725cf4f6ff9/src/main/resources/permissions/mod-search.csv.
I also notice thie weird address which seems like an undefined variable for an okapi url. |
| Comment by Marc Johnson [ 08/Mar/21 ] |
|
What is slightly odd about this error is that the permission is defined by mod-inventory-storage which should be enabled before mod-search based upon it depending upon on the instance-storage interface. Craig McNally Does mod-permissions apply any rules about which permissions can be granted to a user? |
| Comment by Marc Johnson [ 08/Mar/21 ] |
That is odd and could easily be the cause of an error like this. However, the text Attempting to add non-existent permissions is triggered by mod-permissions which suggests the actual module is being invoked. The check for this in mod-permissions doesn't seem to include any other criteria other than the permission name. |
| Comment by Marc Johnson [ 08/Mar/21 ] |
|
I think we are left with the same two options:
|
| Comment by Ian Hardy [ 08/Mar/21 ] |
|
Marc Johnson I think you're right I'm inclined to revert the changes I guess, not sure if others have an opinion.
fwiw, mod-inventory-storage is enabled on diku on the broken build (as expected) but there is no entry in the mod-permissions db for the perm in question: okapi_modules=# select jsonb->>'permissionName' from diku_mod_permissions.permissions where jsonb->>'permissionName' like 'inventory-storage.inventory%'; ?column? ---------- (0 rows) |
| Comment by Marc Johnson [ 08/Mar/21 ] |
Then it might be worth getting the Core Platform team involved to understand why that permission isn't loaded. |
| Comment by Adam Dickmeiss [ 08/Mar/21 ] |
|
I think this is a side effect of
Dummy permissions are for modules that refer to yet-to-be defined permissions. I'm not sure users can be assigned those. It's really a problem that we are going to have a gazilion users and passwords floating around. The better approach would be that tenant init provided a token that the module could use (forever). Then there would be no login. Since the the token is handed out at tenant init, Okapi and mod-authtoken would essentially know that the request (due to the token) originates from a backend module. So it is more trustworthy that something coming from the "outside". It would also makes the code more elegant for each module. |
| Comment by Hongwei Ji [ 08/Mar/21 ] |
|
Adam Dickmeiss , to be on the same page, so it is not mod-search's fault at this moment. We are going to fix Okapi, correct? |
| Comment by Marc Johnson [ 08/Mar/21 ] |
A long lived access token is a reasonable alternative to consider. My understanding is that expiring tokens are part of the roadmap for the platform in 2021. These two ideas might not be aligned with each other. |
| Comment by Adam Dickmeiss [ 08/Mar/21 ] |
|
Hongwei Ji. I think it's better to fix OKAPI. I don't think mod-search is doing anything wrong WRT this. So I'd suggest no revert in mod-search |
| Comment by Ian Hardy [ 08/Mar/21 ] |
|
Thanks Adam and Hongwei, I removed mod-search/ui-inventory-elasticsearch from the build for now so we could get a build. Happy to work w/mod-search devs to test enabling this module again when ready. |
| Comment by Magda Zacharska [ 08/Mar/21 ] |
|
Ian Hardy - from the comments above it seems the issue wasn't mod-search issue. Could you please re-enable both module in the reference environments? |
| Comment by Oleksii Petrenko [ 09/Mar/21 ] |
|
Please verify and close |
| Comment by Marc Johnson [ 09/Mar/21 ] |
|
Why has this been assigned to me, because I was the reporter? I do not think I am appropriately placed to verify that this is resolved. Adam Dickmeiss made the changes necessary. Ian Hardy rebuilt the environment yesterday and confirmed the resolution of this. I am not able to offer anything more than he did. I suggest this be owned by the Falcon team to check that mod-search is present and working as expected. |
| Comment by Hongwei Ji [ 09/Mar/21 ] |
|
Technically this is not a mod-search bug. I just verified testing/snapshot env and can see the system user mod-search was created and the perm |
| Comment by Hongwei Ji [ 09/Mar/21 ] |
|
Not a mod-search bug. |