[FOLIO-2366] The folio-testing-backend builds fail, missing dependency mod-authtoken requires users Created: 27/Nov/19 Updated: 03/Jun/20 Resolved: 03/Dec/19 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Bug | Priority: | TBD |
| Reporter: | David Crossley | Assignee: | Ian Hardy |
| Resolution: | Done | Votes: | 0 |
| Labels: | platform-backlog | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||
| Issue links: |
|
||||||||||||
| Sprint: | CP: sprint 77, CP: sprint 78 | ||||||||||||
| Story Points: | 3 | ||||||||||||
| Development Team: | Core: Platform | ||||||||||||
| Description |
|
Jenkins says at https://jenkins-aws.indexdata.com/job/Automation/job/folio-testing-core-backend/247/console ... "Missing dependency: mod-authtoken-2.4.0-SNAPSHOT.61 requires users: 15.0", ... With
Perhaps this is due to the order of declarations in folio-ansible group_vars/testing* files. |
| Comments |
| Comment by Ian Hardy [ 27/Nov/19 ] |
|
re-ordered the modules in folio-ansible's testing and testing-core group vars which allows the backend to proceed. Frontend build is failing on the stripes build role during enable modules handler--likely related. From the okapi log: 2019-11-27 21:09:32,067 ERROR HttpResponse HTTP response code=500 msg=HEAD request for mod-authtoken-2.4.0-SNAPSHOT.61 /_/tenantpermissions failed with 2019-11-27 21:09:32,067 INFO DockerModuleHandle mod-authtoken-2.4.0-SNAPSHOT.61 Nov 27, 2019 9:09:32 PM mod-auth-authtoken-module 2019-11-27 21:09:32,067 INFO DockerModuleHandle mod-authtoken-2.4.0-SNAPSHOT.61 SEVERE: Unable to retrieve permissions for UNDEFINED_USER__10.36.1.167:40586__2019-11-27T21:09:32.056+0000: User with id bddd70cd-036b-5456-976e-01ee5e84a05f does not exist request took 10 ms 2019-11-27 21:09:32,067 INFO DockerModuleHandle mod-authtoken-2.4.0-SNAPSHOT.61 Nov 27, 2019 9:09:32 PM mod-auth-authtoken-module 2019-11-27 21:09:32,067 INFO DockerModuleHandle mod-authtoken-2.4.0-SNAPSHOT.61 SEVERE: Invalid token: User with id bddd70cd-036b-5456-976e-01ee5e84a05f does not exist Note that bddd70cd-036b-5456-976e-01ee5e84a05f is the supertenant admin which does exist: $ http https://folio-testing-okapi.aws.indexdata.com/users/bddd70cd-036b-5456-976e-01ee5e84a05f "x-okapi-token:$tt"
...
x-okapi-user-id: bddd70cd-036b-5456-976e-01ee5e84a05f
{
"active": true,
"createdDate": "2019-11-27T15:56:46.701+0000",
"id": "bddd70cd-036b-5456-976e-01ee5e84a05f",
"metadata": {
"createdDate": "2019-11-27T15:56:46.699+0000",
"updatedDate": "2019-11-27T15:56:46.699+0000"
},
"proxyFor": [],
"updatedDate": "2019-11-27T15:56:46.701+0000",
"username": "super_admin"
}
|
| Comment by Hongwei Ji [ 28/Nov/19 ] |
|
Ian Hardy, is it possible the user belongs to supertenant, but the actual request has the diku as the tenant header? That could explain why the user cannot be found. |
| Comment by Ian Hardy [ 29/Nov/19 ] |
|
It does look like the problem is looking up the user in the wrong tenant. The ansible task that is triggering this failing request is hard-coded to "x-okapi-tenant:supertenant" and hasn't been updated recently: https://github.com/folio-org/folio-ansible/blob/20c6182f9fc8e4b7a184f781b91172c4eede6f04/roles/stripes-build/handlers/main.yml#L114 I think this is a more complete look at the okapi log that corresponds with the call highlighted above. You can see the post request for /_/proxy/tenants/diku/modules w/the user agent ansible-httpget, and then a failure to find the supertenant user in the diku tenant. 2019-11-29 03:17:41,324 INFO ProxyContext 154668/proxy REQ 10.36.1.94:10712 supertenant POST /_/proxy/tenants/diku/modules mod-authtoken-2.4.0-SNAPSHOT.61 okapi-2.33.0
2019-11-29 03:17:41,324 INFO ProxyService Request headers size=886
2019-11-29 03:17:41,324 INFO ProxyService Accept: application/json
2019-11-29 03:17:41,324 INFO ProxyService Accept-Encoding: identity
2019-11-29 03:17:41,324 INFO ProxyService Content-Length: 34
2019-11-29 03:17:41,324 INFO ProxyService Content-Type: application/json
2019-11-29 03:17:41,325 INFO ProxyService Host: folio-testing-core-okapi.aws.indexdata.com
2019-11-29 03:17:41,325 INFO ProxyService User-Agent: ansible-httpget
2019-11-29 03:17:41,325 INFO ProxyService X-Amzn-Trace-Id: Root=1-5de08dd5-2feed760477642204fe3a9d0
2019-11-29 03:17:41,325 INFO ProxyService X-Forwarded-For: 52.23.169.168
2019-11-29 03:17:41,325 INFO ProxyService X-Forwarded-Port: 443
2019-11-29 03:17:41,325 INFO ProxyService X-Forwarded-Proto: https
2019-11-29 03:17:41,325 INFO ProxyService X-Okapi-Match-Path-Pattern: /*
2019-11-29 03:17:41,325 INFO ProxyService X-Okapi-Module-Permissions: {}
2019-11-29 03:17:41,325 INFO ProxyService X-Okapi-Permissions-Required: okapi.proxy.tenants.modules.post
2019-11-29 03:17:41,326 INFO ProxyService X-Okapi-Request-Id: 154668/proxy
2019-11-29 03:17:41,326 INFO ProxyService X-Okapi-request-ip: 10.36.1.94
2019-11-29 03:17:41,326 INFO ProxyService X-Okapi-request-method: POST
2019-11-29 03:17:41,326 INFO ProxyService X-Okapi-request-timestamp: 1574997461324
2019-11-29 03:17:41,326 INFO ProxyService X-Okapi-Tenant: supertenant
2019-11-29 03:17:41,326 INFO ProxyService X-Okapi-Token: eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJzdXBlcl9hZG1pbiIsInVzZXJfaWQiOiJiZGRkNzBjZC0wMzZiLTU0NTYtOTc2ZS0wMWVlNWU4NGEwNWYiLCJpYXQiOjE1NzQ5OTcxNzMsInRlbmFudCI6InN1cGVydGVuYW50In0.-SI0IPTOkul-Q3LgsyRkp3ulDqGQcupEIrDkR93LzdI
2019-11-29 03:17:41,326 INFO ProxyService X-Okapi-Url: http://10.36.1.220:9130
2019-11-29 03:17:41,327 INFO ProxyContext 154668/proxy RES 202 3424us mod-authtoken-2.4.0-SNAPSHOT.61 http://10.36.1.220:9133/_/proxy/tenants/diku/modules
2019-11-29 03:17:41,333 INFO OkapiClient 154668/proxy;523693/tenantpermissions REQ okapiClient diku HEAD http://10.36.1.220:9133/_/tenantpermissions
2019-11-29 03:17:41,337 INFO ProxyContext 154668/proxy;523693/tenantpermissions;685172/users REQ 172.17.0.4:53870 diku GET /users/bddd70cd-036b-5456-976e-01ee5e84a05f mod-authtoken-2.4.0-SNAPSHOT.61 mod-users-15.7.0-SNAPSHOT.107
2019-11-29 03:17:41,337 INFO ProxyService Request headers size=774
2019-11-29 03:17:41,337 INFO ProxyService Accept: application/json
2019-11-29 03:17:41,337 INFO ProxyService Content-Type: application/json
2019-11-29 03:17:41,337 INFO ProxyService Host: 10.36.1.220:9130
2019-11-29 03:17:41,338 INFO ProxyService X-Okapi-Match-Path-Pattern: /*
2019-11-29 03:17:41,338 INFO ProxyService X-Okapi-Module-Permissions: {}
2019-11-29 03:17:41,338 INFO ProxyService X-Okapi-Permissions-Desired: users.read.restricted,users.read.basic
2019-11-29 03:17:41,338 INFO ProxyService X-Okapi-Permissions-Required: users.item.get
2019-11-29 03:17:41,338 INFO ProxyService X-Okapi-Request-Id: 154668/proxy;523693/tenantpermissions;685172/users
2019-11-29 03:17:41,338 INFO ProxyService X-Okapi-request-ip: 172.17.0.4
2019-11-29 03:17:41,338 INFO ProxyService X-Okapi-request-method: GET
2019-11-29 03:17:41,338 INFO ProxyService X-Okapi-request-timestamp: 1574997461337
2019-11-29 03:17:41,339 INFO ProxyService X-Okapi-Tenant: diku
2019-11-29 03:17:41,339 INFO ProxyService X-Okapi-Token: eyJhbGciOiJIUzI1NiJ9.eyJkdW1teSI6dHJ1ZSwic3ViIjoiX0FVVEhaX01PRFVMRV8iLCJleHRyYV9wZXJtaXNzaW9ucyI6WyJ1c2Vycy5pdGVtLmdldCJdLCJyZXF1ZXN0X2lkIjoiMTU0NjY4XC9wcm94eTs1MjM2OTNcL3RlbmFudHBlcm1pc3Npb25zIiwidGVuYW50IjoiZGlrdSJ9.YIYeYi7_DL2zMwbrMAWbZZmxMa4MCU1W0uSF4tPMRTM
2019-11-29 03:17:41,339 INFO ProxyService X-Okapi-Url: http://10.36.1.220:9130
2019-11-29 03:17:41,340 INFO ProxyContext 154668/proxy;523693/tenantpermissions;685172/users RES 202 2611us mod-authtoken-2.4.0-SNAPSHOT.61 http://10.36.1.220:9133/users/bddd70cd-036b-5456-976e-01ee5e84a05f
2019-11-29 03:17:41,340 INFO ProxyService Request headers size=677
2019-11-29 03:17:41,340 INFO ProxyService Accept: application/json
2019-11-29 03:17:41,340 INFO ProxyService Content-Type: application/json
2019-11-29 03:17:41,340 INFO ProxyService Host: 10.36.1.220:9130
2019-11-29 03:17:41,340 INFO ProxyService X-Okapi-Match-Path-Pattern: /users/{id}
2019-11-29 03:17:41,340 INFO ProxyService X-Okapi-Permissions: ["users.item.get"]
2019-11-29 03:17:41,340 INFO ProxyService X-Okapi-Request-Id: 154668/proxy;523693/tenantpermissions;685172/users
2019-11-29 03:17:41,340 INFO ProxyService X-Okapi-request-ip: 172.17.0.4
2019-11-29 03:17:41,340 INFO ProxyService X-Okapi-request-method: GET
2019-11-29 03:17:41,340 INFO ProxyService X-Okapi-request-timestamp: 1574997461337
2019-11-29 03:17:41,340 INFO ProxyService X-Okapi-Tenant: diku
2019-11-29 03:17:41,341 INFO ProxyService X-Okapi-Token: eyJhbGciOiJIUzI1NiJ9.eyJkdW1teSI6dHJ1ZSwic3ViIjoiX0FVVEhaX01PRFVMRV8iLCJleHRyYV9wZXJtaXNzaW9ucyI6WyJ1c2Vycy5pdGVtLmdldCJdLCJyZXF1ZXN0X2lkIjoiMTU0NjY4XC9wcm94eTs1MjM2OTNcL3RlbmFudHBlcm1pc3Npb25zIiwidGVuYW50IjoiZGlrdSJ9.YIYeYi7_DL2zMwbrMAWbZZmxMa4MCU1W0uSF4tPMRTM
2019-11-29 03:17:41,341 INFO ProxyService X-Okapi-Url: http://10.36.1.220:9130
2019-11-29 03:17:41,342 WARN ProxyService Removing X-Okapi-Token returned by module mod-users-15.7.0-SNAPSHOT.107 (RMB-478)
2019-11-29 03:17:41,342 INFO ProxyContext 154668/proxy;523693/tenantpermissions;685172/users RES 404 2772us mod-users-15.7.0-SNAPSHOT.107 http://10.36.1.220:9132/users/bddd70cd-036b-5456-976e-01ee5e84a05f
2019-11-29 03:17:41,343 INFO DockerModuleHandle mod-users-15.7.0-SNAPSHOT.107 03:17:41 INFO LogUtil org.folio.rest.RestVerticle start invoking getUsersByUserId
2019-11-29 03:17:41,343 INFO DockerModuleHandle mod-users-15.7.0-SNAPSHOT.107 03:17:41 INFO LogUtil 10.36.1.220:34428 GET /users/bddd70cd-036b-5456-976e-01ee5e84a05f null HTTP_1_1 404 9 1 tid=diku Not Found
2019-11-29 03:17:41,344 INFO OkapiClient 154668/proxy;523693/tenantpermissions RES 401 10235us okapiClient http://10.36.1.220:9133/_/tenantpermissions
2019-11-29 03:17:41,344 WARN ProxyService HEAD request for mod-authtoken-2.4.0-SNAPSHOT.61 /_/tenantpermissions failed with
2019-11-29 03:17:41,344 WARN ProxyService Auth check for systemInterface failed!
2019-11-29 03:17:41,344 INFO ProxyContext 154668/proxy RES 500 16678us okapi HEAD request for mod-authtoken-2.4.0-SNAPSHOT.61 /_/tenantpermissions failed with
2019-11-29 03:17:41,344 ERROR HttpResponse HTTP response code=500 msg=HEAD request for mod-authtoken-2.4.0-SNAPSHOT.61 /_/tenantpermissions failed with
2019-11-29 03:17:41,344 INFO DockerModuleHandle mod-authtoken-2.4.0-SNAPSHOT.61 Nov 29, 2019 3:17:41 AM mod-auth-authtoken-module
2019-11-29 03:17:41,345 INFO DockerModuleHandle mod-authtoken-2.4.0-SNAPSHOT.61 SEVERE: Unable to retrieve permissions for UNDEFINED_USER__10.36.1.220:39634__2019-11-29T03:17:41.334+0000: User with id bddd70cd-036b-5456-976e-01ee5e84a05f does not exist request took 9 ms
2019-11-29 03:17:41,345 INFO DockerModuleHandle mod-authtoken-2.4.0-SNAPSHOT.61 Nov 29, 2019 3:17:41 AM mod-auth-authtoken-module
2019-11-29 03:17:41,345 INFO DockerModuleHandle mod-authtoken-2.4.0-SNAPSHOT.61 SEVERE: Invalid token: User with id bddd70cd-036b-5456-976e-01ee5e84a05f does not exist
I took a sample of the okapi log while the ansible play was running, and I'll attach that here (the above excerpt starts at 4739). okapi-sample.log Its strange that there's an issue with this particular routine, but we haven't seen failures in the other builds. |
| Comment by Adam Dickmeiss [ 02/Dec/19 ] |
| Comment by Wayne Schneider [ 03/Dec/19 ] |
|
Testing with the mod-authtoken branch referenced above seems to show that the change addresses at least this use case (a user with Okapi interface permissions can enable modules for a different tenant). |
| Comment by Ian Hardy [ 03/Dec/19 ] |
|
Thanks Adam Dickmeiss and Wayne Schneider, since the work in modat-58 has been merged this is no longer breaking testing frontend builds. |