[FOLIO-2366] The folio-testing-backend builds fail, missing dependency mod-authtoken requires users Created: 27/Nov/19  Updated: 03/Jun/20  Resolved: 03/Dec/19

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Bug Priority: TBD
Reporter: David Crossley Assignee: Ian Hardy
Resolution: Done Votes: 0
Labels: platform-backlog
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: File okapi-sample.log    
Issue links:
Relates
relates to MODAT-56 validate user deactivation when check... Closed
relates to MODAT-58 Do NOT check user for dummy token Closed
Sprint: CP: sprint 77, CP: sprint 78
Story Points: 3
Development Team: Core: Platform

 Description   

Jenkins says at https://jenkins-aws.indexdata.com/job/Automation/job/folio-testing-core-backend/247/console
(and folio-testing-backend)

...
"Missing dependency: mod-authtoken-2.4.0-SNAPSHOT.61 requires users: 15.0", 
...

With MODAT-56 Closed the mod-authtoken now requires users.

Perhaps this is due to the order of declarations in folio-ansible group_vars/testing* files.



 Comments   
Comment by Ian Hardy [ 27/Nov/19 ]

re-ordered the modules in folio-ansible's testing and testing-core group vars which allows the backend to proceed. Frontend build is failing on the stripes build role during enable modules handler--likely related.

From the okapi log:

2019-11-27 21:09:32,067 ERROR HttpResponse         HTTP response code=500 msg=HEAD request for mod-authtoken-2.4.0-SNAPSHOT.61 /_/tenantpermissions failed with 
2019-11-27 21:09:32,067 INFO  DockerModuleHandle   mod-authtoken-2.4.0-SNAPSHOT.61 Nov 27, 2019 9:09:32 PM mod-auth-authtoken-module
2019-11-27 21:09:32,067 INFO  DockerModuleHandle   mod-authtoken-2.4.0-SNAPSHOT.61 SEVERE: Unable to retrieve permissions for UNDEFINED_USER__10.36.1.167:40586__2019-11-27T21:09:32.056+0000: User with id bddd70cd-036b-5456-976e-01ee5e84a05f does not exist request took 10 ms
2019-11-27 21:09:32,067 INFO  DockerModuleHandle   mod-authtoken-2.4.0-SNAPSHOT.61 Nov 27, 2019 9:09:32 PM mod-auth-authtoken-module
2019-11-27 21:09:32,067 INFO  DockerModuleHandle   mod-authtoken-2.4.0-SNAPSHOT.61 SEVERE: Invalid token: User with id bddd70cd-036b-5456-976e-01ee5e84a05f does not exist

Note that bddd70cd-036b-5456-976e-01ee5e84a05f is the supertenant admin which does exist:

$ http https://folio-testing-okapi.aws.indexdata.com/users/bddd70cd-036b-5456-976e-01ee5e84a05f "x-okapi-token:$tt"
...
x-okapi-user-id: bddd70cd-036b-5456-976e-01ee5e84a05f

{
    "active": true,
    "createdDate": "2019-11-27T15:56:46.701+0000",
    "id": "bddd70cd-036b-5456-976e-01ee5e84a05f",
    "metadata": {
        "createdDate": "2019-11-27T15:56:46.699+0000",
        "updatedDate": "2019-11-27T15:56:46.699+0000"
    },
    "proxyFor": [],
    "updatedDate": "2019-11-27T15:56:46.701+0000",
    "username": "super_admin"
}
Comment by Hongwei Ji [ 28/Nov/19 ]

Ian Hardy, is it possible the user belongs to supertenant, but the actual request has the diku as the tenant header? That could explain why the user cannot be found.

Comment by Ian Hardy [ 29/Nov/19 ]

It does look like the problem is looking up the user in the wrong tenant. The ansible task that is triggering this failing request is hard-coded to "x-okapi-tenant:supertenant" and hasn't been updated recently: https://github.com/folio-org/folio-ansible/blob/20c6182f9fc8e4b7a184f781b91172c4eede6f04/roles/stripes-build/handlers/main.yml#L114

I think this is a more complete look at the okapi log that corresponds with the call highlighted above. You can see the post request for /_/proxy/tenants/diku/modules w/the user agent ansible-httpget, and then a failure to find the supertenant user in the diku tenant.

2019-11-29 03:17:41,324 INFO  ProxyContext         154668/proxy REQ 10.36.1.94:10712 supertenant POST /_/proxy/tenants/diku/modules mod-authtoken-2.4.0-SNAPSHOT.61 okapi-2.33.0
2019-11-29 03:17:41,324 INFO  ProxyService         Request headers size=886
2019-11-29 03:17:41,324 INFO  ProxyService         Accept: application/json
2019-11-29 03:17:41,324 INFO  ProxyService         Accept-Encoding: identity
2019-11-29 03:17:41,324 INFO  ProxyService         Content-Length: 34
2019-11-29 03:17:41,324 INFO  ProxyService         Content-Type: application/json
2019-11-29 03:17:41,325 INFO  ProxyService         Host: folio-testing-core-okapi.aws.indexdata.com
2019-11-29 03:17:41,325 INFO  ProxyService         User-Agent: ansible-httpget
2019-11-29 03:17:41,325 INFO  ProxyService         X-Amzn-Trace-Id: Root=1-5de08dd5-2feed760477642204fe3a9d0
2019-11-29 03:17:41,325 INFO  ProxyService         X-Forwarded-For: 52.23.169.168
2019-11-29 03:17:41,325 INFO  ProxyService         X-Forwarded-Port: 443
2019-11-29 03:17:41,325 INFO  ProxyService         X-Forwarded-Proto: https
2019-11-29 03:17:41,325 INFO  ProxyService         X-Okapi-Match-Path-Pattern: /*
2019-11-29 03:17:41,325 INFO  ProxyService         X-Okapi-Module-Permissions: {}
2019-11-29 03:17:41,325 INFO  ProxyService         X-Okapi-Permissions-Required: okapi.proxy.tenants.modules.post
2019-11-29 03:17:41,326 INFO  ProxyService         X-Okapi-Request-Id: 154668/proxy
2019-11-29 03:17:41,326 INFO  ProxyService         X-Okapi-request-ip: 10.36.1.94
2019-11-29 03:17:41,326 INFO  ProxyService         X-Okapi-request-method: POST
2019-11-29 03:17:41,326 INFO  ProxyService         X-Okapi-request-timestamp: 1574997461324
2019-11-29 03:17:41,326 INFO  ProxyService         X-Okapi-Tenant: supertenant
2019-11-29 03:17:41,326 INFO  ProxyService         X-Okapi-Token: eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJzdXBlcl9hZG1pbiIsInVzZXJfaWQiOiJiZGRkNzBjZC0wMzZiLTU0NTYtOTc2ZS0wMWVlNWU4NGEwNWYiLCJpYXQiOjE1NzQ5OTcxNzMsInRlbmFudCI6InN1cGVydGVuYW50In0.-SI0IPTOkul-Q3LgsyRkp3ulDqGQcupEIrDkR93LzdI
2019-11-29 03:17:41,326 INFO  ProxyService         X-Okapi-Url: http://10.36.1.220:9130
2019-11-29 03:17:41,327 INFO  ProxyContext         154668/proxy RES 202 3424us mod-authtoken-2.4.0-SNAPSHOT.61 http://10.36.1.220:9133/_/proxy/tenants/diku/modules
2019-11-29 03:17:41,333 INFO  OkapiClient          154668/proxy;523693/tenantpermissions REQ okapiClient diku HEAD http://10.36.1.220:9133/_/tenantpermissions
2019-11-29 03:17:41,337 INFO  ProxyContext         154668/proxy;523693/tenantpermissions;685172/users REQ 172.17.0.4:53870 diku GET /users/bddd70cd-036b-5456-976e-01ee5e84a05f mod-authtoken-2.4.0-SNAPSHOT.61 mod-users-15.7.0-SNAPSHOT.107
2019-11-29 03:17:41,337 INFO  ProxyService         Request headers size=774
2019-11-29 03:17:41,337 INFO  ProxyService         Accept: application/json
2019-11-29 03:17:41,337 INFO  ProxyService         Content-Type: application/json
2019-11-29 03:17:41,337 INFO  ProxyService         Host: 10.36.1.220:9130
2019-11-29 03:17:41,338 INFO  ProxyService         X-Okapi-Match-Path-Pattern: /*
2019-11-29 03:17:41,338 INFO  ProxyService         X-Okapi-Module-Permissions: {}
2019-11-29 03:17:41,338 INFO  ProxyService         X-Okapi-Permissions-Desired: users.read.restricted,users.read.basic
2019-11-29 03:17:41,338 INFO  ProxyService         X-Okapi-Permissions-Required: users.item.get
2019-11-29 03:17:41,338 INFO  ProxyService         X-Okapi-Request-Id: 154668/proxy;523693/tenantpermissions;685172/users
2019-11-29 03:17:41,338 INFO  ProxyService         X-Okapi-request-ip: 172.17.0.4
2019-11-29 03:17:41,338 INFO  ProxyService         X-Okapi-request-method: GET
2019-11-29 03:17:41,338 INFO  ProxyService         X-Okapi-request-timestamp: 1574997461337
2019-11-29 03:17:41,339 INFO  ProxyService         X-Okapi-Tenant: diku
2019-11-29 03:17:41,339 INFO  ProxyService         X-Okapi-Token: eyJhbGciOiJIUzI1NiJ9.eyJkdW1teSI6dHJ1ZSwic3ViIjoiX0FVVEhaX01PRFVMRV8iLCJleHRyYV9wZXJtaXNzaW9ucyI6WyJ1c2Vycy5pdGVtLmdldCJdLCJyZXF1ZXN0X2lkIjoiMTU0NjY4XC9wcm94eTs1MjM2OTNcL3RlbmFudHBlcm1pc3Npb25zIiwidGVuYW50IjoiZGlrdSJ9.YIYeYi7_DL2zMwbrMAWbZZmxMa4MCU1W0uSF4tPMRTM
2019-11-29 03:17:41,339 INFO  ProxyService         X-Okapi-Url: http://10.36.1.220:9130
2019-11-29 03:17:41,340 INFO  ProxyContext         154668/proxy;523693/tenantpermissions;685172/users RES 202 2611us mod-authtoken-2.4.0-SNAPSHOT.61 http://10.36.1.220:9133/users/bddd70cd-036b-5456-976e-01ee5e84a05f
2019-11-29 03:17:41,340 INFO  ProxyService         Request headers size=677
2019-11-29 03:17:41,340 INFO  ProxyService         Accept: application/json
2019-11-29 03:17:41,340 INFO  ProxyService         Content-Type: application/json
2019-11-29 03:17:41,340 INFO  ProxyService         Host: 10.36.1.220:9130
2019-11-29 03:17:41,340 INFO  ProxyService         X-Okapi-Match-Path-Pattern: /users/{id}
2019-11-29 03:17:41,340 INFO  ProxyService         X-Okapi-Permissions: ["users.item.get"]
2019-11-29 03:17:41,340 INFO  ProxyService         X-Okapi-Request-Id: 154668/proxy;523693/tenantpermissions;685172/users
2019-11-29 03:17:41,340 INFO  ProxyService         X-Okapi-request-ip: 172.17.0.4
2019-11-29 03:17:41,340 INFO  ProxyService         X-Okapi-request-method: GET
2019-11-29 03:17:41,340 INFO  ProxyService         X-Okapi-request-timestamp: 1574997461337
2019-11-29 03:17:41,340 INFO  ProxyService         X-Okapi-Tenant: diku
2019-11-29 03:17:41,341 INFO  ProxyService         X-Okapi-Token: eyJhbGciOiJIUzI1NiJ9.eyJkdW1teSI6dHJ1ZSwic3ViIjoiX0FVVEhaX01PRFVMRV8iLCJleHRyYV9wZXJtaXNzaW9ucyI6WyJ1c2Vycy5pdGVtLmdldCJdLCJyZXF1ZXN0X2lkIjoiMTU0NjY4XC9wcm94eTs1MjM2OTNcL3RlbmFudHBlcm1pc3Npb25zIiwidGVuYW50IjoiZGlrdSJ9.YIYeYi7_DL2zMwbrMAWbZZmxMa4MCU1W0uSF4tPMRTM
2019-11-29 03:17:41,341 INFO  ProxyService         X-Okapi-Url: http://10.36.1.220:9130
2019-11-29 03:17:41,342 WARN  ProxyService         Removing X-Okapi-Token returned by module mod-users-15.7.0-SNAPSHOT.107 (RMB-478)
2019-11-29 03:17:41,342 INFO  ProxyContext         154668/proxy;523693/tenantpermissions;685172/users RES 404 2772us mod-users-15.7.0-SNAPSHOT.107 http://10.36.1.220:9132/users/bddd70cd-036b-5456-976e-01ee5e84a05f
2019-11-29 03:17:41,343 INFO  DockerModuleHandle   mod-users-15.7.0-SNAPSHOT.107 03:17:41 INFO  LogUtil              org.folio.rest.RestVerticle start  invoking getUsersByUserId
2019-11-29 03:17:41,343 INFO  DockerModuleHandle   mod-users-15.7.0-SNAPSHOT.107 03:17:41 INFO  LogUtil              10.36.1.220:34428 GET /users/bddd70cd-036b-5456-976e-01ee5e84a05f null HTTP_1_1 404 9 1 tid=diku Not Found
2019-11-29 03:17:41,344 INFO  OkapiClient          154668/proxy;523693/tenantpermissions RES 401 10235us okapiClient http://10.36.1.220:9133/_/tenantpermissions
2019-11-29 03:17:41,344 WARN  ProxyService         HEAD request for mod-authtoken-2.4.0-SNAPSHOT.61 /_/tenantpermissions failed with
2019-11-29 03:17:41,344 WARN  ProxyService         Auth check for systemInterface failed!
2019-11-29 03:17:41,344 INFO  ProxyContext         154668/proxy RES 500 16678us okapi HEAD request for mod-authtoken-2.4.0-SNAPSHOT.61 /_/tenantpermissions failed with
2019-11-29 03:17:41,344 ERROR HttpResponse         HTTP response code=500 msg=HEAD request for mod-authtoken-2.4.0-SNAPSHOT.61 /_/tenantpermissions failed with
2019-11-29 03:17:41,344 INFO  DockerModuleHandle   mod-authtoken-2.4.0-SNAPSHOT.61 Nov 29, 2019 3:17:41 AM mod-auth-authtoken-module
2019-11-29 03:17:41,345 INFO  DockerModuleHandle   mod-authtoken-2.4.0-SNAPSHOT.61 SEVERE: Unable to retrieve permissions for UNDEFINED_USER__10.36.1.220:39634__2019-11-29T03:17:41.334+0000: User with id bddd70cd-036b-5456-976e-01ee5e84a05f does not exist request took 9 ms
2019-11-29 03:17:41,345 INFO  DockerModuleHandle   mod-authtoken-2.4.0-SNAPSHOT.61 Nov 29, 2019 3:17:41 AM mod-auth-authtoken-module
2019-11-29 03:17:41,345 INFO  DockerModuleHandle   mod-authtoken-2.4.0-SNAPSHOT.61 SEVERE: Invalid token: User with id bddd70cd-036b-5456-976e-01ee5e84a05f does not exist

I took a sample of the okapi log while the ansible play was running, and I'll attach that here (the above excerpt starts at 4739). okapi-sample.log

Its strange that there's an issue with this particular routine, but we haven't seen failures in the other builds.

Comment by Adam Dickmeiss [ 02/Dec/19 ]

https://github.com/folio-org/mod-authtoken/pull/68

Comment by Wayne Schneider [ 03/Dec/19 ]

Testing with the mod-authtoken branch referenced above seems to show that the change addresses at least this use case (a user with Okapi interface permissions can enable modules for a different tenant).

Comment by Ian Hardy [ 03/Dec/19 ]

Thanks Adam Dickmeiss and Wayne Schneider, since the work in modat-58 has been merged this is no longer breaking testing frontend builds.

Generated at Thu Feb 08 23:20:08 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.