[MGRENTITLE-1] CO - mgr-tenant-entitlements - Failed to create routes Created: 31/Dec/23 Updated: 05/Feb/24 |
|
| Status: | In Review |
| Project: | mgr-tenant-entitlements |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Bug | Priority: | P3 |
| Reporter: | Craig McNally | Assignee: | Yauhen Viazau |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | back-end, epam-eureka, eureka-phase4, need-review | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Attachments: |
|
||||||||
| Issue links: |
|
||||||||
| Sprint: | Eureka Sprint 45 | ||||||||
| Story Points: | 0 | ||||||||
| Development Team: | Eureka | ||||||||
| Description |
|
mgr-tenant-entitlements periodically fails to create a routes {{"errors"": [ { ""message"": ""Application flow '88f6c07d-6b19-4983-ba78-789417891e1d' executed with status: FAILED"", ""type"": ""FlowExecutionException"", ""code"": ""service_error"", ""parameters"": [ ...SKIPPED.... { ""key"": ""KongGatewayRouteRegistrar"", ""value"": ""FAILED: [IntegrationException] Failed to create routes, parameters: [{key: RoutingEntry(methods=[GET], pathPattern=/identifier-types/ {id}, path=null), value: [502 Bad Gateway] during [PUT] to http://kong-admin.evrk.folio-eis.us-west-2:8001/services/f1b2323c-9565-41d7-996d-5048782bfec7/routes/e2a92cf21171873411c8ad67cab1f1505c8766a3 KongAdminClient#upsertRoute(String,String,KongRoute): [<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n</body>\r\n</html>\r\n]}]"" }, { ""key"": ""KongGatewayRouteRegistrar"", ""value"": ""FAILED: [IntegrationException] Failed to remove routes, parameters: [}} {{ {key: 8df188b1-ae7b-4d50-a9c1-fe2293d67143, value: [502 Bad Gateway] during [DELETE] to http://kong-admin.evrk.folio-eis.us-west-2:8001/services/13f65407-bf15-43e9-a250-89be33f2900c/routes/8df188b1-ae7b-4d50-a9c1-fe2293d67143 [KongAdminClient#deleteRoute(String,String)]: [<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n</body>\r\n</html>\r\n]}}} ]"" },1694509224627,com.ebsco.entitlement.integration.IntegrationException: Failed to create routes1694509224627, at com.ebsco.entitlement.integration.kong.KongGatewayService.addRoutes(KongGatewayService.java:55) ~[classes!/:1.0.0-SNAPSHOT]1694509224627, at com.ebsco.entitlement.integration.kong.KongGatewayRouteRegistrar.execute(KongGatewayRouteRegistrar.java:30) ~[classes!/:1.0.0-SNAPSHOT]1694509224627, at jdk.internal.reflect.GeneratedMethodAccessor328.invoke(Unknown Source) ~[?:?]1694509224627, at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]1694509224627, at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]1694509224627, at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-6.0.6.jar!/:6.0.6]1694509224627, at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) ~[spring-aop-6.0.6.jar!/:6.0.6]1694509224627, at com.ebsco.entitlement.integration.kong.KongGatewayRouteRegistrar$$SpringCGLIB$$0.execute(<generated>) ~[classes!/:1.0.0-SNAPSHOT]1694509224627, at com.ebsco.flow.api.ExecutableStage.executeStage(ExecutableStage.java:66) ~[folio-flow-engine-1.0.2-SNAPSHOT.jar!/:1.0.2-SNAPSHOT]1694509224627, at com.ebsco.flow.api.ExecutableStage.execute(ExecutableStage.java:32) ~[folio-flow-engine-1.0.2-SNAPSHOT.jar!/:1.0.2-SNAPSHOT]1694509224627, at com.ebsco.flow.api.ParallelStage.lambda$composeExecutionFuture$7(ParallelStage.java:93) ~[folio-flow-engine-1.0.2-SNAPSHOT.jar!/:1.0.2-SNAPSHOT]1694509224627, at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.CompletableFuture$AsyncSupply.exec(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.ForkJoinPool$WorkQueue.helpAsyncBlocker(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.ForkJoinPool.helpAsyncBlocker(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.CompletableFuture.waitingGet(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.CompletableFuture.join(Unknown Source) ~[?:?]1694509224627, at com.ebsco.flow.api.ParallelStage.execute(ParallelStage.java:57) ~[folio-flow-engine-1.0.2-SNAPSHOT.jar!/:1.0.2-SNAPSHOT]1694509224627, at com.ebsco.flow.api.ExecutableStage.executeStage(ExecutableStage.java:66) ~[folio-flow-engine-1.0.2-SNAPSHOT.jar!/:1.0.2-SNAPSHOT]1694509224627, at com.ebsco.flow.api.ExecutableStage.execute(ExecutableStage.java:32) ~[folio-flow-engine-1.0.2-SNAPSHOT.jar!/:1.0.2-SNAPSHOT]1694509224627, at com.ebsco.flow.api.FlowImpl.lambda$executeStageAsync$6(FlowImpl.java:270) ~[folio-flow-engine-1.0.2-SNAPSHOT.jar!/:1.0.2-SNAPSHOT]1694509224627, at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.CompletableFuture$AsyncSupply.exec(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.ForkJoinPool$WorkQueue.helpAsyncBlocker(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.ForkJoinPool.helpAsyncBlocker(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.CompletableFuture.waitingGet(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.CompletableFuture.join(Unknown Source) ~[?:?]1694509224627, at com.ebsco.flow.api.FlowImpl.execute(FlowImpl.java:151) ~[folio-flow-engine-1.0.2-SNAPSHOT.jar!/:1.0.2-SNAPSHOT]1694509224627, at com.ebsco.flow.api.ExecutableStage.executeStage(ExecutableStage.java:66) ~[folio-flow-engine-1.0.2-SNAPSHOT.jar!/:1.0.2-SNAPSHOT]1694509224627, at com.ebsco.flow.api.ExecutableStage.execute(ExecutableStage.java:32) ~[folio-flow-engine-1.0.2-SNAPSHOT.jar!/:1.0.2-SNAPSHOT]1694509224627, at com.ebsco.flow.api.FlowImpl.lambda$executeStageAsync$6(FlowImpl.java:270) ~[folio-flow-engine-1.0.2-SNAPSHOT.jar!/:1.0.2-SNAPSHOT]1694509224627, at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.CompletableFuture$AsyncSupply.exec(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.ForkJoinPool.scan(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) ~[?:?]1694509224627, at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) ~[?:?] Tenant entitlement result must be stable and return 200 OK after installation Scope: Investigate and find the root cause and document findings on the wiki. Either fix the issue if it's simple, or create a follow-on story to be triaged/prioritized NOTE: Upgrading to the latest version of Kong may resolve this. It might be worth doing that task first to see if this is still an issue afterwards. TODO: @Craig McNally for find and link the Kong upgrade story as a dependency. |
| Comments |
| Comment by Yauhen Viazau [ 12/Jan/24 ] |
|
Checked on Demo env When running POST /entitlements for "app-platform-complete-0.0.2", sometimes entitlement gets cancelled due to timeout error, for example: {
"key": "mod-inventory-20.1.3-moduleInstaller",
"value": "FAILED: [IntegrationException] [HttpTimeoutException] Failed to perform request [method: POST, uri: http://mod-inventory-b.oed.folio-eis.us-west-2:8051/mod-inventory/_/tenant], cause: request timed out"
},
|
| Comment by Yauhen Viazau [ 05/Feb/24 ] |
|
Tested on “evrk“ env - works as expected Verified by entitling and removing “app-platform-minimal“ to 5 different tenants. See example: As per discussion with developers, this issue stopped being reproducible after switching to newer Keycloak version |