500, 502 errors appear while upload file with valid identifiers in Bulk edit on Eureka environment
Description
CSP Request Details
CSP Rejection Details
Potential Workaround
Attachments
blocks
is blocked by
relates to
Checklist
hideActivity
Magda Zacharska January 31, 2025 at 10:02 PM
I’m closing the Jira as done as the issues reported in the original ticket have been addressed. I have added and for the last issues reported in the comments. Those can be addressed in the Sunflower release.
Tatsiana Hryhoryeva January 30, 2025 at 3:30 PM
From large bulk edit jobs done on Eureka bugfest environment today some of them completed successfully (upload, commit), some of them failed with errors, find the details below:
Upload 100K Instances in non-ECS stucked in progress with 100% (twice)
Upload ~100K MARC Instances via Query on non-ECS failed with “Socket closed“ error
Successful job is:
Bulk edit of 100K Holdings done successfully in Central tenant
Tatsiana Hryhoryeva January 29, 2025 at 2:11 PMEdited
From large bulk edit jobs done on Eureka bugfest environment today some of them completed successfully (upload, commit), some of them failed with errors (upload, commit), find the details below:
Upload of 100K Instances in Central tenant failed with 500 error
Upload of 100K MARC Instances in Central tenant failed with “Socket closed“ error
Upload of 100K Items in non-ECS via Query failed with “Socket closed“ error
Commit of 100K Instances in Central tenant with applying changes to holdings, items failed with “Connection reset“ error
Upload of 100K Holdings in Central tenant failed with “Socket closed“ error
Successful jobs are:
Bulk edit of 100K MARC Instances done successfully in Central tenant, file ~180MB downloaded
100K Instances uploaded successfully in Central tenant
100K Items uploaded and committed (with some timeout errors) successfully in non-ECS
Testing is going to be continued for different record types, on different tenants and for different actions used in bulk edit flow.
Tatsiana Hryhoryeva January 29, 2025 at 6:22 AM
As a part of early feedback verification done on Eureka bugfest environment for bulk edit of 40K Instances on non-ECS and 100K Instances on ECS, Central tenant.
Bulk edits completed successfully, however with some timeout errors on Confirmation screen, e.g. 276046cd-abe8-4ea4-9f3b-b2b0874a2d40,timeout executing PUT http://inventory/instances/276046cd-abe8-4ea4-9f3b-b2b0874a2d40
Testing is going to be continued for different record types, on different tenants and for different actions used in bulk edit flow.
Dima Tkachenko January 28, 2025 at 10:23 AMEdited
Successfully imported 3 file in parallel with time shift on BF: 50k then 70k then 100k.
please retest
Details
Details
Assignee
Pavel FilippovReporter
Tatsiana HryhoryevaPriority
P1
The issue noticed for all record type identifiers - Instances, Holdings, Items and Users. The issue reproduced occasionally on small dataset. The issue reproduced constantly on larger dataset ~10K+ file with identifiers
For Instances (bulk edit administrative data):
go to “Inventory“ app > Search & filter for any Instances (~few thousandths) > Click “Actions“ > “Save Instances UUIDs“
go to “Bulk edit“ app
select “Inventory-instances“ record type > “Instances UUIDs“ record identifier
upload .csv file with valid Instance identifiers
Expected result: file is uploaded successfully
Actual result: file upload fails, in DevTools error is displayed
"errorMessage": "[502 Bad Gateway] during [GET] to [http://inventory/instances?query=id%3D%3D83251b4d-dd1b-52d3-8ef3-b9251c82e589&limit=1] [InventoryInstancesClient#getInstanceByQuery(String,long)]: [<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n</body>\r\n</html>\r\n] (BadGateway)"
For Instances (MARC flow):
errors on commit stage
eaa9eb2f-d1c2-408d-b658-3f817c1fdd9b,io.vertx.core.impl.NoStackTraceThrowable: Failed to update MARC record in SRS, instanceId: 'eaa9eb2f-d1c2-408d-b658-3f817c1fdd9b', jobExecutionId: '50bf6264-b340-42cd-afc5-f1c4df0de97b', status code: 502, Record: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
</body>
</html>
null,io.vertx.core.impl.NoStackTraceThrowable: Failed to request records identifiers by matching criteria, responseStatus: '502', body: '<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
</body>
</html>
For Users:
go to “Bulk edit“ app
select “Users“ record type > “User Barcodes“ record identifier
upload .csv file with valid User identifiers (attached)
Expected result: file is uploaded successfully
Actual result: file upload fails, in DevTools error is displayed
"errorMessage": "[502 Bad Gateway] during [GET] to [http://users-keycloak/users/9eb67301-6f6e-468f-9b1a-6134dc39a684/permissions?desiredPermissions=bulk-operations.item.inventory.get&desiredPermissions=bulk-operations.item.users.get&desiredPermissions=users.item.get&desiredPermissions=inventory.items.item.get&desiredPermissions=inventory-storage.holdings.item.get&desiredPermissions=inventory.instances.item.get] [EurekaUserPermissionsClient#getPermissions(String,List)]: [<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n</body>\r\n</html>\r\n] (BadGateway)"
For Items:
"errorMessage": "[502 Bad Gateway] during [GET] to [http://inventory/items?query=id%3D%3Dff76c43b-12a0-46fa-8f49-dc3b72234973&limit=2147483647] [InventoryClient#getItemByQuery(String,long)]: [<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n</body>\r\n</html>\r\n] (BadGateway)"
For Holdings:
"errorMessage": "[500 Internal Server Error] during [GET] to [http://users-keycloak/users/af6382c5-983a-41bc-9d0d-82f2f07d6252/permissions?desiredPermissions=bulk-operations.item.inventory.get&desiredPermissions=bulk-operations.item.users.get&desiredPermissions=users.item.get&desiredPermissions=inventory.items.item.get&desiredPermissions=inventory-storage.holdings.item.get&desiredPermissions=inventory.instances.item.get] [EurekaUserPermissionsClient#getPermissions(String,List)]: [{\"errors\":[{\"message\":\"[500 Internal Server Error] during [GET] to [http://permissions/users/af6382c5-983a-41bc-9d0d-82f2f07d6252?onlyVisible=false&desiredPermissions=bulk-operations.item.inventory.ge... (804 bytes)] (InternalServerError)"
errors on commit stage
0e01a699-e498-5f05-ba6c-4c996dd53a24,[502 Bad Gateway] during [GET] to [http://users-keycloak/users/51ce3975-77c0-49d7-bf64-54186ead4d16/permissions?desiredPermissions=bulk-operations.item.inventory.put&desiredPermissions=bulk-operations.item.users.put&desiredPermissions=users.item.put&desiredPermissions=inventory.items.item.put&desiredPermissions=inventory-storage.holdings.item.put&desiredPermissions=inventory.instances.item.put] [EurekaUserPermissionsClient#getPermissions(String,List)]: [<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
</body>
</html>
]
231896de-fbd8-5bd0-9019-a9c34897cc60,[503 Service Unavailable] during [PUT] to [http://holdings-storage/holdings/231896de-fbd8-5bd0-9019-a9c34897cc60] [HoldingsClient#updateHoldingsRecord(HoldingsRecord,String)]: [<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
</body>
</html>
]