[FOLIO-1624] Publishing and fetching docker artefacts fails Created: 24/Nov/18 Updated: 03/Jun/20 Resolved: 22/Jan/19 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Task | Priority: | P3 |
| Reporter: | Marc Johnson | Assignee: | Marc Johnson |
| Resolution: | Done | Votes: | 0 |
| Labels: | devops, platform-backlog, sprint52, sprint53, sprint54 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||
| Sprint: | |||||||||
| Development Team: | Core: Platform | ||||||||
| Description |
|
The overnight environment builds (folio-testing-backend, folio-snapshot) appear to fail due to not being able to fetch docker artefacts. Whilst the modules referred to are different, the error messages seem to be similar. As an example: TASK [folio-ansible/roles/okapi-tenant-deploy : Post install list for deployment and enabling] *** fatal: [10.36.1.223]: FAILED! => {"changed": false, "connection": "close", "content": "http://localhost:4243/images/folioci/mod-configuration:5.0.2-SNAPSHOT.50/json HTTP error 404\n{\"message\":\"no such image: folioci/mod-configuration:5.0.2-SNAPSHOT.50: No such image: folioci/mod-configuration:5.0.2-SNAPSHOT.50\"}\n", "content_length": "226", "content_type": "text/plain", "msg": "Status code was 400 and not [200]: HTTP Error 400: Bad Request", "redirected": false, "status": 400, "url": "http://10.36.1.223:9130/_/proxy/tenants/diku/install?deploy=true"} Snapshot
failed: [10.36.1.185] (item=mod-feesfines-15.0.2-SNAPSHOT.18) => {"changed": false, "item": "mod-feesfines-15.0.2-SNAPSHOT.18", "msg": "Error pulling image folioci/mod-feesfines:15.0.2-SNAPSHOT.18 - 404 Client Error: Not Found (\"{\"message\":\"error parsing HTTP 404 response body: invalid character 'p' after top-level value: \\\"404 page not found\\\\n\\\"\"}\")"}
changed: [10.36.1.185] => (item=mod-vendors-1.0.2-SNAPSHOT.27)
changed: [10.36.1.185] => (item=mod-codex-inventory-1.4.0-SNAPSHOT.65)
FATAL: command execution failed
Publishing of docker images also appears to fail. For example the [mod-inventory-storage 13.1.0 release build] or mod-inventory-storage-master build: d18b0764eff2: Waiting unauthorized: authentication required [Pipeline] } [Pipeline] // withDockerRegistry [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // dir [Pipeline] echo script returned exit code 1 Master e620de361ca9: Pushed blob upload unknown [Pipeline] } [Pipeline] // withDockerRegistry [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // dir [Pipeline] echo script returned exit code 1 |
| Comments |
| Comment by Marc Johnson [ 24/Nov/18 ] |
|
I've disabled the stripes testing hourly build, in order to avoid continued failures. |
| Comment by Marc Johnson [ 24/Nov/18 ] |
|
This appears to be resolved. The release and master builds have been re-run successfully. As has the folio-testing-backend environment build. Am rebuilding the folio-testing-backend VM and folio-snapshot environment. |
| Comment by Marc Johnson [ 24/Nov/18 ] |
|
It would appear that I spoke too soon about this being fully resolved (or there is a different, related issue). The re-ran snapshot and testing-backend builds failed: Snapshot environment
TASK [folio-ansible/roles/okapi-deploy-config : Pull Docker images] ************
changed: [10.36.1.139] => (item=mod-users-15.3.0-SNAPSHOT.56)
changed: [10.36.1.139] => (item=mod-permissions-5.4.0-SNAPSHOT.34)
changed: [10.36.1.139] => (item=mod-login-4.5.1-SNAPSHOT.32)
changed: [10.36.1.139] => (item=mod-inventory-storage-13.3.0-SNAPSHOT.203)
changed: [10.36.1.139] => (item=mod-password-validator-1.0.1-SNAPSHOT.12)
failed: [10.36.1.139] (item=mod-users-bl-4.0.4-SNAPSHOT.38) => {"changed": false, "item": "mod-users-bl-4.0.4-SNAPSHOT.38", "msg": "Error pulling image folioci/mod-users-bl:4.0.4-SNAPSHOT.38 - 404 Client Error: Not Found (\"{\"message\":\"error parsing HTTP 404 response body: invalid character 'p' after top-level value: \\\"404 page not found\\\\n\\\"\"}\")"}
failed: [10.36.1.139] (item=mod-authtoken-2.0.2-SNAPSHOT.35) => {"changed": false, "item": "mod-authtoken-2.0.2-SNAPSHOT.35", "msg": "Error pulling image folioci/mod-authtoken:2.0.2-SNAPSHOT.35 - 404 Client Error: Not Found (\"{\"message\":\"error parsing HTTP 404 response body: invalid character 'p' after top-level value: \\\"404 page not found\\\\n\\\"\"}\")"}
changed: [10.36.1.139] => (item=mod-configuration-5.0.2-SNAPSHOT.50)
failed: [10.36.1.139] (item=mod-calendar-1.0.5-SNAPSHOT.63) => {"changed": false, "item": "mod-calendar-1.0.5-SNAPSHOT.63", "msg": "Error pulling image folioci/mod-calendar:1.0.5-SNAPSHOT.63 - 404 Client Error: Not Found (\"{\"message\":\"error parsing HTTP 404 response body: invalid character 'p' after top-level value: \\\"404 page not found\\\\n\\\"\"}\")"}
Testing backend [0;32m testing-backend: fatal: [default]: FAILED! => {"changed": false, "connection": "close", "content": "http://localhost:4243/images/folioci/mod-user-import:3.1.1-SNAPSHOT.31/json HTTP error 404\n{\"message\":\"no such image: folioci/mod-user-import:3.1.1-SNAPSHOT.31: No such image: folioci/mod-user-import:3.1.1-SNAPSHOT.31\"}\n", "content_length": "220", "content_type": "text/plain", "msg": "Status code was not [200]: HTTP Error 400: Bad Request", "redirected": false, "status": 400, "url": "http://10.0.2.15:9130/_/proxy/tenants/diku/install?deploy=true"}[0m |
| Comment by Marc Johnson [ 26/Nov/18 ] |
|
This seems intermittent, most of the overnight builds succeeded last night, except the testing-backend (a second run succeeded) [0;32m testing-backend: TASK [okapi-tenant-deploy : Post install list for deployment and enabling] *****[0m [0;32m testing-backend: fatal: [default]: FAILED! => {"changed": false, "connection": "close", "content": "http://localhost:4243/images/folioci/mod-permissions:5.4.0-SNAPSHOT.34/json HTTP error 404\n{\"message\":\"no such image: folioci/mod-permissions:5.4.0-SNAPSHOT.34: No such image: folioci/mod-permissions:5.4.0-SNAPSHOT.34\"}\n", "content_length": "220", "content_type": "text/plain", "msg": "Status code was not [200]: HTTP Error 400: Bad Request", "redirected": false, "status": 400, "url": "http://10.0.2.15:9130/_/proxy/tenants/diku/install?deploy=true"}[0m Maybe it only affects some agents in our CI setup? Or maybe it is based upon when builds run? |
| Comment by Jakub Skoczen [ 22/Jan/19 ] |
|
Marc Johnson is this still an open issue? I am closing – reopen if its not fixed. |