[FOLIO-2992] Snapshot environment is unavailable Created: 29/Jan/21 Updated: 17/Mar/21 Resolved: 29/Jan/21 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Task | Priority: | TBD |
| Reporter: | Marc Johnson | Assignee: | John Malconian |
| Resolution: | Done | Votes: | 0 |
| Labels: | environments, outage | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Sprint: | |
| Development Team: | None |
| Description |
|
Charlotte Whitt reported that the snapshot environment is unavailable. The last error in the logs from the latest build:
TASK [build and start stripes docker container] ********************************
fatal: [10.36.1.140]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'stripes_port' is undefined\n\nThe error appears to be in '/home/jenkins/workspace/FOLIO_Reference_Builds/folio-snapshot/folio-infrastructure/CI/ansible/rebuild-stripes.yml': line 53, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: build and start stripes docker container\n ^ here\n"}
What is slightly odd about this error is that it occurs after the build has been declared stable + ./aws_tag_stable.sh folio_snapshot folio_snapshot and during the UI regression test stage of the build according to the summary. |
| Comments |
| Comment by Marc Johnson [ 29/Jan/21 ] |
|
John Malconian Ian Hardy Zak Burke I think I'm going to need some help diagnosing this. Are the UI regression tests building a separate stripes container purely for the tests (and this is a false failure) or is this where the main UI is built / re-built (after the environment is considered stable and flipped over, meaning the test setup could wipe out the environment)? |
| Comment by Marc Johnson [ 29/Jan/21 ] |
|
The snapshot-core environment is similarly effected, however snapshot-load and testing are not. Neither of which run the regression tests suggesting the cause is in that stage. |
| Comment by Zak Burke [ 29/Jan/21 ] |
|
A change in the behavior of integration tests revealed a flaw in a portion of the post-integration-test pipeline that had not been run in some time. I reverted the change to the integration tests to avoid execution of the buggy branch of the post-integration-test pipeline, and John Malconian has since fixed that branch so I will un-revert the integration test change, i.e. once again move ahead with
|
| Comment by John Malconian [ 29/Jan/21 ] |
|
The issue is related to a change that occurred a while back but hasn't surfaced until now because, until now, we haven't had a successful UI regression test result. Zak's changes to the UI regression tests yesterday give the impression of successful test results and therefore trigger the folio-snapshot->folio-snapshot-stable phase where stripes is rebuilt with a new okapi URL. I've fixed the stripes_port error so this shouldn't be an issue going forward. |