[FOLIO-2659] Add "sleep" statement to end of playbooks to allow logging of all Ansible errors Created: 26/Jun/20 Updated: 26/Jun/20 |
|
| Status: | Open |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Task | Priority: | TBD |
| Reporter: | Wayne Schneider | Assignee: | Unassigned |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | devops, devops-backlog | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Sprint: | |
| Development Team: | FOLIO DevOps |
| Description |
|
Since the latest Jenkins upgrade, when a Jenkins job fails while executing an Ansible playbook, you get the following not very specific error: FATAL: command execution failed hudson.AbortException: Ansible playbook execution failed at org.jenkinsci.plugins.ansible.AnsiblePlaybookBuilder.perform(AnsiblePlaybookBuilder.java:262) at org.jenkinsci.plugins.ansible.workflow.AnsiblePlaybookStep$AnsiblePlaybookExecution.run(AnsiblePlaybookStep.java:400) at org.jenkinsci.plugins.ansible.workflow.AnsiblePlaybookStep$AnsiblePlaybookExecution.run(AnsiblePlaybookStep.java:321) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47) at hudson.security.ACL.impersonate(ACL.java:367) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Unfortunately, the Ansible error that causes the failure is often not written to the log before job execution halts, making troubleshooting very difficult. This is an open bug in Jenkins (https://issues.jenkins-ci.org/browse/JENKINS-54557). One suggested workaround is to add a sleep command to the cleanup phase of the job, to allow log writing to complete, e.g.:
post {
always {
sendNotifications currentBuild.result
}
cleanup {
echo "Sleeping during cleanup to enable ansible error reporting."
sleep 5
}
}
This has been successful in testing. It would probably be good to roll this out across any job using the Ansible plugin. |