[FOLIO-2659] Add "sleep" statement to end of playbooks to allow logging of all Ansible errors Created: 26/Jun/20  Updated: 26/Jun/20

Status: Open
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: TBD
Reporter: Wayne Schneider Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: devops, devops-backlog
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Sprint:
Development Team: FOLIO DevOps

 Description   

Since the latest Jenkins upgrade, when a Jenkins job fails while executing an Ansible playbook, you get the following not very specific error:

FATAL: command execution failed
hudson.AbortException: Ansible playbook execution failed
	at org.jenkinsci.plugins.ansible.AnsiblePlaybookBuilder.perform(AnsiblePlaybookBuilder.java:262)
	at org.jenkinsci.plugins.ansible.workflow.AnsiblePlaybookStep$AnsiblePlaybookExecution.run(AnsiblePlaybookStep.java:400)
	at org.jenkinsci.plugins.ansible.workflow.AnsiblePlaybookStep$AnsiblePlaybookExecution.run(AnsiblePlaybookStep.java:321)
	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
	at hudson.security.ACL.impersonate(ACL.java:367)
	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Unfortunately, the Ansible error that causes the failure is often not written to the log before job execution halts, making troubleshooting very difficult.

This is an open bug in Jenkins (https://issues.jenkins-ci.org/browse/JENKINS-54557). One suggested workaround is to add a sleep command to the cleanup phase of the job, to allow log writing to complete, e.g.:

  post {
    always {
      sendNotifications currentBuild.result
    }
    cleanup {
      echo "Sleeping during cleanup to enable ansible error reporting."
      sleep 5
    }
  }

This has been successful in testing. It would probably be good to roll this out across any job using the Ansible plugin.


Generated at Thu Feb 08 23:22:18 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.