[FOLIO-3344] Clean up Jenkins workspaces after scratch environment build jobs Created: 15/Nov/21 Updated: 18/Nov/21 Resolved: 18/Nov/21 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Task | Priority: | TBD |
| Reporter: | John Malconian | Assignee: | Hleb Surnovich |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Attachments: |
|
| Sprint: | DevOps Sprint 127 |
| Development Team: | FOLIO DevOps |
| Description |
|
Several jobs in the Jenkins "Scratch (Rancher) environment" folder are not cleaning up their workspaces after the job is run which results in disk filling up on the Jenkins server. These jobs include the manage-scratch-environment job and it's associated variations (1.1, 1.2, Firebird manage scratch env, etc). A clean up of the workspace should be performed whether the job was successful, failed, or terminated. A step similar to the following should be added to these Jenkins pipeline jobs: finally {
stage('Clean') {
cleanWs disableDeferredWipeout: true, deleteDirs: true, cleanWhenAborted: true, cleanWhenFailure: true, cleanWhenSuccess: true
|
| Comments |
| Comment by Hleb Surnovich [ 17/Nov/21 ] |
|
Hello, John Malconian ! I took this ticket and tried to apply cleaning with CleanWS, as you mentioned above.
|
| Comment by John Malconian [ 17/Nov/21 ] |
|
Thanks for taking a look at this Hleb Surnovich. I ended up just adding a cron job on the server that will clean up these files. It run's once per day. I think that should be ok for now. Feel free to close this issue. In general, I think it would preferable to run these jobs inside a Docker container like we do for most other jobs. That way, when the job is complete, the Jenkins workspace environment is automatically deleted. On the other hand, if you are using Terraform in these jobs I'm guessing that you have Terraform state files that need to persist and that may prohibit the use of a container. If so, persisting them to an S3 bucket might be an option instead of on the local filesystem. Just something to think about. |
| Comment by Dilshod_Khusanov [ 18/Nov/21 ] |
|
Closed as solution is provided by John Malconian. |