[FOLIO-3672] Jenkins build node broken after OS updates Created: 22/Dec/22 Updated: 23/Jan/23 Resolved: 23/Jan/23 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Bug | Priority: | TBD |
| Reporter: | John Malconian | Assignee: | John Malconian |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Sprint: | DevOps Sprint 155, DevOps Sprint 157 |
| Development Team: | FOLIO DevOps |
| RCA Group: | TBD |
| Description |
|
While investigating another potential issue, OS patches were applied to the jenkins build node called folio-jenkins-slave02. After rebooting the system, jenkins is unable to launch jenkins builds in containers on the node. The error is: There is insufficient memory for the Java Runtime Environment to continue. |
| Comments |
| Comment by John Malconian [ 22/Dec/22 ] |
|
Was able to confirm that Java within both the java-11 and java-17 based build containers does not work. docker run -it --rm folioci/jenkins-slave-all:java-17 /bin/bash root@52380d8833db:/home/jenkins# java -version [0.003s][warning][os,thread] Failed to start thread "GC Thread#0" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached. # # There is insufficient memory for the Java Runtime Environment to continue. # Cannot create worker GC thread. Out of system resources. # An error report file with more information is saved as: # /home/jenkins/hs_err_pid10.log Looks like this issue. https://github.com/adoptium/containers/issues/215#issuecomment-1142046045 I've implemented a workaround by configuring Jenkins to run docker with the option '--security-opt seccomp=unconfined' on folio-jenkins-slave02. This is not idea and should only be temporary. The best solution is to upgrade the build nodes to Jammy. |
| Comment by Jakub Skoczen [ 23/Jan/23 ] |
|
Please cratea a ticket to remove the workaround John Malconian |