|
Before this step occurs in the pipeline, communication between jenkins build agent and the jenkins server is fine. Just so happens that the VPC of the K8s cluster runs in is assigned 172.17.0.0/16. It also just so happens that Docker assigns its docker0 bridge an IP from the very same 172.17 class B address by default. Modern docker versions are smart enough, however, that if it detects that 172.17 is already in use by another interface, it assigns a different class B address. For example, when I enabled the docker bridge (docker0 interface) on the worker node hosts as a step in troubleshooting this issue, it assigned it an IP from 172.18 so it doesn't conflict. However, when the Rancher Docker-in-Docker container is invoked to build a docker image in the build agent pod, it's not so smart and creates a docker0 interface in the pod and assigns it a 172.17 address. At this point, routing from build agent pod to any other K8s pod is borked.
It is not possible to change the cidr block for the AWS VPC, or at least not easily where the cluster can utilize it or the configuration of the Rancher D-in-D container, so a new cluster on a new VPC needs to be created to resolve this. Will create separate Jira issue for that.
|