[FOLIO-2731] Update reference build OS to Ubuntu Focal and OpenJDK 11 Created: 17/Aug/20 Updated: 04/Sep/20 Resolved: 04/Sep/20 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Task | Priority: | TBD |
| Reporter: | John Malconian | Assignee: | Wayne Schneider |
| Resolution: | Done | Votes: | 0 |
| Labels: | devops-backlog | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||
| Sprint: | DevOps: Sprint 95, DevOps: Sprint 96 | ||||||||
| Development Team: | FOLIO DevOps | ||||||||
| Description |
|
Existing reference builds install Ubuntu Xenial and OpenJDK 8 and the Okapi Debian package relies on the locally installed OpenJDK. Okapi 4.x requires OpenJDK 11. This is good time to update the reference OS to Focal and OpenJDK 11. |
| Comments |
| Comment by Wayne Schneider [ 18/Aug/20 ] |
|
We will also need to update the base image for the Vagrant build. |
| Comment by Wayne Schneider [ 01/Sep/20 ] |
|
This turns out to be a bit of a rabbit hole. First attempts to run a folio-snapshot build resulted in an error:
TASK [common : create folio group] *********************************************
fatal: [10.36.1.111]: FAILED! => {"changed": false, "module_stderr": "<stdin>:17: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\nTraceback (most recent call last):\n File \"/tmp/ansible_group_payload_jimnbqyf/ansible_group_payload.zip/ansible/module_utils/basic.py\", line 279, in get_distribution\nAttributeError: module 'platform' has no attribute '_supported_dists'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"<stdin>\", line 113, in <module>\n File \"<stdin>\", line 105, in _ansiballz_main\n File \"<stdin>\", line 48, in invoke_module\n File \"/usr/lib/python3.8/imp.py\", line 234, in load_module\n return load_source(name, filename, file)\n File \"/usr/lib/python3.8/imp.py\", line 169, in load_source\n module = _exec(spec, sys.modules[name])\n File \"<frozen importlib._bootstrap>\", line 604, in _exec\n File \"<frozen importlib._bootstrap_external>\", line 783, in exec_module\n File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n File \"/tmp/ansible_group_payload_jimnbqyf/__main__.py\", line 501, in <module>\n File \"/tmp/ansible_group_payload_jimnbqyf/__main__.py\", line 449, in main\n File \"/tmp/ansible_group_payload_jimnbqyf/__main__.py\", line 89, in __new__\n File \"/tmp/ansible_group_payload_jimnbqyf/ansible_group_payload.zip/ansible/module_utils/basic.py\", line 337, in load_platform_subclass\n File \"/tmp/ansible_group_payload_jimnbqyf/ansible_group_payload.zip/ansible/module_utils/basic.py\", line 289, in get_distribution\nAttributeError: module 'platform' has no attribute 'dist'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
to retry, use: --limit @*****-snapshot-test/folio-infrastructure/CI/ansible/folio-snapshot-test-builds.retry
Research indicated that the issue may be that we need to use Python 3 for Ansible on both the control host (jenkins-slave-all) and the target host. The java-11 tagged version of folioci/jenkins-slave-all is based on Focal and uses Python 3 for Ansible (thanks, Ian Hardy). Unfortunately, the ec2.py dynamic inventory script fails with: [WARNING]: * Failed to parse *****- snapshot-test/folio-infrastructure/CI/ansible/inventory/ec2.py with script plugin: Inventory script (*****-snapshot- test/folio-infrastructure/CI/ansible/inventory/ec2.py) had an execution error: /usr/bin/env: ‘python’: No such file or directory This is because Focal does not include Python 2.x by default, and the executable for Python 3.x is not python. Currently attempting to rebuild jenkins-slave-all image with the python-is-python3 package. |
| Comment by Ian Hardy [ 02/Sep/20 ] |
|
Hi Wayne, wonder if its worth updating the ec2.py script. Here's the version that comes w/ansibel 2.7 stable https://raw.githubusercontent.com/ansible/ansible/stable-2.7/contrib/inventory/ec2.py. Assuming python-is-python3 is some kind of symlink which is probably more globally helpful, but could also change the shebang to python3. |
| Comment by Wayne Schneider [ 02/Sep/20 ] |
|
Ian Hardy thanks, updating the ec2.py script was my next thing to try. The latest run threw a new error, however:
TASK [launch_ec2_instance : launch ec2 instance] *******************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'image_id'\n\nThe error appears to have been in '*****-snapshot-test/folio-infrastructure/CI/ansible/roles/launch_ec2_instance/tasks/main.yml': line 3, column 4, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: launch ec2 instance\n ^ here\n"}
to retry, use: --limit @*****-snapshot-test/folio-infrastructure/CI/ansible/folio-snapshot-test-builds.retry
|
| Comment by Ian Hardy [ 02/Sep/20 ] |
|
Maybe this is empty? https://github.com/folio-org-priv/folio-infrastructure/blob/master/CI/ansible/launch_ec2.yml#L36 Looking at the role a bit you can pass an ami id to either image_id or image. |
| Comment by Wayne Schneider [ 02/Sep/20 ] |
|
Something doesn't make sense:
TASK [debug] *******************************************************************
ok: [localhost] => {
"latest_ami": "{'architecture': 'x86_64', 'creation_date': '2020-08-18T17:26:37.000Z', 'image_id': 'ami-05cf2c352da0bfb2e', 'image_location': '099720109477/ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20200817', 'image_type': 'machine', 'public': True, 'owner_id': '099720109477', 'platform_details': 'Linux/UNIX', 'usage_operation': 'RunInstances', 'state': 'available', 'block_device_mappings': [{'device_name': '/dev/sda1', 'ebs': {'delete_on_termination': True, 'snapshot_id': 'snap-02e73e95ff3e474e4', 'volume_size': 8, 'volume_type': 'gp2', 'encrypted': False}}, {'device_name': '/dev/sdb', 'virtual_name': 'ephemeral0'}, {'device_name': '/dev/sdc', 'virtual_name': 'ephemeral1'}], 'description': 'Canonical, Ubuntu, 20.04 LTS, amd64 focal image build on 2020-08-17', 'ena_support': True, 'hypervisor': 'xen', 'name': 'ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20200817', 'root_device_name': '/dev/sda1', 'root_device_type': 'ebs', 'sriov_net_support': 'simple', 'virtualization_type': 'hvm', 'tags': {}}"
}
TASK [debug] *******************************************************************
ok: [localhost] => {
"latest_ami.image_id": "VARIABLE IS NOT DEFINED!"
}
|
| Comment by Ian Hardy [ 02/Sep/20 ] |
|
wat. My only thought is latest_ami is being treated as a string and not a dictionary. |
| Comment by Wayne Schneider [ 03/Sep/20 ] |
|
Upgrading to Ansible v2.9.13 in the jenkins-slave-all container resulted in a successful test build. Warnings: [WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details [DEPRECATION WARNING]: The 'ec2_ami_facts' module has been renamed to 'ec2_ami_info'. This feature will be removed in version 2.13. [DEPRECATION WARNING]: The 'docker_service' module has been renamed to 'docker_compose'.. This feature will be removed in version 2.12. So I think no emergencies (whew). |
| Comment by Wayne Schneider [ 03/Sep/20 ] |
|
Changes merged into master of folio-infrastructure and folio-tools, new Docker image available as folioci/jenkins-slave-all:java-11. folio-snapshot and folio-testing builds have been testing. Putting issue "In Review" for builds to run overnight. |
| Comment by Wayne Schneider [ 04/Sep/20 ] |
|
Builds look good, closing the issue. |