[FOLIO-691] Create Jenkins job to run packer build of folio/stable and folio/testing boxes Created: 22/Jun/17  Updated: 12/Nov/18  Resolved: 21/Jul/17

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: New Feature Priority: P2
Reporter: Wayne Schneider Assignee: Wayne Schneider
Resolution: Done Votes: 0
Labels: ci, for-next-sprint, sprint17, sprint18
Remaining Estimate: Not Specified
Time Spent: 2 days, 3 hours, 45 minutes
Original estimate: Not Specified

Issue links:
Blocks
blocks FOLIO-669 Fully automate builds of "stable" and... Closed
is blocked by FOLIO-692 Build server for packer builds Closed
Relates
relates to FOLIO-702 Create/test folio/stable v2.0.0 Closed
Sprint:

 Description   

Will require running on a bare-metal server for VirtualBox



 Comments   
Comment by Wayne Schneider [ 30/Jun/17 ]

Updated and tested new packer.json in folio-ansible. Environment on folio-jenkins-de in /home/wayne/folio-ansible. Command to build:

$ packer build -var-file variables.json packer/packer.json

Build targets are stable,testing, and testing-backend. packer build will build Vagrant boxes and push them to Vagrant Cloud.

May need to look at using Vault for storing the Vagrant Cloud key.

Comment by Wayne Schneider [ 11/Jul/17 ]

This is working pretty well. I set up packer.json with 4 targets, stable, stable-backend, testing, and testing-backend. Some stuff is controlled through a variables.json file that is in the /home/wayne/folio-ansible directory (mode 600, since it contains a key for Vagrant Cloud). I think there need to be 3 jobs, currently commands all executed from /home/wayne/folio-ansible, obviously that isn't a good permanent solution:
1. Daily build of testing and testing-backend:

packer build -var-file variables.json -only testing,testing-backend packer/packer.json

2. Build of stable (triggered on update of group_vars/stable in folio-ansible?):

packer build -var-file variables.json -only stable packer/packer.json

3. Build of stable-backend (triggered on update of group_vars/stable-backend?):

packer build -var-file variables.json -only stable-backend packer/packer.json

Currently I'm running the commands myself. The jobs for stable and stable-backend may need some thinking, because the version and release notes are currently set in the variables.json file, and would need to be updated before a new job is run.

Comment by John Malconian [ 12/Jul/17 ]

I'll start on a Jenkins pipeline script to automate the above. In the meantime, let's move the packer stuff into the
folio-infrastructure/CI/packer of the folio-infrastructure project.

Comment by Wayne Schneider [ 12/Jul/17 ]

Plan is to move packer config into folio-infrastructure repository, parse version information from group_vars files.

Comment by Wayne Schneider [ 18/Jul/17 ]

First draft of pipeline: https://jenkins-aws.indexdata.com/job/Automation/job/folio-blackbox/

Comment by John Malconian [ 19/Jul/17 ]

Just a rehash of Slack chat... Following items are outstanding after first draft.

  • Replace Wayne's personal Atlas token in Jenkins with new one.
  • Box tags and box descriptions need to be configurable. Re-implement package.json?
  • Versions stored and parsed from Ansible group_var/stable and group_var/testing
  • Datestamp can be appended to Semver versions.
  • Set up 'post' build notifications in Jenkinsfile
  • Add Jenkinsfile to git (same dir as packer)
  • Job scheduling
Comment by Wayne Schneider [ 20/Jul/17 ]

All of the above is done except for scheduling and configuration of box tags.

Notes:

  • Box tags can be configured by updating the Jenkinsfile, but that's a little heavy. Do we want it to be parameterized?
  • The Atlas token is for the folio-atlas Vagrant Cloud user.
  • Versions and version descriptions are stored in the group_vars files in folio-ansible.
  • post build email notifications are being sent to me. Also send to John Malconian?
Comment by John Malconian [ 20/Jul/17 ]

maybe store the box tags in the group_vars files. Can also parameterize them in Jenkins with default values. You can send post build email notifications to folio-jenkins@indexdata.com (that's John, Wayne, and Charles).

Comment by Wayne Schneider [ 20/Jul/17 ]

OK, they are parameterized in Jenkins with defaults, so you can change on the fly if you need to. Post build notifications have been updated.

I changed how the FOLIO_BUILD parameter works – it now takes a comma-delimited list of builds, e.g. testing,testing-backend. The possible values are stable, stable-backend, testing,testing-backend, and minimal. I'm not planning on keeping the folio/minimal box up to date, unless someone sees value in an Okapi-only Vagrant box, I just put it there to have something reasonably quick to build for testing.

I think the only thing left is scheduling. We may need to look into the Parameterized Scheduler Plugin. IDK if we want any other triggers to schedule a build.

Comment by John Malconian [ 21/Jul/17 ]

I set the FOLIO_BUILD parameter to default to 'testing,testing-backend' in the 'folio-blackbox' jenkins job. The job to run is scheduled to run between 1-3am UTC daily. We could get clever and initiate the job only if folio-testing-backend01 completes successfully (or vice versa), but let's keep it simple for now. For now, we will run a folio-stable build manually.

I also made some minor updates to the status notifications in the pipeline script and committed the changes.

Generated at Thu Feb 08 23:07:39 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.