[FOLIO-1775] SPIKE: establish acceptance criteria for devops tasks Created: 04/Feb/19  Updated: 03/Jun/20  Resolved: 08/Apr/19

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P3
Reporter: Jakub Skoczen Assignee: David Crossley
Resolution: Done Votes: 0
Labels: devdoc, platform-backlog
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Sprint: Core: Platform - Sprint 60, Core: Platform - Sprint 58, Core: Platform - Sprint 59
Story Points: 3
Development Team: Core: Platform

 Description   

We don't have a good way to verify completeness and success of many devops tasks. Especially, changes in the CI/CD pipelines or build pipelines that execute as a result of an upstream (source) project change. It would be great if we could come up with a way to test and verify correctness for performed devops changes indpendently of the upstream projects and integration environments.

1. We have decided that the starting point is for John Malconian to describe the process for verifying Jenkins changes. This description should be published on dev.folio.org.

Some idea (John Malconian Wayne Schneider please edit this section if you have better ideas about what should be done to test)

  • find a way to test Jenkins pipelines (in isolation) – are there any tools for that?
  • set up dummy (mock) project for UI and backend where we could trigger source changes on demand and test CI integration
  • set up an internal testing environment where we can test and verify CI changes before rolling them out to public environments


 Comments   
Comment by John Malconian [ 06/Feb/19 ]

The following describes how two common FOLIO DevOps related scenarios are tested before they are "rolled out" into production.

1. The first scenario involves changes or new Ansible-related features to folio-ansible or folio-infrastructure. These two repos consist of playbooks and roles that automate the provisioning of various FOLIO instances and services. They are used extensively in our development, test, and CI environments. Common DevOps tasks could include adding new modules or wrangling sample data into a FOLIO build environment, adding various options to existing roles to extend functionality, and adding new roles to meet new requirements. Work is typically performed in a development/feature branch of either folio-ansible or folio-infrastructure and manually tested by running the Ansible playbooks in this branch on a local development system or on the host, jenkins-aws.indexdata.com. The latter allows us to test changes that involve use of the AWS infrastructure. For example, changes that would impact folio-snapshot are tested by performing a build of "folio-snapshot-test" which would build a folio-snapshot system from either a branch of folio-ansible or folio-infrastructure. If the test build completes successfully, further examination of the instance might be necessary depending on the nature of the changes. If everything checks out, the new code is merged to the master branches of folio-ansible/infrastructure.

2. The second scenario involves changes to Jenkins Pipelines shared libraries. Jenkins pipelines consists of scripts that automate the build steps and quality gates for individual repo branch and PR builds. Most of these functions are stored in a Jenkins "shared library" in the repository jenkins-pipeline-libs. Typically, changes are performed on a dev/feature branch of jenkins-pipeline-libs. A common method to test changes in our development branch is to create a test branch in an existing module repository. The Jenkinsfile in the module's test branch is updated to point to the branch of jenkins-pipeline-libs we are testing. An example:

We are testing a change to the pipeline that builds, tests, and deploys Node/NPM modules in a test branch of jenkins-pipeline-libs called 'FOLIO-1775'. Good candidate modules to test these changes might be stripes-components or ui-users. Branches for each these modules are created called 'FOLIO-1775-test'. We add the following line to the top of the Jenkinsfile in each of these branches in order to test the changes that reside in our FOLIO-1775 Closed branch of jenkins-pipeline-libs:

@Library ('folio_jenkins_shared_libs@FOLIO-1775') _

We can then initiate a build of the branch FOLIO-1775-test via Jenkins to test our changes. If our changes are related to actions taken only during Pull Requests, we could open PRs in stripes-core and ui-users for our test branches in order to simulate these changes. When finished with testing, the PRs are closed (and not merged). There are limitations to this approach, however. For example, actions that occur only when building on a repository's master branch or when performing release builds for modules. These are often more difficult to test since testing on a module's master branch might introduce problems into that branch.

Comment by David Crossley [ 08/Mar/19 ]

The document with the above-mentioned content is at:
https://dev.folio.org/guides/devops-verify-task-acceptance/

Comment by Jakub Skoczen [ 08/Apr/19 ]

I have added a link to acceptance criteria from the DevOps DoD page.

Generated at Thu Feb 08 23:15:49 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.