Automated builds for FOLIO 'release'

Description

Summary

Continuously integrate and test new releases of both front-end and server-side components into
a current and stable FOLIO instance that is accessible to POs, developers, and manual testers.

Description

Implement automated build pipelines for stripes platforms (platform-core and platform-complete)
designed to integrate newly released modules into a stable FOLIO instance based solely on
"released" versions of artifacts. PRs will be submitted in order add and test newly released
stripes and UI modules into the on-going, stable release branch (master) of each platform's git
repository. The PRs can either be manually created or generated automatically by an NPM dependency
bot app that integrates with Github like Renovate or Greenhopper. These bots check for NPM
dependency updates for specified NPM packages and repositories and automatally initiate
pull requests to merge updated dependencies. Each PR will first trigger a build of the stripes
platform, and, if successful will then attempt to build a FOLIO full stack by utlizing Okapi's
"install" endpoint to resolve required interface dependencies provided by released server-side
modules. If the build is successful, the UI integration test suite is run against the built
FOLIO instance. If all tests pass, the PR can be then be merged to the stable release branch
of the platform which will, in turn, can be used to build a publically accessible current and
stable instance of FOLIO that can be used for additional testing/demos, etc.

The PR test process includes the following quality gates:

  • Successful webpack build of the stripes platform. If the stripes platform cannot be built,
    then the PR will fail.

  • Successful interface dependency resolution between frontend and backend modules (releases only).
    If the new frontend module requires an interface that cannot be resolved by a released backend
    module, then PR will fail. This will ensure that frontend and backend module releases are
    coordinated.

  • Successful build of FOLIO instance to ensure installation processes are configured and working
    as expected and tenent initialization is successful.

  • Ensure passing UI integration tests to indicate either successful integration of the new module
    into the platform or to ensure tests are kept and current and compatible with new functionality.

Tools/Platforms

Each platform git repository will be integated with a Jenkins pipeline responsible for coordinating
builds and testing using additional tools like folio-ansible and AWS. Each platform will also
be configured with a NPM dependency bot like Renovate to provide updates to the platform and
initiate PRs.

Additional Outputs

  • A yarn.lock file representing a stable set of fixed NPM dependencies committed to the platform's
    stable, release branch.

  • install.json files committed to each platform's stable, release branch that can be used to
    build a FOLIO system from the most recent set of released and stable module.

Prerequisites/Considerations

  • Front-end and backend module releases must occur on a regular, continuous basis in order
    for this process to work. Integration is more successful at regular, steady intervals (see FOLIO-1614)

  • An existing set of released compatible modules must be exist in order to build initial system
    (initial set to be based on Q4 release).

  • AWS resource utilization. We need to design a system that is cognizant of AWS resource
    utilization. Each PR will utilize an AWS EC2 instance and attached storage to build a
    FOLIO stack. Additionally, the process could be simplified to initialize a new tenant
    for each PR on an existing okapi cluster as long as larger number of released backend modules
    can be distributed across a cluster. Such a setup would most likely require additional
    orchestration tools, however. These ideas can be explored at a latter stage.

  • Determine how new backend module releases that are not explicitly required by frontend
    modules will be integrated into stable, release system.

  • Some Renovate initiated changes will fail, e.g consider the following scenario:
    "PR is issued to include new ui-users that depends on mod-circulation v2 but ui-checkin still depends on v1" in this and similar cases, a manual intervention will be required and a single PR that updates all ui- versions and a version of a shared dependency will be needed

Definition of Done

  • Determine if prerequisites and considerations described above have been initially addressed.

  • Implement the process as described above and produce initial folio-release instances
    for platform-core and platform-complete.

Environment

None

Potential Workaround

None

Checklist

hide

TestRail: Results

Activity

Show:

John Malconian March 25, 2019 at 12:24 PM

John Malconian November 19, 2018 at 7:50 PM

Next build error while attempting to build platform-core/next-release with releases only:

Incompatible version for module mod-users-bl-4.0.2 interface service-points. Need 2.1. Have 3.0

mod-inventory-storage 13.0.1 provides service-points 3.0 but the last release of mod-users-bl, v4.0.2 requires service-points 2.1. A new release of mod-users-bl compatible with service-points 3.0 is required.

Marc Johnson November 15, 2018 at 3:29 PM

Thanks, I'll be curious to see if mod-circulation 12.1.0 is picked up and resolves this, or if there is something else driving this version conflict

John Malconian November 15, 2018 at 2:37 PM

No, I haven't tried a build for a couple of weeks so it's probable the item-storage/mod-circulation issue has been resolved. I'll try another build today.

Marc Johnson November 15, 2018 at 10:33 AM
Edited

Thanks for digging deeper into this.

UI Inventory requiring only inventory 7.0, which is provided only by mod-inventory 10.0.0 which requires only item-storage 6.0 effectively forces the resolution to item-storage 6.0 across the board.

Hmm, as far as I can tell, the latest tagged released version of mod-circulation 12.1.0 supports item-storage 5.3 or item-storage 6.0. This was done last week (https://jenkins-aws.indexdata.com/job/folio-org/job/mod-circulation/view/tags/job/v12.1.0/) and published to the registry (http://folio-registry.aws.indexdata.com/_/proxy/modules/mod-circulation-12.1.0).

I wonder what meant this wasn't picked up?

Maybe there is something else that is forcing resolution to item-storage 5.3 and so we have an unresolvable divergence. What I'm less sure about is why the conflict was discovered at mod-inventory.

Did you try this build yesterday?

Done

Details

Assignee

Reporter

Priority

Development Team

Core: Platform

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs
Created October 12, 2018 at 6:31 PM
Updated April 4, 2019 at 1:48 PM
Resolved April 4, 2019 at 1:48 PM
TestRail: Cases
TestRail: Runs

Flag notifications