CI architecture and process to allow deployment of feature branches

Description

The architecture for feature branch deployment is being captured in the following document:

https://docs.google.com/document/d/1tM6cLUMO9O_85moCYk2ICjqNEQxsaHm9R0oGXzu6YAg/edit?ts=5d5fe77a

Components of the solution::

  • the K8s cluster will include a new namespace for deployment of containers based on PR/fb artefacts (called preview, running next to the standard main namespace used for deployment of release and snapshot (master) artefacts)

  • the preview namespace will run a dedicated Okapi, that will include ModuleDescriptors for release, snapshot and PR/fb artefacts. (Okapi in the main namespace only includes MDs for release and snapshots). MDs for PR/fb artefact will be cleaned up periodically.

  • actual Docker containers resulting from PR/fb builds will be published to a private Docker registry available to the CI (unlike release and snapshot -based containers that are published to public DockerHub)

  • NPM PR/fb builds (front-end) will be published to a separate namespace in FOLIO Nexus, a virtual namespace will be configured for the CI that includes release/snapshot artefacts and PR/fb ones.

Summary of the development process::

Developer working on a feature branch opens a PR which triggers CI build and deployment process:

  • for backend modules (mod-): Docker container is build and published to the private Docker registry. MD is generated and uploaded to the preview namespace Okapi and the container is deployed in the CI's K8s preview namespace.

  • for front-end modules (ui- or library like stripes): NPM is build and uploaded to the preview namespace in Nexus

  • for "platform-" (platform-core, platform-complete, others) repo: bundle is created and deployed to S3, tenant is created in the K8s preview namespace, UI integration tests are executed and the PR preview environment link is reported by Jenkins in the PR comments section. If the PR is issued against the "master" (which in platform- repos represents the "release" branch) the CI performs an additional "dependency check" when the PR is closed – this check verifies that all PR/fb dependencies present when the PR was open have been removed by the developer. In practice, this means that developers are required to release all dependencies (front-end and backend) where they had outstanding PRs first. [TODO: relax this requirement through snapshot dependencies]

Snapshot dependencies:

As mentioned above, when a platform- PR is closed the CI dependency check ensures that all PR/fb dependencies are resolved to releases. What this means in practice is that developers must not only merge all PR/fb dependencies to master branches in individual repos but also perform formal releases of those dependencies. We would like to relax the formal release requirement by allowing "snapshot" dependencies on the appropriate branch of the platform- repo. This way the PR would be issues not againt the platform- "master" (release) branch but against the "snapshot-". To do thins in a reliable way, however, we need to be able to "order" snapshots by their "version numbers" and developers must know them in advance.

The snapshot artefacts will use a standard preRelease versioning: 1.2.3-alpha.1, 1.2.3-alpha.2, ...

Priority

Fix versions

Development Team

Core: Platform

Assignee

Solution Architect

Parent Field Value

None

Parent Status

None

Checklist

hide

TestRail: Results

Activity

Show:
Done

Details

Reporter

Back End Estimate

XXL < 30 days

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created August 26, 2019 at 11:54 AM
Updated August 10, 2020 at 1:20 PM
Resolved January 13, 2020 at 12:13 PM
TestRail: Cases
TestRail: Runs