Platform, DevOps and Release Management (UXPROD-1814)

[UXPROD-2066] CI architecture and process to allow deployment of feature branches Created: 26/Aug/19  Updated: 10/Aug/20  Resolved: 13/Jan/20

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: Q4 2019
Parent: Platform, DevOps and Release Management

Type: New Feature Priority: P2
Reporter: Jakub Skoczen Assignee: Jakub Skoczen
Resolution: Done Votes: 0
Labels: cap-mvp, platform-backlog, po-mvp
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
blocks UXPROD-2118 Preview capability for PRs (Q4, final... Closed
is blocked by FOLIO-2265 Create preview namespace and Okapi fo... Closed
is blocked by FOLIO-2267 Jenkins pipeline to deploy backend mo... Closed
is blocked by FOLIO-2268 preview tenant/bundle creation for fe... Closed
is blocked by FOLIO-2275 create a docker registry in nexus for... Closed
is blocked by UXPROD-1827 CI-integrated continuous deployment (... Closed
is blocked by FOLIO-2266 SPIKE: determine how to build/deploy ... Closed
Relates
relates to UXPROD-1817 Preview capability for PRs (Q3, initi... Closed
relates to UXPROD-1827 CI-integrated continuous deployment (... Closed
relates to UXPROD-2118 Preview capability for PRs (Q4, final... Closed
relates to FOLIO-2011 initial roll out of PR preview capabi... Closed
relates to FOLIO-2131 SPIKE: how to deploy containers based... Closed
relates to FOLIO-2092 use -alpha.NNN pre-release version sy... Blocked
Epic Link: Platform, DevOps and Release Management
Back End Estimate: XXL < 30 days
Development Team: Core: Platform

 Description   

The architecture for feature branch deployment is being captured in the following document:

https://docs.google.com/document/d/1tM6cLUMO9O_85moCYk2ICjqNEQxsaHm9R0oGXzu6YAg/edit?ts=5d5fe77a

Components of the solution::

  • the K8s cluster will include a new namespace for deployment of containers based on PR/fb artefacts (called preview, running next to the standard main namespace used for deployment of release and snapshot (master) artefacts)
  • the preview namespace will run a dedicated Okapi, that will include ModuleDescriptors for release, snapshot and PR/fb artefacts. (Okapi in the main namespace only includes MDs for release and snapshots). MDs for PR/fb artefact will be cleaned up periodically.
  • actual Docker containers resulting from PR/fb builds will be published to a private Docker registry available to the CI (unlike release and snapshot -based containers that are published to public DockerHub)
  • NPM PR/fb builds (front-end) will be published to a separate namespace in FOLIO Nexus, a virtual namespace will be configured for the CI that includes release/snapshot artefacts and PR/fb ones.

Summary of the development process::

Developer working on a feature branch opens a PR which triggers CI build and deployment process:

  • for backend modules (mod-): Docker container is build and published to the private Docker registry. MD is generated and uploaded to the preview namespace Okapi and the container is deployed in the CI's K8s preview namespace.
  • for front-end modules (ui- or library like stripes): NPM is build and uploaded to the preview namespace in Nexus
  • for "platform-" (platform-core, platform-complete, others) repo: bundle is created and deployed to S3, tenant is created in the K8s preview namespace, UI integration tests are executed and the PR preview environment link is reported by Jenkins in the PR comments section. If the PR is issued against the "master" (which in platform- repos represents the "release" branch) the CI performs an additional "dependency check" when the PR is closed – this check verifies that all PR/fb dependencies present when the PR was open have been removed by the developer. In practice, this means that developers are required to release all dependencies (front-end and backend) where they had outstanding PRs first. [TODO: relax this requirement through snapshot dependencies]

Snapshot dependencies:

As mentioned above, when a platform- PR is closed the CI dependency check ensures that all PR/fb dependencies are resolved to releases. What this means in practice is that developers must not only merge all PR/fb dependencies to master branches in individual repos but also perform formal releases of those dependencies. We would like to relax the formal release requirement by allowing "snapshot" dependencies on the appropriate branch of the platform- repo. This way the PR would be issues not againt the platform- "master" (release) branch but against the "snapshot-". To do thins in a reliable way, however, we need to be able to "order" snapshots by their "version numbers" and developers must know them in advance.

The snapshot artefacts will use a standard preRelease versioning: 1.2.3-alpha.1, 1.2.3-alpha.2, ...


Generated at Fri Feb 09 00:20:50 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.