PoC for hosted development environment for FOLIO teams (FOLIO-2601)

[FOLIO-2624] SPIKE: Build and deploy Stripes bundle from Rancher pipeline Created: 01/Jun/20  Updated: 06/Jul/20  Resolved: 06/Jul/20

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None
Parent: PoC for hosted development environment for FOLIO teams

Type: Task Priority: P2
Reporter: John Malconian Assignee: jroot
Resolution: Done Votes: 0
Labels: dev-environment
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: Zip Archive okapi-pipeline-master.zip     Zip Archive stripes-pipeline-master.zip    
Issue links:
Relates
relates to FOLIO-2623 SPIKE: Deploy platform-core backend ... Closed
relates to FOLIO-2675 Make the Okapi URL and Tenant ID be e... Open
relates to FOLIO-2669 SPIKE: Determine method of deploying ... Closed
relates to FOLIO-2672 SPIKE: figure out how to workaround r... Closed
Sprint: DevOps: sprint 92, DevOps: sprint 90
Development Team: FOLIO DevOps
Epic Link: PoC for hosted development environment for FOLIO teams

 Description   

It should be possible to build and deploy a Stripes bundle from the snapshot branch of platform-core in a dev environment namespace by utilizing a Rancher pipeline. The Rancher pipeline code can live on the snapshot branch of platform-core. This will give the developers to rebuild a Stripes bundle in their K8s namespace at anytime. The pipeline will require that distinct environment variables be set for each dev namespace that define tenant and okapi url.



 Comments   
Comment by Jakub Skoczen [ 03/Jun/20 ]

jroot I've assigned this ticket to you during our standup – please let me know if you are not working on this.

Comment by jroot [ 09/Jun/20 ]

Right now, I am running into several issues with Pipelines on the newer versions of Rancher in both AWS and my Tamu infrastructure.

There are issues in Rancher Github for the problems I am having:
https://github.com/rancher/rancher/issues/26816

In AWS Rancher, this is the error I am getting:

can't change directory to '/lib/modules': No such file or directory`, error: exit status 1"
time="2020-06-05T17:54:46.132394628Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
Cannot contact buildpod.pipeline-p-vlqtw-2.1-p7sz5-r7ttt: java.lang.InterruptedException

On my v2.4.4 Rancher version, on Tamu infrastructure it is hanging when pulling packages:

Step 2/46 : RUN apk add --no-cache curl git
 ---> Running in e9ff54eabc18
time="2020-06-05T17:47:41Z" level=info msg="shim containerd-shim started" address="/containerd-shim/moby/e9ff54eabc185338fbd2549cebd0f3b7f511590eab8bd091f0b19c1844cd940f/shim.sock" debug=false pid=351 
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
Comment by jroot [ 09/Jun/20 ]

The code I have crafted for an initial PoC for Stripes and Okapi pipelines is attached. It uses my Dockerfile builds that I have had at Tamu for some years now.

I am not as familiar with the Folioorg Okapi Dockerfile. It appears to be templated, so impossible to read how it is crafted - and the options it provides seem limited to me. For example: With the Tamu Dockerfile build, I can pass a switch to Okapi in order to init the database or not.

okapi-pipeline-master.zip stripes-pipeline-master.zip

Comment by jroot [ 29/Jun/20 ]

Folio Rancher cluster needed some re-configuring for the pipelines to execute properly. John Malconian performed the cluster re-create and re-import to Rancher, Projects were re-imported and now pipelines execute properly.

Created a branch called stripes-pipeline in platform-complete: https://github.com/folio-org/platform-complete/tree/stripes-pipeline Successfully built a Stripes bundle and deployed in the Stripes Project.

Researched and discovered some undocumented features for Rancher pipelines to use secrets to set Docker build arguments. Now we can create a secret for each Project, and set the .rancher-pipeline.yml to use them:

envFrom:
      - sourceName: tenant-config
        sourceKey: TENANT_ID
      - sourceName: tenant-config
        sourceKey: OKAPI_URL
    env:
      PLUGIN_DEBUG: "true"
      PLUGIN_INSECURE: "true"
      PLUGIN_BUILD_ARGS_FROM_ENV: OKAPI_URL, TENANT_ID

tenant-config Secret:

OKAPI_URL = https://stripes.ci.folio.org
TENANT_ID = diku

Running with AWS/Rancher Ingress here: https://stripes-test.ci.folio.org/

Comment by Jakub Skoczen [ 06/Jul/20 ]

jroot can this ticket be closed or does it wait for code review on some branch? Am I right assuming that this work is not in scope of the "team envs" now that we are facing resource limitations in the Rancher pipelines?

Comment by jroot [ 06/Jul/20 ]

For now, I would agree with that assessment. Some work needs to go into determining the best course of action considering the resource limitations.

Stanislav Miroshnichenko I believe tested the deploy code - in Rancher with success.

Comment by jroot [ 06/Jul/20 ]

There are some resource limitations and restrictions to take into account with the in-Rancher Jenkins pipelines. I believe we may approach this from a different perspective.

Generated at Thu Feb 08 23:22:02 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.