Manage Performance Testing environments

Description :

This pipeline was designed to build a performance Bugfest-like environment.

  1. HOW TO CREATE PERF ENV
  2. HOW TO BUILD UI
  3. HOW TO UPDATE PERF ENV
  4. POSSIBILITY ISSUES AFTER UPDATE
  5. HOW TO CHECK LOGS OF MODULES TO RANCHER


  1. HOW TO CREATE PERF ENV

Link: https://jenkins-aws.indexdata.com/job/performance_framework/job/spin_up_perf_create/

Build with parameter

Available parameters:

release_version, teamName, UI_platform_complete, autoApprove, buildUI, pauseTime



Parameter Release_version

kiwi-bugfest-10-28-2021
juniper-bugfest-01-10-2021
ptf-iris-2-2021-07-08
bis1-v2-community-2021-04-01
ptf-kcp1-kiwi-01-18-2022


kcon-kiwi-sanitized-2022-02-14Module list of the latest release (KIWI):

okapi-4.9.0
mod-finance-storage-8.0.0

mod-orders-storage-13.1.0
mod-configuration-5.7.1
mod-inventory-storage-22.0.0
mod-users-18.1.1
mod-login-7.5.0
mod-permissions-5.14.2
mod-pubsub-2.4.0
mod-circulation-storage-13.1.0
mod-source-record-storage-5.2.0
mod-inventory-18.0.0
mod-organizations-storage-4.1.0
mod-finance-4.3.0
mod-calendar-1.13.0
mod-event-config-2.1.0
mod-template-engine-1.15.0
mod-email-1.12.0
mod-sender-1.6.0
mod-notify-2.9.0
mod-feesfines-17.0.0
mod-patron-blocks-1.4.0
mod-notes-2.13.0
mod-circulation-22.1.0
mod-tags-1.0.0
mod-orders-12.2.0
mod-agreements-5.0.0
mod-licenses-4.0.0
mod-audit-2.2.0
mod-courses-1.4.1
mod-service-interaction-1.0.0
mod-data-export-4.2.1
mod-data-import-converter-storage-1.12.0

mod-source-record-manager-3.2.2
mod-data-import-2.2.0
mod-kb-ebsco-java-3.9.0
mod-erm-usage-4.2.0
mod-data-export-spring-1.1.3
mod-invoice-storage-5.2.0
mod-invoice-5.2.0
mod-inn-reach-1.0.1
mod-search-1.5.0
mod-organizations-1.3.0
mod-ldp-1.0.0
mod-oai-pmh-3.6.0
mod-quick-marc-2.2.1
mod-authtoken-2.9.0
mod-remote-storage-1.4.0
mod-codex-mux-2.11.0
mod-password-validator-2.2.0
mod-users-bl-7.1.0
mod-login-saml-2.3.0
mod-copycat-1.1.1
mod-ncip-1.9.0
mod-gobi-2.2.0
mod-patron-5.0.0
mod-rtac-3.1.0
mod-ebsconet-1.1.0
mod-codex-inventory-2.1.0
mod-codex-ekb-1.9.0
mod-eusage-reports-1.0.1
mod-erm-usage-harvester-3.1.2
mod-user-import-3.6.2
mod-graphql-1.9.0
mod-z3950-2.4.0
mod-data-export-worker-1.1.6
edge-caiasoft-1.1.0
edge-connexion-1.0.2
edge-dematic-1.3.0
edge-inn-reach-1.0.1
edge-ncip-1.6.0
edge-oai-pmh-2.4.0
edge-orders-2.4.0
edge-patron-4.6.0
edge-rtac-2.3.0
edge-sip2-2.1.0

Parameter teamName:

This parameter specifies the name of environment and access parameters in Rancher. Admin and members of github teamName team have access to Rancher to configure module deployment.

Values:

metadata, ptf, concorde, core-functional, core-platform, falcon, volaris, folijet, firebird, thunderjet, spitfire, vega

Also teamName specifies endpoint for: FOLIO, OKAPI, pgAdmin, edge modules

EndpointURL template
FOLIOhttps://<teamName>-perf.ci.folio.org
OKAPIhttps://<teamName>-perf-okapi.ci.folio.org
pgAdminhttps://<teamName>-perf-pgadmin.ci.folio.org
edge module namehttps://<teamName>-perf-<edge_module_name>.ci.folio.org/

Parameters UI_platform_complete & buildUI:

By default, docker image docker.dev.folio.org/platform-complete:<teamName>-perf-latest is used. If buildUI = true then repo platform-complete/<UI_platform_complete> is used to build that docker image.

UI_paltform_complete values: master, R2-2021, R1-2021.
Note!!! Pipeline BUILD-UI only applicable for simple Rancher env

Parameter  pauseTime:

Specifies duration of work of environment.
Values: 1h, 2h, ..., 8h, week, infinity(not tested)
Env can be stoped in any time if there is no more activities with it.

Updates:
Date Changes:
28.10.2021
Rancher-46:

New option kiwi-bugfest-10-28-2021 is added.

New parameter added UI_platform_complete is added.

With Kiwi release new version of postgre is used. (10.4 => 12.7)

27.09.2021 Rancher-29:
New option juniper-bugfest-01-10-2021 is added.


2. HOW TO BUILD UI  ( if you need only ui-image)

link: https://jenkins-aws.indexdata.com/job/performance_framework/job/perf-env.ui/

Parameter teamName:

This parameter specifies the name of environment and access parameters in Rancher. Admin and members of github teamName team have access to Rancher to configure module deployment.

Values:

metadata, ptf, concorde, core-functional, core-platform, falcon, volaris, folijet, firebird, thunderjet, spitfire, vega

Parameter TenantName

add your tenant

Parameters UI_platform_complete & buildUI:

By default, docker image docker.dev.folio.org/platform-complete:<teamName>-perf-latest is used. If buildUI = true then repo platform-complete/<UI_platform_complete> is used to build that docker image.

for example :

docker.dev.folio.org/platform-complete docker.dev.folio.org/platform-complete:bulk-edit-34

UI_paltform_complete values: snapshot, master


How to use UI

  1. go to rancher →choose your environment → apps → platform-complete





2. upgrate  module need to change image 



3. HOW TO UPDATE PERF ENV ( backend modules , ui moldules)

link: https://jenkins-aws.indexdata.com/job/performance_framework/job/spin_up_perf_migrate/


for example using perf-bulk-edit to update env 


1. choose release_version parameter 


        kcon-kiwi-sanitized-2022-02-14
        kiwi-bugfest-10-28-2021
        juniper-bugfest-01-10-2021
       ptf-iris-2-2021-07-08
       bis1-v2-community-2021-04-01
       ptf-kcp1-kiwi-01-18-2022


2.  choose branch ( build modules to https://github.com/folio-org/platform-complete)

snapshot - includes all  actually version testing modules

master - includes reliase versions to modules


3. TENANT_ID  - add your tenant for use


4.  autoApprove - choose if you want automatically run pipeline after plan


5. buildUI - it possible build UI into pipeline 


4. POSSIBILITY ISSUES AFTER UPDATE  ( for example using perf-builk-edit)


  1. failed pipeline to start 


was found off problems with  subversions to postgres  :

  • click on failed build
  • replay 
  • main script 
  • if needs using  kiwi-bugfest-10-28-2021  should be change  7 --> 8 subversion
  • run



was found off problems with size to versions of modules  ( local problems ) 

  • click on failed build 
  • replay
  • script1
  • find string 41-42 , need to change:
  •         long numA = verA[i].toLong()
            long numB = verB[i].toLong()
  • run






2.   issues connected with rancher 

  • if not possible open UI for tenant :  need to check mod-permissions add more limits  to module (pods)    
  • go to rancher → add your env
  • apps → mod-permissions
  • upgrate → add answer  "resources.limits.memory 2048Mi"
  •  


if need to rebuild manually pod (module)

  • apps → mod-permissions 
  • click on module
  • find option "config scale/ready scale"
  • click on "-" one time / waiting when status "running" will change to "remove"
  • click on "+" / wainting when module "running"




5. HOW TO CHECK LOGS OF MODULES TO RANCHER