[FOLIO-2575] New CI/CD pipeline for Integration tests Created: 29/Apr/20  Updated: 13/Aug/20  Resolved: 01/Jun/20

Status: Closed
Project: FOLIO
Components: Continuous Integration
Affects versions: None
Fix versions: None

Type: Task Priority: P2
Reporter: Vasily Gancharov Assignee: David Crossley
Resolution: Done Votes: 0
Labels: ci
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
is blocked by MODSOURMAN-318 Remove hardcoded diku tenant in db sc... Closed
is blocked by ERM-903 Migration scripts has hardcoded diku ... Closed
Relates
relates to FOLIO-1380 set up Postman API test Jenkins job Closed
relates to FOLIO-2574 Create folio-integration-tests github... Closed
relates to FOLIO-2616 Jenkins job folio-perf-test fails ins... Closed
Sprint: DevOps: sprint 89
Development Team: FOLIO DevOps

 Description   

Please, create a new CI/CD pipeline for Integration tests. The pipeline should be a copy of CI/CD for Performance tests with the exception that it should trigger Integration tests.
The Integration tests are created by using Karate framework.

Link to the tests repo will be added as a comment to this ticket.

FOLIO-2574 Closed https://github.com/folio-org/folio-integration-tests



 Comments   
Comment by Vasily Gancharov [ 29/Apr/20 ]

A ticket to create folio-integration-tests github repository for Integration tests: https://folio-org.atlassian.net/browse/FOLIO-2574

Comment by Wayne Schneider [ 29/Apr/20 ]

We will address FOLIO-2574 Closed in this sprint, further information is needed for this ticket however. We will try to pull it into the next sprint. John Malconian will follow up.

Comment by Vasily Gancharov [ 30/Apr/20 ]

Wayne Schneider, John Malconian, what additional information is required to fulfill this ticket?

Comment by Anton Emelianov (Inactive) [ 04/May/20 ]

The goal is to create a Jenkins job similar to https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-platform/
1) This job can use the same folio system ( if I remember correctly it is being deployed before test execution)
2) Instead of executing jMeter tests it will execute API integration tests located in the folio-integration-tests repo
3) Any test committed to folio-integration-tests repo will be automatically picked up and executed on by this Jenkins job.
4) Build fails If system unavailable or failed to deploy. On the contrary, one failed test doesn't render build as failed (just like in https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-test/)
5) Also, it would be great to have a carbon copy Jenkins job for https://github.com/folio-org/folio-api-tests
6) All performance and integration test jobs can be executed against the same system and it can provisioned once for all tests. My understanding that we're already doing this for folio-perf-test and folio-perf-platform jobs.

Comment by Vasily Gancharov [ 14/May/20 ]

Wayne Schneider, John Malconian, on the one hand, this is a pretty important CI/CD pipeline and on the other, I believe it should be easy to implement it since it re-uses almost all steps of existing CI/CD (https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-test/).
Please, handle this ticket as high-priority.

When DevOps team will be able to implement the pipeline?

Comment by Jakub Skoczen [ 20/May/20 ]

GUys,

https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-platform/ is dedicated for Platform team's perf test and SHOULD NOT be used for any other type of testing.

https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-test/ COULD be used for running integration tests but looking at the maven build it's been broken for many weeks.

Comment by Vasily Gancharov [ 22/May/20 ]

Jakub Skoczen, is there any ticket to fix folio-perf-test pipeline: https://jenkins-aws.indexdata.com/job/Automation/job/folio-perf-test/ ? Is this issue related to the pipeline itself or the environment it uses?

David Crossley, can you please:

Comment by David Crossley [ 22/May/20 ]

Regarding the folio-perf-test Jenkins job, following the general fix by Hongwei Ji yesterday i expected it to be okay. There now seems to be some additional failure today. Not sure what yet. So yes, someone should raise a new ticket for that.

Regarding the focus of this FOLIO-2575 Closed , yes, i will be doing that on Monday. Thanks for the clarification.

Comment by Vasily Gancharov [ 22/May/20 ]

Thank you David Crossley. Can I ask you to create a ticket for folio-perf-test Jenkins job fix, please, and add the link to it to this thread?
Appreciate your help.

Comment by Hongwei Ji [ 22/May/20 ]

David Crossley, looks like that folio-perf-platform failed to build because it was trying to get okapi version from https://folio-snapshot-core-okapi.aws.indexdata.com/_/version, but unfortunately, that site returned 503 (still does at this moment).

Comment by David Crossley [ 26/May/20 ]

Thanks Hongwei Ji. Yes we subsequently got past that intermittent problem with folio-perf-platform Jenkins job.

The failure with folio-perf-test Jenkins job is something different, and seems to have been happening for many runs. That is now described at FOLIO-2616 Closed .

Comment by David Crossley [ 26/May/20 ]

I have made some progress with this, but have reached some blockages.

There is a Jenkins job "folio-integration" which is essentially a replica of the folio-perf-test Jenkins job. It utilises a new Jenkinsfile.integration in https://github.com/folio-org/folio-perf-test (on a branch at the moment).

That uses the configuration and resources in folio-perf-test repo to set up the FOLIO instance. Until FOLIO-2616 Closed is solved, this setup is using the module install lists from folio-snapshot-core reference environment (it should eventually be using folio-snapshot). Yes, the integration tests will fail against that temporary setup.

The Jenkins job then goes on to checkout the folio-integration-tests repo and run maven there to operate the tests. Using a branch there which accepts the Okapi IP address of the test instance.

Currently failing because we also need a away to provide the admin user credentials, so that the maven tests can login, install tenants, and run the tests.

Comment by Hongwei Ji [ 26/May/20 ]

David Crossley, I added a comment to FOLIO-2616 Closed . Basically it is blocked by ERM-903 Closed and MODSOURMAN-318 Closed

Comment by David Crossley [ 27/May/20 ]

Other issues are now addressed, and the recent run of Jenkins job folio-integration #14 is good.

Except of course that it still does not use folio-snapshot until FOLIO-2616 Closed .

Comment by David Crossley [ 29/May/20 ]

The FOLIO-2616 Closed is fixed, and now we have our first complete run at folio-integration #17

Still need some tweaks to the job, but this is good progress.

Comment by David Crossley [ 01/Jun/20 ]

The Jenkins job is now ready at folio-integration.

It uses the FOLIO system built by the configuration in the repository "folio-perf-test" (see Jenkinsfile.integration there). That uses the set of modules from "folio-snapshot" reference environment daily build. So each time that the job is run, it rebuilds that system.

It then follows on, to checkout master branch of the repository "folio-integration-tests", and runs all of the maven tests that are configured there.

The Jenkins job is configured to automatically run once per day, starting around 05:00 UTC. It could be triggered manually at other times, but be aware that the "folio-snapshot" reference environment must be available.

Comment by Marc Johnson [ 07/Aug/20 ]

What does the populate data stage do in that pipeline?

Comment by Marc Johnson [ 07/Aug/20 ]

David Crossley

It uses the FOLIO system built by the configuration in the repository "folio-perf-test" (see Jenkinsfile.integration there). That uses the set of modules from "folio-snapshot" reference environment daily build. So each time that the job is run, it rebuilds that system.

Does that mean that the folio-integration pipeline relies on an environment built by the folio-perf-test pipeline?

Comment by David Crossley [ 10/Aug/20 ]

No, the environments are separate.

The "folio-integration" Jenkins job uses the Jenkinsfile.integration file which is housed in the repository "folio-perf-test".

That "Jenkinsfile.integration" uses the same "cloudformation configuration" that is provided by that "folio-perf-test" repository. See the "sharedLib" references in the various "Jenkinsfile.*" files there.

So the configuration is very similar, to essentially piggyback on the configuration of the other older jobs. But the built environments are separate.

Comment by David Crossley [ 10/Aug/20 ]

Marc asked:

What does the populate data stage do in that pipeline?

The "Populate data" stage (of each of those pipelines that are configured via that repository) obtains a package from S3 that contains a large amount of library data, and various shell scripts to create users and load data.

Search the Jenkins console output for "folio-public-sample-data" to see it download and unpack the huge archive, then do various postgres ops to add users, then load inventory and circulation test data.

The EBSCO devops people would be able to explain better than me.

Comment by Marc Johnson [ 10/Aug/20 ]

Thanks David Crossley

Generated at Thu Feb 08 23:21:40 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.