[FOLIO-1407] Publish karma/nightmare coverage reports in Jenkins/integration with Sonarqube Created: 07/Aug/18  Updated: 12/Nov/18  Resolved: 29/Sep/18

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P3
Reporter: John Malconian Assignee: John Malconian
Resolution: Done Votes: 0
Labels: ci, sprint44, sprint45, sprint47, uitestingteam
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
blocks UXPROD-1052 UI Test coverage reporting Closed
is blocked by UITEST-39 Add coverage functionality to stripes... Closed
Relates
relates to STCLI-71 Enable --coverage for Nightmare tests Closed
relates to FOLIO-1421 Enable SonarCloud static code analysi... Closed
Sprint:

 Description   

Karma coverage reports are generated during Jenkins builds as HTML and lcov.info , but are not published by Jenkins. Publish the HTML report and possibly use lcov.info for integration with Sonarqube.



 Comments   
Comment by John Malconian [ 07/Aug/18 ]

I've implemented the publishing of test coverage reports in Jenkins on a branch. You can see an example in branch 'folio-1407-test' of stripes-components here:

https://jenkins-aws.indexdata.com/job/folio-org/job/stripes-components/job/folio-1407-test/16/Karma_20Coverage_20Report/

I attempted Sonarqube integration utilizing the output file, lcov.info, here:

https://sonarcloud.io/component_measures?id=folio-org%3Astripes-components&metric=coverage

It's difficult for me to discern at this point if the integration is doing what we want it to. Please take a look, Jeffrey Cherewaty, and let me know what you think.

Comment by Jeffrey Cherewaty [ 07/Aug/18 ]

The HTML report artifact on Jenkins looks good

Looks like SonarCloud is including every file in its coverage metrics, including the tests themselves. Example: lib/Accordion/tests/Accordion-test.js. Istanbul doesn't measure test files by default, and I agree with that strategy (they'd all be 100% all the time, not really useful).

Is there a setting for SonarCloud to either...

  • only report on files that appear in lcov.info?
  • or exclude files that end in -test.js?
Comment by John Malconian [ 07/Aug/18 ]

Right. The easiest method would be to exclude -test.js. I will try that.

Comment by John Malconian [ 08/Aug/18 ]

Tests should be excluded now. https://sonarcloud.io/component_measures?id=folio-org%3Astripes-components&metric=Coverage

Comment by John Malconian [ 08/Aug/18 ]

Couple of TODOs to myself so I don't forget -

  • The Sonarqube scan should only be triggered on PR and master branch builds.
  • Set up a separate Github account for Sonarqube. Sonarqube has a nasty habit of removing all previous PR comments when it adds a comment. This may mitigate that.
Comment by Jeffrey Cherewaty [ 08/Aug/18 ]

Looks like we can also exclude any .stories.js or /stories/*.js files.

Comment by John Malconian [ 29/Sep/18 ]

This is enabled for all frontend components by default that generate lcov.info reports.

Generated at Thu Feb 08 23:13:06 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.