[FOLIO-2920] stripes-core and -cli jobs die in CI during "Run SonarQube" Created: 16/Dec/20  Updated: 04/Feb/21  Resolved: 04/Feb/21

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Bug Priority: P2
Reporter: Zak Burke Assignee: John Malconian
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: Text File master-resolved.txt    
Issue links:
Relates
relates to STCLI-168 lib/platform/alias-service.js causes ... Closed
Sprint: DevOps: Sprint 105, DevOps Sprint 107, DevOps: Sprint 104, DevOps Sprint 106
Development Team: FOLIO DevOps

 Description   

UI PRs are dying in Jenkins during the “Run Sonarqube” step with an error message like this:

ERROR: Error during SonarScanner execution
java.lang.IndexOutOfBoundsException: toIndex = 3
	at java.base/java.util.AbstractList.subListRangeCheck(AbstractList.java:507)
	at java.base/java.util.ArrayList.subList(ArrayList.java:1138)
	at com.sonar.security.analysis.C.A.E.A(na:2711)
	[...]


 Comments   
Comment by John Malconian [ 16/Dec/20 ]

Zak Burke Can you point me to a job with this error?

Comment by Zak Burke [ 16/Dec/20 ]

e.g. stripes-core, stripes-cli

Comment by John Malconian [ 16/Dec/20 ]

So far only stripes-core and stripes-cli seem impacted whatever is going on here. stripes-cli does something with stripes-core so it may come down to an issue with stripes-core. I don't see this kind of error for any other repo. It may take a couple of days to fully debug this, so I would advise disabling Sonarqube in the repo's Jenkinsfile if necessary.

Comment by Zak Burke [ 16/Dec/20 ]

Thanks for the update, John Malconian!

Comment by Zak Burke [ 22/Dec/20 ]

Interestingly, I also see this issue in a bug-fix patch to stripes-core v5, the Goldenrod release. This suggests a change in a third-party dependency is the root cause here, as the changes themselves (see STCOR-496 Closed ) were merged to master in August and did not involve any changes to package.json. IOW, the lock file is different between the v5.0.3 build and the STCOR-409 Closed build not because of stripes-core changes but because of third-party lib changes.

Comment by Zak Burke [ 22/Dec/20 ]
$ diff yarn.STCOR-481.lock yarn.808-master.lock | grep resolved > master-resolved.txt

STCOR-481 Closed was the most-recent PR where SonarQube ran successfully, so master-resolved.txt shows the ~60 changes in dependencies.

Comment by John Malconian [ 08/Jan/21 ]
  • Anything in ./node_modules is excluded from analysis so I don't any 3rd party node packages are causing this issue.
  • I tested with multiple versions of the Scanner app and they all fail the same way.
  • I think the scanner may be getting tripped on one particular rule, S2083, which is part of the default Sonar rules profile for JS. If so, it doesn't necessarily mean that the failure is due to a change to stripes-core/cli. It could be related to a Sonar update to the rule.
  • It could also be something else.

I'm testing on the FOLIO-2920-test branch of stripes-core. I've enabled DEBUG level logging of Sonarqube on that branch. This very well could be a bug that should be reported to the Sonarqube folks for further analysis. I will need to capture some additional information, before sending them a report.

Comment by John Malconian [ 08/Jan/21 ]

Error with DEBUG enabled:

19:35:48.263 INFO: Analyzing 741 ucfgs to detect vulnerabilities.
19:35:48.323 DEBUG: loaded 2 sources for rule S2083
19:35:48.359 DEBUG: Resource file jssecurity/sinks/common.json was not read
19:35:48.360 DEBUG: loaded 1 sinks for rule S2083
19:35:48.401 DEBUG: loaded 3 sources for rule S3649
19:35:48.410 DEBUG: Resource file jssecurity/sinks/common.json was not read
19:35:48.411 DEBUG: loaded 2 sinks for rule S3649
19:35:48.436 DEBUG: loaded 2 sources for rule S6096
19:35:48.443 DEBUG: Resource file jssecurity/sinks/common.json was not read
19:35:48.444 DEBUG: loaded 1 sinks for rule S6096
19:35:48.567 INFO: rule: S2083, entrypoints: 189
19:35:48.567 DEBUG: Running rule jssecurity:S2083
19:35:48.567 INFO: Running symbolic analysis
19:35:48.571 DEBUG: loaded 5 sanitizers for rule S2083
19:35:48.571 DEBUG: Resource file jssecurity/passthroughs/common.json was not read
19:35:48.571 DEBUG: Resource file jssecurity/passthroughs/S2083.json was not read
19:35:48.571 DEBUG: loaded 0 passthroughs for rule S2083
19:35:48.572 DEBUG: Resource file jssecurity/collectionHandlers/common.json was not read
19:35:48.572 DEBUG: Resource file jssecurity/collectionHandlers/S2083.json was not read
19:35:48.572 DEBUG: loaded 0 collectionHandlers for rule S2083
19:35:48.827 DEBUG: eslint-bridge server will shutdown
19:35:54.101 DEBUG: stylelint-bridge server will shutdown
19:35:54.103 INFO: ------------------------------------------------------------------------
19:35:54.103 INFO: EXECUTION FAILURE
19:35:54.103 INFO: ------------------------------------------------------------------------
19:35:54.104 INFO: Total time: 1:24.439s
19:35:54.198 INFO: Final Memory: 31M/128M
19:35:54.198 INFO: ------------------------------------------------------------------------
19:35:54.199 ERROR: Error during SonarScanner execution
java.lang.IndexOutOfBoundsException: toIndex = 3
	at java.base/java.util.AbstractList.subListRangeCheck(AbstractList.java:507)
	at java.base/java.util.ArrayList.subList(ArrayList.java:1138)
	at com.sonar.security.analysis.C.A.E.A(na:2711)
	at com.sonar.security.analysis.C.A.E.A(na:2399)
	at com.sonar.security.analysis.C.A.A.D.B(na:3360)
	at com.sonar.security.analysis.C.C.A(na:2841)
	at com.sonar.security.analysis.D.C.A(na:673)
	at com.sonar.security.analysis.D.C.A(na:101)
	at com.sonar.security.analysis.D.C.A(na:883)
	at com.sonar.security.analysis.D.I$_B.A(na:2454)
	at com.sonar.security.analysis.D.I$_B.C(na:1304)
	at com.sonar.security.analysis.D.I$_B.A(na:1238)
	at com.sonar.security.analysis.D.E.A(na:1306)
	at com.sonar.security.analysis.D.I.A(na:1225)
	at com.sonar.security.analysis.D.J.C(na:2334)
	at com.sonar.security.analysis.D.J.A(na:1511)
	at com.sonar.security.analysis.D.A.B(na:877)
	at com.sonar.security.analysis.D.A.A(na:1059)
	at com.sonar.security.analysis.B.A(na:2276)
	at com.sonar.security.analysis.B.A(na:1069)
	at com.sonar.security.rules.E.A(na:2391)
	at com.sonar.security.rules.E.A(na:515)
	at com.sonar.security.D.A(na:525)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at com.sonar.security.D.executeChecks(na:2754)
	at com.sonar.security.D.execute(na:2914)
	at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:45)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:75)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:51)
	at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:68)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
	at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:438)
	at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:434)
	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:392)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
	at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:126)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:123)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:109)
	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:58)
	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:52)
	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
	at com.sun.proxy.$Proxy0.execute(Unknown Source)
	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
	at org.sonarsource.scanner.cli.Main.main(Main.java:61)
Comment by John Malconian [ 04/Feb/21 ]

I'm not convinced that the cause of the sonarqube crash is the same on both projects, but I have taken the following actions.

  • Added a per project configurable CI option for Node-based builds to configure specific directories Sonarqube should scan (recursively) This option is configurable int the project's Jenkinsfile by specifying the parameter 'sonarScanDirs' and giving it a value of the directory or directories Sonarqube should scan. Multiple directories are delimited with a ','. Previously, the default SQ scan directory was the top-level directory of the project. The default is now './src' which is fine for most projects. Several Stripes and ui-plugin directories don't have a top-level 'src' directory so I've configured the relevant 'sonarScanDirs' for those projects appropriately.
  • By changing the default SQ scan directory to 'src', SQ is able to successfully analyze the stripes-core project. I've re-enabled SQ scans for that project.
  • I configured the SQ scan directory for stripes-cli to be './lib'. However, SQ scans still fails. Through a trial and error process I was able to determine that the failure is due to some issue with the file ./lib/platform/alias-service.js. I've added that file to the SQ exclusion list and re-enabled SQ analysis. I've also opened a separate issue, STCLI-168 Closed , for further analysis by the Stripes team.
Generated at Thu Feb 08 23:24:14 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.