[FOLIO-1027] Integrate RAML linting for all raml-based projects in CI Created: 12/Jan/18 Updated: 12/Nov/18 Resolved: 11/Sep/18 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Task | Priority: | P3 |
| Reporter: | John Malconian | Assignee: | David Crossley |
| Resolution: | Done | Votes: | 0 |
| Labels: | ci, core, raml, sprint32, sprint33, sprint42, sprint43, sprint44, sprint45, sprint46 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | 4 days, 4 hours, 50 minutes | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||||||||||||||||||||||||||||||
| Sprint: | |||||||||||||||||||||||||||||||||||||
| Description |
|
Investigate the possibility of implement RAML linting for all projects that include RAML as part of the CI process. |
| Comments |
| Comment by John Malconian [ 12/Jan/18 ] |
|
David Crossley - It seems like you've done some work in this area utilizing raml-cop. What I'd like to implement is a standard lint configuration that can be executed across the RAML in all projects and which is automatically triggered by Jenkins as part of the build and verification project for each repository. Could something like this be implemented and would it be useful? |
| Comment by Marc Johnson [ 12/Jan/18 ] |
|
John Malconian David Crossley I've just been investigating the security vulnerability in marked 0.3.6 that Peter Murray raised (having received an email from github). This is a nested dependency underneath raml-cop 5.0.0 and raml-1-parser 1.1.39 and there don't appear to be releases of these modules that upgrade the version of marked. I tried use npm shrinkwrap to specify a different version, but that seemed to fail, so instead have migrated the script to use yarn instead (which fits better with the tooling of this project anyway) and used it's resolutions capability, which seemed to work. These changes are available in master, as I don't think can check the dependencies without merging. If this resolves the security issue, I shall roll this out to the other modules (and maybe it could be the basis for this work?) |
| Comment by David Crossley [ 15/Jan/18 ] |
|
Regarding that vulnerability: The "marked" is a direct dependency to "raml-1-parser". When the raml-1-parser updated release happens, then the package.json of current raml-cop will automatically incorporate it. There is a raml-1-parser issue and pull/807 to update the dependency. Note that that also moves it to devDependencies. |
| Comment by David Crossley [ 15/Jan/18 ] |
|
Yes, i believe that raml-cop in both CI, and in local project use, would be useful as part of linting. It validates the RAML and associated JSON schema, and any provided examples. Some notes ... I have found that "warnings" should be heeded – they are often more severe than is implied by that label. The messages from the raml-1-parser and friends can be confusing. We do have experience with their interpretation. I have found that all warnings should be attended to. They can mask other issues. So sometimes there are only a few messages. Attend to those and more may be encountered. Often the issues arise because of faulty examples. So temporarily disable them via the raml file to ensure no other issues. The messages can be voluminous and would be annoying in CI if they are just that examples still need to be updated. The location of RAML files in each project is not standard. Perhaps that would be declared in each project's package.json "scripts". There are no command-line options needed, just 'raml-cop <file>'. It can be run on multiple files ('raml-cop *.raml') but that does make any warnings even more obscure because one does not know from which raml file they arise. So perhaps use find. Some projects currently have warnings happening (raml-module-builder mod-configuration mod-finance mod-orders). The mod-configuration could serve as a test, as its issues are minor and obvious. Perhaps exclude the others for now. |
| Comment by David Crossley [ 20/Jan/18 ] |
|
The messages from 'raml-cop' do normally indicate which raml or json filename is causing the issue. In some cases of failing examples, the messages are more obscure. Running raml-cop with *.raml or using find-and-exec does not make a difference (see its issue 15). |
| Comment by David Crossley [ 20/Jan/18 ] |
|
Been exploring how to enable both CI, and local use prior to commit. Some notes ... Do not want to want to have 56MB node_modules in each raml-using repo local checkout. Especially not in the shared "raml" repo (raml-util). Wondering about a separate repository (e.g. folio-dev-tools) checked out at a parallel level. This would just have a package.json and yarn.lock files. (Initially only have raml-cop but later might also want other shared linting tools e.g. "prettier" for validating and consistently formatting JSON files.) Then have a shell script in each raml-using project repository (e.g. mod-configuration/lint-raml-cop.sh) ../folio-dev-tools/node_modules/.bin/raml-cop ramls/configuration/*.raml Some projects might need to configure their script to handle a more complex setup of ramls, so could use find-and-prune-and-exec. Some projects (e.g. mod-codex-mux) only have the shared "raml-util" which would have already been validated. Perhaps developers experiment with local changes to raml-util before moving them to the shared "raml" repo. The Jenkinsfile would be configured to not run this for CI. |
| Comment by David Crossley [ 07/Feb/18 ] |
|
I have done some experiments locally with the different ways. Method A): Similar to that already in mod-circulation/lint.sh which does the 'yarn install' and runs raml-cop with a declared path to its "*.raml" files (which may be different for each repository). Could use yarn scripts, rather than shell script. So each raml-using repo would have its own yarn install, etc. Not sure how that would work for the shared raml-util, or for separate shared raml (
Method B): Expect developers to have the "folio-tools" (or some such) repository checked out at a parallel level (../folio-tools). That provides the directory "folio-tools/run-raml-cop" which has a shell script "run-raml-cop.sh" which would ensure that its own checkout is up-to-date and does 'yarn install'. The script accepts one argument being the paths to any raml files. Each raml-using repository would have a shell script "lint-raml-cop.sh" which ensures that the "../folio-tools" is present, then calls that script, passing the paths to its raml files. If there is interest in this method then i could publish the branches. Method C): Encourage each developer to have raml-cop installed globally 'npm install -g raml-cop' and keep up-to-date. Each raml-using repository would have a simple "lint-raml-cop.sh" which expects that, and runs it on the declared path to its raml files. For CI, have a Python script similar to the current folio-tools/generate-api-docs/ which retrieves the managed api.yml data file (
|
| Comment by David Crossley [ 20/Feb/18 ] |
|
With Method C, the CI could have raml-cop globally installed itself, and run each repository's "lint-raml-cop.sh". |
| Comment by David Crossley [ 29/Mar/18 ] |
|
Added a new document at dev.folio.org/guides/raml-cop/ to assist with installation, usage, and interpretation of messages. Added a shell script at github.com/folio-org/mod-notes/blob/master/lint-raml-cop.sh Currently only added to mod-notes. The same script would be added to other raml-using projects. For CI, raml-cop would be installed globally. Next step is to investigate how to run this in CI. Would there be an entry in each raml-using project's Jenkinsfile, and corresponding handling in Jenkins? Should each repository have a top-level "scripts" directory to house such, and so not clutter with this and other scripts, and enable Jenkins to reliably locate it? |
| Comment by David Crossley [ 19/Jun/18 ] |
|
See folio-tools/lint-raml/lint_raml_cop.py added at
|
| Comment by David Crossley [ 09/Jul/18 ] |
|
See the related PRs at: This installs and runs that lint_raml_cop.py script. It is configured similarly to the "generate-api-docs" job, and is modelled on the "Lint" stage of front-end repositories and its runLintNPM.groovy script. So if there are issues then it will report them, but not block anything. |
| Comment by David Crossley [ 22/Aug/18 ] |
|
This is now implemented. Currently at the end of the pipeline. See example at mod-configuration/pull/50 Developers can run the same script locally with `folio-tools/lint-raml/lint_raml_cop.py` In CI it will only report its findings, and not stop anything. With a PR, any errors will be listed, otherwise quiet. See [folio-tools/lint-raml|https://github.com/folio-org/folio-tools/tree/master/lint-raml) |
| Comment by David Crossley [ 22/Aug/18 ] |
|
Still need to add to each repository's Jenkinsfile. |
| Comment by Marc Johnson [ 22/Aug/18 ] |
|
David Crossley Does this mean that the scripts directory will be disappearing from all of the backend modules and we will run the lint-raml-cop.sh script from a checkout of folio-tools instead? |
| Comment by David Crossley [ 22/Aug/18 ] |
|
I reckon so. The centralised tools would be easier to maintain (and is already improved). I would use the lint_raml_cop.py in preference – same as used by CI. Also it detects various other issues, especially for pre RMB v20. |
| Comment by Marc Johnson [ 23/Aug/18 ] |
|
I think having consistency with what the CI runs and having a single place to change outweighs needing to have a different repository cloned and the tools globally installed. |
| Comment by Marc Johnson [ 24/Aug/18 ] |
|
David Crossley I've been trying this out and it seems to work smoothly with existing modules And I tried it out with a standalone module and local config and that worked nicely too. |
| Comment by David Crossley [ 25/Aug/18 ] |
|
Thanks for the feedback, Marc. We had a glitch with the raml-1-parser dependency in raml-cop for RAML-1.0 files. See notes at
Now can proceed with setting up CI for the rest. |
| Comment by David Crossley [ 04/Sep/18 ] |
|
This is almost done and is operating successfully. There is a PR to add it to the "raml" shared repo. The other mod-* repos are done, except for mod-orders-storage (which currently has too many issues
There are some recently new mod-* repos. Will get to those soon. |
| Comment by David Crossley [ 11/Sep/18 ] |
|
All relevant repositories (including the "raml" repository) are now configured. The mod-orders-storage is configured but not yet enabled. |