[FOLIO-1490] Enable various tools for validation and editing of RAML files and examples Created: 12/Dec/16 Updated: 12/Nov/18 Resolved: 12/Sep/18 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | New Feature | Priority: | P3 |
| Reporter: | David Crossley | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | lint, raml | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | 3 weeks, 2 days, 2 hours, 45 minutes | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||||||||||
| Sprint: | |||||||||||||||||
| Description |
|
While doing DMOD-88, some issues arose with the RAML and associated JSON files. Also a pre-processing workaround is currently required, to replace the $ref which links parent and child JSON schema files. |
| Comments |
| Comment by David Crossley [ 12/Dec/16 ] |
|
Add .eslintrc to relevant modules to declare the plugins eslint-plugin-raml and eslint-plugin-json. (Not automating via build for the first phase.) |
| Comment by David Crossley [ 12/Dec/16 ] |
|
Some of the error and warning messages are obscure. Will list some here with their fix. |
| Comment by David Crossley [ 12/Dec/16 ] |
|
eslint mod-metadata/doc/api/catalogue.raml Interpretation: Fixed in 6a685b8 |
| Comment by Marc Johnson [ 12/Dec/16 ] |
|
Well that is embarrassing, I don't know how those got switched around The knowledge base core and catalogue core modules will be deprecated pretty soon. How does es-lint determine which RAML / JSON.Schema files are checked? Is it run on every build? |
| Comment by David Crossley [ 13/Dec/16 ] |
|
It is not automated to run for every build (as noted above). I should have explained better how to use it manually. Install the plugins (eslint-plugin-raml and eslint-plugin-json). The schema and examples and other RAML files are linked from each top-most raml file. So for example do 'eslint mod-metadata/doc/api/catalogue.raml' – the eslint-plugin-raml does find and process all of its pieces. The eslint-plugin-raml will also notify when a linked JSON file has issues. Then use the eslint-plugin-json to report the detail, e.g. 'eslint schema/instance.json' (a real example is upcoming). |
| Comment by David Crossley [ 13/Dec/16 ] |
|
Now some examples from mod-users. These occur in other modules too, but mod-users is the smallest so easier to analyse. |
| Comment by David Crossley [ 13/Dec/16 ] |
|
eslint mod-users/ramls/users.raml Interpretation: The same warning is reported four times. The first two: "35:13 and 44:17" are where "exampleItem" is referenced in raml-util/rtypes/collection.raml file. The second two: "13:15 and 56:13" are where "exampleItem" is referenced in raml-util/rtypes/item-collection.raml file. Fixed in 8d315b2 To see the detail of the JSON issue: The eslint-plugin-json wants files with *.json extension. |
| Comment by David Crossley [ 13/Dec/16 ] |
|
eslint mod-users/ramls/users.raml Interpretation: Fix: Not yet applied. 5,6c5,6 < example: | < <<example>> --- > example: <<example>> > type: "object" |
| Comment by David Crossley [ 13/Dec/16 ] |
|
eslint mod-users/ramls/users.raml Interpretation: Fix: Not yet applied. 10c10 < "$ref" : "userdata" --- > "$ref" : "userdata.json" |
| Comment by Marc Johnson [ 02/Jan/17 ] |
|
Hi David, How is eslint run for the modules? Is it intended to be part of the build process during the work on this issue? |
| Comment by David Crossley [ 03/Jan/17 ] |
|
See above for temporary instructions for how to run eslint manually. And note the need to apply a temporary workaround for "Reference could not be resolved". No, not yet intending to add this eslint to the build process. At the moment just using it manually to tidy up all our RAML files. I have also been exploring using "raml-tester", which is used in the "okapi/okapi-core" build process. |
| Comment by Marc Johnson [ 03/Jan/17 ] |
|
Thanks David, I thought I'd been through the comments, sorry. I thought I'd have a quick go at scripting this, mod-metadata now has a lint.sh script to run eslint (and it replaces the references so the checks pass currently). |
| Comment by Adam Dickmeiss [ 03/Jan/17 ] |
|
The RAML parser of RMB does not like the latest commit (
java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:294) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalArgumentException: Invalid RAML definition: ERROR mapping values are not allowed here (line 5, col 16) at org.raml.jaxrs.codegen.core.AbstractGenerator.run(AbstractGenerator.java:792) at org.raml.jaxrs.codegen.core.GeneratorProxy.run(GeneratorProxy.java:40) at org.folio.rest.tools.GenerateRunner.main(GenerateRunner.java) ... 6 more I suggest we revert it. Besides.. The query is supposedly going to be string anyway - because it's eventually CQL. |
| Comment by David Crossley [ 04/Jan/17 ] |
|
Reverted in commit 4731f00. (Oops – a lesson in testing.) So users of eslint-raml, ignore the warning described at comment-12923 above. |
| Comment by David Crossley [ 11/Jan/17 ] |
|
Added a Python script to folio-infrastructure/run-raml-tools. Different tools are used to process a set of RAML files for each module repository. Each tool has certain capabilities. None seem to report all issues. So it is useful to run a number of them, and review their output. As explained above in this
Currently applying "eslint-raml", "ramlev", "ramlfications". The outputs are proving useful for tidying our RAML files, and enhancing the api docs (DMOD-88). |
| Comment by Marc Johnson [ 12/Jan/17 ] |
|
David David Crossley, would it be useful to use that Python script (using a submodule) in mod-metadata? |
| Comment by David Crossley [ 16/Jan/17 ] |
|
I had only intended it to be used manually, external to any particular repository, and then review the output. It does install heaps of stuff in NPM local "./node_modules". Also currently it is in a private repo, as i do not yet know where to put such. |
| Comment by David Crossley [ 08/Feb/17 ] |
|
Improved the run-raml-tools script: Now also apply "raml-tester" via its local proxy. |
| Comment by David Crossley [ 09/Mar/17 ] |
|
As explained above, some processing tools handle schema references differently. |
| Comment by David Crossley [ 16/Mar/17 ] |
|
Improved the run-raml-tools script: Added 'raml-cop' which keeps well up-to-date. It is now my favourite tool. |
| Comment by David Crossley [ 12/Sep/18 ] |
|
Now we have consistent $ref, and the dereferencer in RMB. |