Thunderjet - Onboarding plan
(this is a draft)
This documentation provides some help for backend developers to get started. It is most likely incomplete, so please don't hesitate to ask and add information here as needed.
See also: Overview for a new developer and Newcomer's first steps
FOLIO Design & Architecture
- Okapi : Gateway/Discovery service and Security model - https://folio-org.atlassian.net/wiki/display/~vijay/FOLIO+-+Developer+Onboarding+Guide#FOLIODeveloperOnboardingGuide-Design&Architecture
- Eureka (replacing Okapi)
- RMB framework : Core module for all Vertx based modules - https://github.com/folio-org/raml-module-builder#overview
Introduction to FOLIO Acquisitions
Some demo videos are useful to understand FOLIO acquisition apps (these are using old versions):
See also: Thunderjet - Guidelines for backend code organization, unit tests and Karate tests
Running in IDEA
Installation
- Install Java JDK 21
Install IDEA as IDE : https://www.jetbrains.com/idea/
- Clone from git modules and add all of them in one IDEA project (import as Maven)
- ThunderJet back-end modules : mod-orders, mod-orders-storage, mod-invoice, mod-invoice-storage, mod-finance, mod-finance-storage, mod-organizations, mod-organizations-storage, mod-gobi, edge-orders, edge-common, mod-ebsconet, acq-models
- Modules with integration tests: folio-integration-tests
- Use eureka-cli to run a local environment, see requirements there; use the combined config
IDEA project / modules organization and plugins
In IDEA, a single project can use the acquisition git repositories as modules. It can use a project directory containing all the git repositories.
A module is added with the Maven configuration. After compile, the directory target/generated-sources should be set as a generated source folder for the module.
The following plugins can be useful: uppercut, YAML.
Debugging
A "Remote JVM Debug" configuration can be used to debug a module in IDEA, without having to redeploy it (check the port with docker).
Modules can be executed in debug mode in IDEA, they just need to be intercepted (see below). Unit tests can also be debugged in the same way.
change log level for mod-invoice-storage (currently not working):
http://localhost:8097/admin/loglevel?level=INFO&java_package=org.folio.rest.persist.PostgresClient
Creating and running unit tests
Unit tests are executed with Maven. The files are in src/test.
Tests are using Mockito, RestAssured, JUnit and Hamcrest.
Remove src/main/resources/postgres-conf.json + mvn clean compile before running unit tests (if the file is present); put it back afterwards and recompile (it's needed to run storage mods)
Creating and running integration tests
Most of the time, a local environment with eureka-cli (karate.env: dev) should be used for running tests.
Integration tests are using Karate.
Create karate tests in folio-integration-tests (currently in the master-eureka branch) (this should be started before submitting a PR, to check results).
Use @parallel=false at the beginning if the scenarios should be executed one after the other as opposed to in parallel.
Look at features without @parallel=false for good examples.
Tests can be executed either with an ApiTest class in acquisitions/src/test/java/org/folio, or by running the karate test (such as orders.feature) directly with an IDEA plugin such as the karate plugin (not free) or uppercut (free). Running the feature directly makes it possible to run tests in parallel, with a given number of threads.
To run a junit class, use the following environment variable: karate.env=dev
When using uppercut to run a feature file or scenario:
- In the IDEA settings, Tools - Karate: use Default environment:
devand Default parallelism:1. - To run them the first time: right-click on the file, More Run/Debug, Modify Run Configuration, and in the dialog, set: Parallelism:
4, Environment:dev(to run locally with eureka-cli) and change "Working Directory" to the acquisitions directory so it saves reports in the target directory there (as with junit). Then just "Run". The configuration should be saved so the test can just be started with "Run" the next time.
See Karate Coding Standards & Best Practices.
See how to do a module interception with eureka-cli, to debug a new version of a module.
Troubleshooting
Error when running tests:setup-users.feature:105 - status code was: 422, expected: 201, response time: 18, url: http://localhost:9130/perms/users, response: {"errors":[{"code":"-1","message":"Unable to update derived fields: Attempting to add non-existent permissions invoice.all to permission user with id 4334831c-02d7-4936-849c-2f44d5491300","type":"1",
-> just deploy mod-invoice or whatever permissions are required for
Permission problems, for instance2021-05-19T14:00:10,684 ERROR [vert.x-eventloop-thread-1] RestClient Exception calling GET /organizations-storage/organizations/c6dace5d-4574-411e-8ba1-036102fcdc9b - org.folio.rest.core.models.RequestContext@4286b5d9java.util.concurrent.CompletionException: org.folio.invoices.rest.exceptions.HttpException: Access requires permission: organizations-storage.organizations.item.get
-> Check descriptor (clean+compile), redeploy
maven-resources-test:mod-finance-storage: java.lang.OutOfMemoryError: Java heap space
-> Increase Settings - Build, Execution etc - Compiler - Shared build process heap size and reload all projects
PR workflow
API and module versions
When modifying the API in ModuleDescriptor-template, update the API version number in the related path in it, for instance:
"id": "finance.transactions",
"version": "4.2",
Make a field required -> major API version (for storage + business)
Change a pattern (validation) -> no API version change.
Migration → no module version change
Major API version change -> major module version change.
Never modify pom.xml or NEWS.md unless this is a release PR.
Basic git operations
(assuming the JIRA ticket worked on is "MODORDSTOR-212")
Create the new branch locally:git checkout -b MODORDSTOR-212
(even if there are changes: they will be moved to the new branch)
(if that doesn't work, try "git switch -c <new-branch>")
Commit changes:git add . (to undo, use git reset)git commit -m "[MODORDSTOR-212] ..." (to undo, git reset HEAD~ )
Push to the repo the first time (this is reminded when using simply "git push")git push --set-upstream origin MODORDSTOR-212
after:git push
To go back to the master branch:git checkout master
Finish the PR creation on github
Go to the repo on github, use the new Create pull request button.
See sample at https://github.com/folio-org/mod-orders-storage/pull/230
Assign PR to myself.
After PR is created
Fix code smells etc.
If the build fails with continuous-integration/jenkins/pr-merge, try to click login and rerun buttons on top in jenkins.
Attach a screenshot with Karate tests to show it does not introduce a regression (no new test needed at this point).
Add folio-org/acquisitions team to reviewers on github.
Change JIRA ticket to "Code review".
After the PR is approved by 2 people, the initial poster has to merge it.
Merge with master before if needed.
Check boxes in the PR.
Squash&Merge PR.
After the pull request is merged in github:
Change the "Fix Version" (version can be found in pom.xml, just remove "-SNAPSHOT") and change JIRA ticket to "Review".
Ticket is still assigned to me.
Tests
Create a new PR for the karate tests in folio-integration-tests, with a screenshot of the test results, using the same branch name based on the JIRA ticket.
Possibly test user interface and add a comment to the JIRA ticket about how to test.
Review
Set the JIRA ticket "Assignee" to "Unassigned".
Based on your steps other guy from team will recheck it again and close if everything fine.