Thunderjet - Onboarding plan

Thunderjet - Onboarding plan

(this is a draft)


This documentation provides some help for backend developers to get started. It is most likely incomplete, so please don't hesitate to ask and add information here as needed.

See also: Overview for a new developer and Newcomer's first steps

FOLIO Design & Architecture

Introduction to FOLIO Acquisitions

Some demo videos are useful to understand FOLIO acquisition apps (these are using old versions):

See also: Thunderjet - Guidelines for backend code organization, unit tests and Karate tests

Running in IDEA

Installation

  1. Install Java JDK 21
  2. Install IDEA as IDE : https://www.jetbrains.com/idea/

  3. Clone from git modules and add all of them in one IDEA project (import as Maven)
    • ThunderJet back-end modules : mod-orders, mod-orders-storage, mod-invoice, mod-invoice-storage, mod-finance, mod-finance-storage, mod-organizations, mod-organizations-storage, mod-gobi, edge-orders, edge-common, mod-ebsconet, acq-models
    • Modules with integration tests: folio-integration-tests
  4. Use eureka-cli to run a local environment, see requirements there; use the combined config

IDEA project / modules organization and plugins

In IDEA, a single project can use the acquisition git repositories as modules. It can use a project directory containing all the git repositories.

A module is added with the Maven configuration. After compile, the directory target/generated-sources should be set as a generated source folder for the module.

The following plugins can be useful: uppercut, YAML.

Debugging

A "Remote JVM Debug" configuration can be used to debug a module in IDEA, without having to redeploy it (check the port with docker).

Modules can be executed in debug mode in IDEA, they just need to be intercepted (see below). Unit tests can also be debugged in the same way.

change log level for mod-invoice-storage (currently not working):
http://localhost:8097/admin/loglevel?level=INFO&java_package=org.folio.rest.persist.PostgresClient

Creating and running unit tests

Unit tests are executed with Maven. The files are in src/test.

Tests are using Mockito, RestAssured, JUnit and Hamcrest.

Remove src/main/resources/postgres-conf.json + mvn clean compile  before running unit tests (if the file is present); put it back afterwards and recompile (it's needed to run storage mods)

Creating and running integration tests

Most of the time, a local environment with eureka-cli (karate.env: dev) should be used for running tests.

Integration tests are using Karate.

Create karate tests in folio-integration-tests (currently in the master-eureka branch) (this should be started before submitting a PR, to check results).

Use @parallel=false at the beginning if the scenarios should be executed one after the other as opposed to in parallel.

Look at features without @parallel=false for good examples.

Tests can be executed either with an ApiTest class in acquisitions/src/test/java/org/folio, or by running the karate test (such as orders.feature) directly with an IDEA plugin such as the karate plugin (not free) or uppercut (free). Running the feature directly makes it possible to run tests in parallel, with a given number of threads.

To run a junit class, use the following environment variable: karate.env=dev

When using uppercut to run a feature file or scenario:

  • In the IDEA settings, Tools - Karate: use Default environment: dev and Default parallelism: 1.
  • To run them the first time: right-click on the file, More Run/Debug, Modify Run Configuration, and in the dialog, set: Parallelism: 4, Environment: dev (to run locally with eureka-cli) and change "Working Directory" to the acquisitions directory so it saves reports in the target directory there (as with junit). Then just "Run". The configuration should be saved so the test can just be started with "Run" the next time.

See Karate Coding Standards & Best Practices.

See how to do a module interception with eureka-cli, to debug a new version of a module.

Troubleshooting

Error when running tests:
setup-users.feature:105 - status code was: 422, expected: 201, response time: 18, url: http://localhost:9130/perms/users, response: {"errors":[{"code":"-1","message":"Unable to update derived fields: Attempting to add non-existent permissions invoice.all to permission user with id 4334831c-02d7-4936-849c-2f44d5491300","type":"1",

-> just deploy mod-invoice or whatever permissions are required for


Permission problems, for instance
2021-05-19T14:00:10,684 ERROR [vert.x-eventloop-thread-1] RestClient Exception calling GET /organizations-storage/organizations/c6dace5d-4574-411e-8ba1-036102fcdc9b - org.folio.rest.core.models.RequestContext@4286b5d9
java.util.concurrent.CompletionException: org.folio.invoices.rest.exceptions.HttpException: Access requires permission: organizations-storage.organizations.item.get

-> Check descriptor (clean+compile), redeploy


maven-resources-test:mod-finance-storage: java.lang.OutOfMemoryError: Java heap space

-> Increase Settings - Build, Execution etc - Compiler - Shared build process heap size and reload all projects

PR workflow

API and module versions

When modifying the API in ModuleDescriptor-template, update the API version number in the related path in it, for instance:
      "id": "finance.transactions",
      "version": "4.2",
Make a field required -> major API version (for storage + business)
Change a pattern (validation) -> no API version change.
Migration → no module version change
Major API version change -> major module version change.
Never modify pom.xml or NEWS.md unless this is a release PR.

Basic git operations

(assuming the JIRA ticket worked on is "MODORDSTOR-212")

Create the new branch locally:
git checkout -b MODORDSTOR-212
(even if there are changes: they will be moved to the new branch)
(if that doesn't work, try "git switch -c <new-branch>")

Commit changes:
git add .     (to undo, use git reset)
git commit -m "[MODORDSTOR-212] ..."  (to undo, git reset HEAD~ )

Push to the repo the first time (this is reminded when using simply "git push")
git push --set-upstream origin MODORDSTOR-212

after:
git push

To go back to the master branch:
git checkout master

Finish the PR creation on github

Go to the repo on github, use the new Create pull request button.
See sample at https://github.com/folio-org/mod-orders-storage/pull/230
Assign PR to myself.

After PR is created

Fix code smells etc.

If the build fails with continuous-integration/jenkins/pr-merge, try to click login and rerun buttons on top in jenkins.

Attach a screenshot with Karate tests to show it does not introduce a regression (no new test needed at this point).

Add folio-org/acquisitions team to reviewers on github.

Change JIRA ticket to "Code review".

After the PR is approved by 2 people, the initial poster has to merge it.
Merge with master before if needed.
Check boxes in the PR.
Squash&Merge PR.

After the pull request is merged in github:

Change the "Fix Version" (version can be found in pom.xml, just remove "-SNAPSHOT") and change JIRA ticket to "Review".
Ticket is still assigned to me.

Tests

Create a new PR for the karate tests in folio-integration-tests, with a screenshot of the test results, using the same branch name based on the JIRA ticket.

Possibly test user interface and add a comment to the JIRA ticket about how to test.

Review

Set the JIRA ticket "Assignee" to "Unassigned".

Based on your steps other guy from team will recheck it again and close if everything fine.