Testing platform code before rollout

Owned by Jakub Skoczen
Last updated: Dec 03, 2020
2 min read
  1. Unit testing: For RMB and Okapi we require at least 80% ("complete" coverage is highly recommended) unit test coverage (jUnit).
  2. Component-level integration testing: new web API implementations in Okapi and RMB must be verified with integration tests (RestAssured) relying on the embedded (or testContainer) Postgres
  3. Performance regression testing: for major refactoring or optimisations we request performance testing from PTF (Martin Tran) with carrier.io, preferably by providing a branch or a snapshot build of Okapi or a specific module (in case of RMB optimisations), before releasing Okapi or RMB. Our own performance regression testing environment (perf-test-platform) is now defunct. Note: a "community" performance testing environment is being created by PTF and EPAM DevOps engineers (Stanislav) once it is available (R1?) we should consider selecting specific test cases for verification on the environment.
  4. System integration testing:
    1. for major changes in RMB (e.g upgrading RMB to vert.x 4) we need to perform local "smoke" testing by upgrading at least 2 modules (e.g mod-inventory-storage, mod-configuration, etc) to include the new version of RMB and verifying the module's own unit and integration tests work. We also want the updated modules using RMB-snapshot to be deployed on folio-snapshot at least for a week before we attempt a release.
    2. For changes related to migrations a migration process should be retested: partially on local workstation (or a "scratch" environment) on at least 2 major modules (e.g mod-inventory-storage and mod-users). Any changes close to the release deadline (especially during the bugfix period) must be also tested by running a complete FOLIO migration (reach out to Jakub who can arrange for this to be performed by FOLIO DevOps). During standup let's discuss batching changes up for testing. 
    3. For fixes/changes specific to a particular deployment environments and/or infrastructure (e.g related to Aurora DB) – we provide a snapshot build of Okapi or a specific module (RMB) and request that the fix is first verified on the environment where the problem occurs. For fixes provided close to the release deadline (during the bugfix period) we request additional verification to be performed on the Bugfest-like environment (reach out to Hongwei or Craig to arrange for this). Only after successful verification we release the artefacts.