Page Comparison

Overrides/Supersedes 

...

N/A

Constraints

N/A

Rationale

Analysis and options

Basically, 2 main alternatives can be suggested for the provided context:

...

Alternative option

...

Brief description

...

Potential use cases

...

• cascade deletion of outdated records

...

• manual deletion made by end users

It looks that both options can be viable for FOLIO depending on particular use case.

It worth also noting that Image RemovedUICR-125 - Data corruption. When holding/item data are moved in Inventory, then the item in Courses is not updated accordingly OPENcase is slightly different and is about dangling references while not deletion but moving of a core-module record; neither cascade nor soft deletion is applicable here. At the same time, this case and cascade deletion case have one great similarity - in both, non-core-modules are to be notified about a change in core-module. In this context, a robust notification channel with guaranteed delivery to transfer change events from one module (say, module-source) to another module (say, module-recipient) is required.  More details are on Data consistency and message driven approach # Notification channel.

Soft deletion option

More details regarding this option.

• Q: It requires that every client understands that deleted records might be present and when the API will provide them and when it will not.
  • A: Sure, every client should know the rules of the game. Though once the rules will be agreed and shared, it's just a platform-wide knowledge
• Q: Let's say a loan references an item, that item is then soft-deleted, when the loan is presented in the UI, would the item information be presented or not?
  • A: Yes, the loan is available, and the item information still can be retrieved by a single endpoint. At the same time, loan UI should be able to notify end user that the item has been deleted (via a warning message, or via a dedicated UI field, or via disabling of editing particular values received from the item

High level actions required for core / non-core modules are shown below:

...

Core module

...

Another module referencing core module

...

• add an is_deleted flag to schema via Liquibase with false as a default value
• Create - new records are to be created with is_deleted=false
• Read
  • collection endpoint only returns not-deleted records
  • single endpoint can return both deleted and not-deleted records
• Update - add a check for every update - update of a record with is_deleted=false is only allowed
• Delete - implement soft-deletion with setting is_deleted to true

...

• add an UI element showing if a record from code module is deleted or not, or disable editing of fields came from core module, or add a message to end user with the warning that a core module record this record is referencing to, is deleted

Decision

Yikes

API Integration Test Goal

1. Automated regression
2. Cross module interaction 
3. Validate Producer/Consumer relationship
4. Permissions
5. No mock objects

QA Goals for 2021

Scope 

1. Identify most important business flows
2. Add test case to cover a bug condition
3. Don't write tests for all possible conditions

Please note that the intent is not to replace rest assured/junit with karate but instead to get the benefits of both approaches

Create tests for new features

1. Review if new feature has important business workflows and include test creation for these workflows into estimates
2. Features are all ranked and it should determine if tests have to be created 
3. SM need to drive NFR and make sure that Karate tests are scheduled into Sprints 

Test plan creation and estimation

1. Before the end of R1 teams have to analyze existing API and identify important business flows  and created stubs for them.
2. Create Jira tickets for implementation of this tests.
3. Create an estimate for each Jira ticket
4. Evaluate each workflow based on downstream dependencies

Blocked by:

Image AddedFAT-145 - Annotate scenarios and customize Cucumber report for viewing "Skip" tests in results. CLOSED

Image AddedFAT-146 - POC : Solution for mapping pure Gherkin scenarios with Karate feature runs OPEN

Coverage calculation, reporting

1. Coverage defined as % of implemented test cases to total test case count.
2. Coverage will be calculated and reported by Jenkins pipeline

Test execution before and after PR

1. Part of acceptance criteria - execute Katate tests before PR
2. Deploy module in Scratch env. and run applicable tests
3. Module changes and test changes are merged at the same time
4. Test will be run automatically after PR in Jenkins pipeline the next day:

   1. https://jenkins-aws.indexdata.com/job/FOLIO_Reference_Builds/job/folio-api-tests-karate/

   2. https://jenkins-aws.indexdata.com/job/FOLIO_Reference_Builds/job/folio-api-tests-postman/

These pipelines are currently broken -  Image AddedFOLIO-2982 - API tests environment (for karate, postman) is fail to spin up CLOSED

Changes to DOD

1. Every team should adjust DOD to require API Integration Tests 

Deprecation of Postman tests

1. Mostly affects Spitfire and Thunderjet
2. Continue to use Postman tests if team doesn't have desire to switch
3. Passing tests are required  by acceptance criteria
4. Postman test pipeline must be:
   1. operational 
   2. have no failures
5. Defer decision when to deprecate these tests. 

Decision

Accepted

Implications

• Pros
  • N/A
• Cons
  • N/A

Other Related Resources

...

Proposed Action Items

•  Each team should change definition of done to make API integration tests required
•  All new BE features must include Karate tests. 
•  R1 Goal: Each team has to create a test plan for existing API interfaces
•  R1 Goal: Each team should provide estimates for the test plan. Estimates will be presented at the Capacity Planning meeting so that this work can be officially scheduled into sprints.

Meeting Notes

• GitHub repo and documentation: https://github.com/folio-org/folio-integration-tests
• Karate resources are at the bottom of README.md
• Karate API Integration Testing  implementation tracking page