| Page Properties | ||||||||
|---|---|---|---|---|---|---|---|---|
|
Overrides/Supersedes
This decision was migrated from the Tech Leads Decision Log as part of a consolidation process. The original decision record can be found here.
RFC
N/A
Stakeholders
- All Backend Technical Leads, Developers
Contributors
Anton Emelianov (Deactivated)Julian Ladisch
Approvers
This decision was made by the Tech Leads group prior to the adoption of current decision making processes within the FOLIO project.
Background/Context
...
- Technical council, via lazy consensus. See 2023-10-09 Meeting notes
Background/Context
Request of the Security Group on 2021-08-20:
- Binary files must be stored with strict tenant and module separation.
- A FOLIO MinIO security guide for developers and sysops must be published and reviewed by the security team before more modules start using it.
- e.g. Including guidance for how to do the tenant/module separation.
- The tech leads group will discuss this as noted during the TC meeting (see TC 2021-08-18 Meeting notes).
Tech Leads Meeting on 2021-08-25:
Note: we have discussed this topic during the Tech Leads meeting and decided to form a working group. The task of the group will be to define a recommended standard for object storage, including handling of tenant seperation. Since developers involved in the S3/MinIO inlcusion in Data Import are not longer on the project (Kruthi Vuppala. Taras Spashchenko) we will need volunteers to participate in this effort. Ian Ibbotson, Owen Stephens: I asked Marc Johnson to reach out to you guys regarding this effort as you may be active working in this area at the moment. If you are OK with that I'd propose that we create a slack channel for the group.
Hi Jakub Skoczen, please include Vladimir Shalaev, Aliaksandr Fedasiuk and myself to this group. Thank you!
Assumptions
N/A
Constraints
N/A
Rationale
API Integration Test Goal
- Automated regression
- Cross module interaction
- Validate Producer/Consumer relationship
- Permissions
- No mock objects
QA Goals for 2021
...
Priority
...
Action Item
...
R1
...
R2
...
R3
...
1
...
Move testing activities to teams scratch environments
...
50%
...
100%
...
2
...
Build API Integration Test Coverage (Karate framework)
...
35%
...
50%
...
75%
...
3
...
Implement UI Integration Test (New BigTest) - 109 test cases
...
40%
...
100%
...
4
...
Implement manual sprint regression using testing community
...
20%
...
50%
...
100%
...
5
...
Implement UAT events using SIG resources
...
20%
...
50%
...
100%
...
6
...
PO to complete specs for performance acceptance test
...
100%
...
7
...
Implement performance acceptance test
...
50%
...
100%
...
8
...
Extend BugFest time frame (Approved)
Scope
- Identify most important business flows
- Add test case to cover a bug condition
- Don't write tests for all possible conditions
Please note that the intent is not to replace rest assured/junit with karate but instead to get the benefits of both approaches
Create tests for new features
- Review if new feature has important business workflows and include test creation for these workflows into estimates
- Features are all ranked and it should determine if tests have to be created
- SM need to drive NFR and make sure that Karate tests are scheduled into Sprints
Test plan creation and estimation
- Before the end of R1 teams have to analyze existing API and identify important business flows and created stubs for them.
- Create Jira tickets for implementation of this tests.
- Create an estimate for each Jira ticket
- Evaluate each workflow based on downstream dependencies
Blocked by:
FAT-145 - Annotate scenarios and customize Cucumber report for viewing "Skip" tests in results. CLOSED
FAT-146 - POC : Solution for mapping pure Gherkin scenarios with Karate feature runs OPEN
Coverage calculation, reporting
- Coverage defined as % of implemented test cases to total test case count.
- Coverage will be calculated and reported by Jenkins pipeline
Test execution before and after PR
- Part of acceptance criteria - execute Katate tests before PR
- Deploy module in Scratch env. and run applicable tests
- Module changes and test changes are merged at the same time
- Test will be run automatically after PR in Jenkins pipeline the next day:
These pipelines are currently broken - FOLIO-2982 - API tests environment (for karate, postman) is fail to spin up CLOSED
Changes to DOD
- Every team should adjust DOD to require API Integration Tests
Deprecation of Postman tests
- Mostly affects Spitfire and Thunderjet
- Continue to use Postman tests if team doesn't have desire to switch
- Passing tests are required by acceptance criteria
- Postman test pipeline must be:
- operational
- have no failures
- Defer decision when to deprecate these tests.
Decision
AcceptedDecision
Decision of the TC on 2021-08-18:
- The S3 API is the protocol for external (non-Postgres) object storage in FOLIO and standard environment variables/configuration of an S3 server on a per module basis.
- The module implementation of object storage must conform to the FOLIO standards for tenant separation
- The recommended standards will be referenced/pointed out by the tech leads.
- e.g. how tenant separation should be handled, etc.
- Jakub Skoczen will raise the topic with the tech leads
- The recommended standards will be referenced/pointed out by the tech leads.
- Should the min.io client library be the recommended client as object storage choice agnostic library, but modules could also choose the native S3 implementation?
- Yes.
- Should teams be allowed to store binary data in Postgres? In some cases (amount of data is small, etc.)?
- Yes. If external object storage (not Postgres) is required, it must be S3.
- Regardless of which is used, tenant separation still must be done. (x3)
Implications
- Pros
- N/A
- Cons
- N/A
Other Related Resources
Proposed Action Items
- Each team should change definition of done to make API integration tests required
- All new BE features must include Karate tests.
- R1 Goal: Each team has to create a test plan for existing API interfaces
- R1 Goal: Each team should provide estimates for the test plan. Estimates will be presented at the Capacity Planning meeting so that this work can be officially scheduled into sprints.
Meeting Notes
...
...
...