UXPROD-3944 NFR Scorecard
Please, refer to NFR Scorecard practice for detailed information
Quality Attribute | NFR ID | Non-Functional Requirement | Preliminary Analysis (Before feature started)- Date and Status | Final Analysis (After feature completed) - Date and Status | Notes and Comments |
|---|---|---|---|---|---|
Availability | NFR.Availability.1 | Modules are designed and implemented following the Stateless principle | NON COMPLIANT | COMPLIANT | Now scheduling configuration stored in memory and module can work only with 1 instance. After feature implementation scheduling configuration will be stored in Quartz internal tables, can be scaled and will filly satisfy stateless principles |
NFR.Availability.2 | Load / performance testing must be conducted for at least 2 instances | NOT APPLICABLE | COMPLIANT | Before feature started scheduling can work only with single module of mod-data-export-spring enabled. After feature implemented we need to retest with 2 instances to check that implementation is reliable, we don’t expect that performance increased a lot, but we should verify that export completed successfully with 2 instances. We need to test by our self to run multiple jobs at the same time, test that scheduling remain working after instances restart without manually enabling them, check that after some instance destroyed another instance pick ups scheduling work. Also we need to create separate ticket to Kitfox to re-test export(they already have jmx script to test it) | |
Manageability | NFR.Manageability.1 | Application logs are collected in unified form and location | COMPLIANT | Standard logging approach | |
NFR.Manageability.2 | All custom configuration values are placed in the settings, not in the program code | COMPLIANT | Will use Spring application.yml for all Quartz properties, so Quartz can be configured without modifying source code(clustered or not, checkinInterval, etc) See https://github.com/folio-org/mod-data-export-spring/blob/master/src/main/resources/application.yml | ||
Performance | NFR.Performance.1 | Components are performance tested and compared to the prior release baseline; performance may not degrade more than 5% in exceptional cases | NOT APPLICABLE | We don’t have any performance baseline for old scheduler and for new implementation with Quartz it will also not applicable | |
Security | NFR.Security.1 | Tenant data must be isolated from other tenants | COMPLIANT | COMPLIANT | We will try to use only configId and tenantId in common quartz schema for all tenants. All job details will be not stored in common SQL schema |
NFR.Security.2 | Secrets (such like usernames, passwords, API keys and/or their combinations) are not stored in source repositories (i.e. Github) | COMPLIANT | COMPLIANT | Will need code review for the final implementation | |
NFR.Security.3 | No sensitive information in logs (logins, passwords, API keys) | COMPLIANT | COMPLIANT | Will need code review for the final implementation, check logs don’t have any sensitive information | |
Testability | NFR.Testability.1 | Unit-test coverage for new code created / changed during the implementation of the feature >= 80% | COMPLIANT | COMPLIANT | Will need check Sonar code coverage after final implementation See https://sonarcloud.io/summary/new_code?id=org.folio%3Amod-data-export-spring |
NFR.Testability.2 | E2E-test coverage - # of automated tests cases from test rail to # of all test cases at particular feature | COMPLIANT | COMPLIANT | We already had 4 automation tests for Edifact Exports: We already have different manual tests cases for Edifact Export, we need to write automation tests for them based on priority We don’t have any automation tests for bursar export, we have only manual tests cases written that need to be passed after Quartz implementation There are Bursar test cases and scheduling test cases in TestRail- please make QA runs applicable tests before the story is closed | |
NFR.Testability.3 | Karate-test coverage - # of test to # of new endpoints that were created (or existing endpoints that were changed) in the feature scope | NON COMPLIANT | COMPLIANT | We did not have karate tests for both Edifact and Bursar exports before feture started. After feature implementation we are going to add some karate tests to check basic scheduling flows https://folio-org.atlassian.net/browse/MODEXPS-202 (no tests for failure scenarios) |