UXPROD-3944 NFR Scorecard

Availability

NFR.Availability.1

Modules are designed and implemented following the Stateless principle

Apr 26, 2023 NON COMPLIANT

Jul 7, 2023 COMPLIANT

Now scheduling configuration stored in memory and module can work only with 1 instance. After feature implementation scheduling configuration will be stored in Quartz internal tables, can be scaled and will filly satisfy stateless principles

NFR.Availability.2

Load / performance testing must be conducted for at least 2 instances

Apr 26, 2023 Not Applicable

Jul 7, 2023 COMPLIANT

Before feature started scheduling can work only with single module of mod-data-export-spring enabled. After feature implemented we need to retest with 2 instances to check that implementation is reliable, we don’t expect that performance increased a lot, but we should verify that export completed successfully with 2 instances. We need to test by our self to run multiple jobs at the same time, test that scheduling remain working after instances restart without manually enabling them, check that after some instance destroyed another instance pick ups scheduling work.

Also we need to create separate ticket to Kitfox to re-test export(they already have jmx script to test it)

See https://folio-org.atlassian.net/wiki/display/DD/Quartz+Scheduling+Implementation+in+mod-data-export-spring#QuartzSchedulingImplementationinmoddataexportspring-Testing

Manageability

NFR.Manageability.1

Application logs are collected in unified form and location

Jul 7, 2023 COMPLIANT

Standard logging approach

NFR.Manageability.2

All custom configuration values are placed in the settings, not in the program code

Jul 7, 2023 COMPLIANT

Will use Spring application.yml for all Quartz properties, so Quartz can be configured without modifying source code(clustered or not, checkinInterval, etc)

See https://github.com/folio-org/mod-data-export-spring/blob/master/src/main/resources/application.yml

Performance

NFR.Performance.1

Components are performance tested and compared to the prior release baseline; performance may not degrade more than 5% in exceptional cases

Jul 7, 2023 Not Applicable

We don’t have any performance baseline for old scheduler and for new implementation with Quartz it will also not applicable

Security

NFR.Security.1

Tenant data must be isolated from other tenants

Apr 26, 2023 COMPLIANT

Jul 7, 2023 COMPLIANT

We will try to use only configId and tenantId in common quartz schema for all tenants. All job details will be not stored in common SQL schema

NFR.Security.2

Secrets (such like usernames, passwords, API keys and/or their combinations) are not stored in source repositories (i.e. Github)

Apr 26, 2023 COMPLIANT

Jul 7, 2023 COMPLIANT

Will need code review for the final implementation

NFR.Security.3

No sensitive information in logs (logins, passwords, API keys)

Apr 26, 2023 COMPLIANT

Jul 7, 2023 COMPLIANT

Will need code review for the final implementation, check logs don’t have any sensitive information

Testability

NFR.Testability.1

Unit-test coverage for new code created / changed during the implementation of the feature >= 80%

Apr 26, 2023 COMPLIANT

Jul 7, 2023 COMPLIANT

Will need check Sonar code coverage after final implementation

See https://sonarcloud.io/summary/new_code?id=org.folio%3Amod-data-export-spring

NFR.Testability.2

E2E-test coverage - # of automated tests cases from test rail to # of all test cases at particular feature

Apr 26, 2023 COMPLIANT

Jul 7, 2023 COMPLIANT

We already had 4 automation tests for Edifact Exports:

https://foliotest.testrail.io/index.php?/suites/view/21&group_by=cases:section_id&group_order=asc&display_deleted_cases=0&group_id=17493

We already have different manual tests cases for Edifact Export, we need to write automation tests for them based on priority

We don’t have any automation tests for bursar export, we have only manual tests cases written that need to be passed after Quartz implementation

There are Bursar test cases and scheduling test cases in TestRail- please make QA runs applicable tests before the story is closed

NFR.Testability.3

Karate-test coverage - # of test to # of new endpoints that were created (or existing endpoints that were changed) in the feature scope

Apr 26, 2023 NON COMPLIANT

Jul 7, 2023 COMPLIANT

We did not have karate tests for both Edifact and Bursar exports before feture started. After feature implementation we are going to add some karate tests to check basic scheduling flows

https://folio-org.atlassian.net/browse/MODEXPS-202

(no tests for failure scenarios)