Overview

A non-functional requirement (NFR) is a requirement that specifies criteria that can be used to judge the operation of a system, rather than specific behaviors. The issue of NFRs and their importance for the stability and high quality of the FOLIO platform is mentioned and discussed quite regularly. At the same time, at the moment there is no practical application of these requirements, their accounting, and control of implementation for functional features. The described approach with the NFR Scorecard aims to close this gap.

The idea is to have a list of NFRs, so that when analyzing and accepting each feature, the team (PO + SM + TL + SA) checks with it and explicitly marks compliance/applicability / non-applicability. Accordingly, the list should be relatively small and manageable so as not to complicate things. On the contrary, the main idea is to simplify and focus only on key requirements.

Important note - this scorecard is intended for development teams and only covers those requirements and quality attributes that are the responsibility of the development teams (and not DevOps or hosting providers).

Scope: Components being released within FOLIO.

Owner(s) (owners approve additions, modifications, or removals of NFRs): Raman Auramau

Using the NFR Scorecard

The following steps are needed to start receiving benefits from the NFR Scorecard practice:

As one can see from the presented workflow, the summarized expected duration of activities associated with the NFR Scorecard process is 90-150 minutes per feature.

Expected Benefits

Although following the described approach may seem like unnecessary overhead, it provides an excellent opportunity to analyze compliance with key non-functional requirements at an early stage. This allows the team to detect problems earlier, even at the development stage, before providing the code to the Bugfest and beyond. In turn, this means a faster feedback cycle and cheaper fixes.

Review and Modifications of the NFR Scorecard

Once per release, the NFRs should be reviewed by the Owner(s) for Applicability, Effectiveness, ROI and TCO, and Improvements. Changes and additions may be made as a result of this process.

FOLIO NFR Scorecard v0.4

Note: The NFR ID is suggested as a unique identifier for specific non-functional requirements, such as when detailing tasks for NFR Scorecard compliance or during discussions. It's essential to recognize that the NFR ID's uniqueness is not automatically controlled or enforced but it is a recommended aspect of the process. For this, the proposed approach involves using the Baseline keyword for identifiers from the baseline NFR scorecard, such as NFR.Baseline.Performance.1. This keyword can be substituted with a feature number or another value for additional, feature-specific non-functional requirements, for instance, NFR.3944.Performance.1 (for UXPROD-3944) or NFR.DI.Performance.1 (for Data Import).

Glossary

• Availability defines the proportion of time that the system is functional and working. It can be measured as a percentage of the total system downtime over a predefined period. Availability is affected by system errors, infrastructure problems, malicious attacks, and system load.

• Manageability defines how easy it is for system administrators to manage the application, usually through sufficient and useful instrumentation exposed for use in monitoring systems and for debugging and performance tuning.

• Performance is an indication of the responsiveness of a system to execute any action within a given time interval. It can be measured in terms of latency or throughput. Latency is the time required to respond to any event. Throughput is the number of events that take place within a given amount of time.

• Security is the capability of a system to prevent malicious or accidental actions outside of the designed usage and to prevent disclosure or loss of information. A secure system aims to protect assets and prevent unauthorized modification of information.

• Testability is a measure of how easy it is to create test criteria for the system and its components and to execute these tests to determine if the criteria are met. Good testability makes it more likely that faults in a system can be isolated in a timely and effective manner.

Pilot run in Poppy R2 2023

Next steps

• Review the FOLIO NFR Scorecard with stakeholders and interested, agree on working and metrics,

• Choose 2 features in the Poppy release to give a try in practice (different teams, SAs, POs),

  • Poppy R2 2023 - NFR Compliance Review - create a NFR epic in ARCH project - https://folio-org.atlassian.net/browse/ARCH-45

  • Collect feedback and adjust the NFR Scorecard if needed, agree on particular features for piloting

  • Pilot NFR Scorecard

  • Collect feedback and adjust the NFR Scorecard if needed

• Introduce the NFR Scorecard as a recommended practice in the Quesnelia release

  • Develop a tool for monitoring and analysis

• Collect feedback after Quesnelia and adjust the NFR Scorecard if needed