Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


AuthorAndrei Makaranka
JIRA task

Jira Legacy
serverSystem JiraJIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyUXPROD-2899

Business Requirements
Architects Review

Raman Auramau

Status
colourYellowGreen
titleIn ProgressDONE

PO review

Dennis Bridges

Status
colourYellowGreen
titleIn ProgressDONE

Table of Contents
maxLevel5

...

VersionDateOwnerDescriptionReason
v1.006.14.2022Andrei Makaranka Initial version
v2.006.22.2022Schema changed and WBS added

Overview

Allow user to generate a rollover report that will tell the user what errors may occur if rolling over with current data.

Preview report should contain : finance rollover errors (budget errors) and clone of new generated budgets.

Constraints

  • The current approach can cover rollover for the only one ledger though in real life there can be several ledgers - but this needs to be considered (warning)

Assumptions, dependencies

  • Ledger Fiscal Year Rollover already implemented.
  • It is likely that we will be creating upwards of 75000 - 100000 encumbrances during this process. (statistics from Library). 
  • Prepare report with Unpaid invoices is UI responsibility.

Out of Scope

  • Prepare report for orders rollover.
  • Prepare report for encumbrances rollover.
  • Support exchange rate.

Architecturally Significant Requirements

  • The preview process should not make changes in real tables, but store the results in a temporary or separate tables.
  • Finance rollover preview should be done by the same SQL script as used for Real rollover process.
  • Must be compatible with PostgreSQL 10.x

Solution Design

...


v2.106.30.2022Status for preview added
v3.007.07.2022Andrei Makaranka Redesign and rollover logs functionality added Requirements changed
v4.008.03.2022Andrei Makaranka Rename "action" field tp "rolloverType"

Overview

Allow user to generate a rollover report that will tell the user what errors may occur if rolling over with current data.

Preview report should contain : finance rollover errors (budget errors) and clone of new generated budgets.

Constraints

  • The current approach can cover rollover for the only one ledger though in real life there can be several ledgers - but this needs to be considered (warning)

Assumptions, dependencies

  • Ledger Fiscal Year Rollover already implemented.
  • It is likely that we will be creating upwards of 75000 - 100000 encumbrances during this process. (statistics from Library). 
  • Prepare report with Unpaid invoices is UI responsibility.

Out of Scope

  • Prepare report for orders rollover.
  • Prepare report for encumbrances rollover.
  • Support exchange rate.

Architecturally Significant Requirements

  • The preview process should not make changes in real tables, but store the results in a temporary or separate tables.
  • Finance rollover preview should be done by the same SQL script as used for Real rollover process.
  • Must be compatible with PostgreSQL 10.x

Solution Design

  • Current scheme for rollover records - add a column/flag to mark every rollover if it is a real or a preview one - this will also keep the history of completed previews
  • mod-finance-storage-master\src\main\resources\templates\db_scripts\budget_encumbrances_rollover.sql - make this script parameterizable; pass a real/preview flag as the input parameter; use different tables depending on the flag value. This will allow not to duplicate the SQL script, but to use the same script for both the real rollover and the preview, and thereby get the correct results for the preview.
  • Add another table to store preview data, including Export budget and errors that have occurred; data from this table can then be exported to any convenient format or displayed on the UI

...

PropertyTypeRead OnlyDefaultRequiredNotes

actionrolloverType

enum ["Preview", "Commit", "Rollback"]


truefalseCommit

Y

  • Current scheme for rollover records - add a column/flag to mark every rollover if it is a real or a preview one - this will also keep the history of completed previews

Create

...

ledger fiscal year rollover Budget schemas

Shema name : ledger_fiscal_year_rollover_budget

Schema should be copy of ramls/acq-models/mod-finance/schemas/budget.json where all amounts are "persistent"(remove readOnly flag from all amounts).

Create new table "preview_ledger_fiscal_year_rollover"

Schema for the table "preview_ledger_fiscal_year_rollover"Also link on Ledger Rollover ("ledgerRolloverId")

Code Block
languagejsgroovy
themeMidnight
title ledger_fiscal_year_rollover_budget
collapsetrue
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "description": "Preview ledger fiscal year rolloverA budget",
  "type": "object",
  "propertiesjavaName": {
    "id": {
   budget",
  "descriptionextends" : "The unique id of this category",
 {
    "$ref" : "../../common/schemas/uuidentity.json"
 
  },

    "ledgerFiscalYearRolloverproperties": {
 
    "descriptionid": "Fiscal{
year rollover",
      "typedescription": "objectUUID of this budget",
      "$ref": "ledger_fiscal_year_rollover../../common/schemas/uuid.json"
    },
    "previewResultledgerRolloverId": {
      "description": "ResultLedger fiscal ofyear previewrollover runUUID",
      "type$ref": "object",../../common/schemas/uuid.json"
    },
 "$ref": "preview_ledger_fiscal_year_rollover_result.json"
    },
    "totalRecordsname": {
      "description": "The numbername of records contained in this collectionthe budget",
      "type": "integerstring",
    },
    "minimumbudgetStatus": 0
   {
}   },   "additionalPropertiesdescription": false,"The status of the budget",
      "type": "string",
      "requiredenum": [
        "ledgerFiscalYearRolloverActive",
        "totalRecords"Frozen",
       ]
}

Schema for the preview result "preview_ledger_fiscal_year_rollover_result"

Code Block
languagejs
themeMidnight
collapsetrue
{
  "$schema": "http://json-schema.org/draft-04/schema#", "Inactive",
        "descriptionPlanned":,
"Preview ledger fiscal year rollover result",   "type": "object",Closed"
      ]
   "properties": { },
    "errorsallowableEncumbrance": {
      "description": "The listencumbrance ofpercentage fiscallimit yearfor rolloverthis errorsbudget",
      "type": "arraynumber",
    },
    "itemsallowableExpenditure": {
 
      "$refdescription": "ledger_fiscal_year_rollover_error.json"
 The expenditure percentage limit for this budget",
       }"type": "number"
    },
    "budgetsallocated": {
      "description": "The amount currently listallocated ofto plannedthis budgetsbudget",
      "type": "arraynumber",
    },
    "itemsawaitingPayment": {
 
      "$refdefault": "preview_rollover_budget.json"0,
      }"description": "The amount currently awaiting payment for this budget",
      "type": "number"
    },
    "totalRecordsavailable": {
      "description": "The numberamount ofcurrently recordsavailable contained infor this collectionbudge",
      "type": "integernumber",
    },
    "minimumencumbered": 0{
    }  "default": }0,
      "additionalPropertiesdescription": false, "The amount currently encumbered for this budget",
      "requiredtype": [ "number"
    },
    "totalRecords"expenditures": {
     ]
}

APIs

...

 "default": 0,
      "description": "The amount currently expended for this budget",
      "type": "number"
    },
    "netTransfers": {
      "default": 0,
      "description": "Summing all the transfers on this budget",
      "type": "number"
    },
    "unavailable" : {
      "description": "Unavailable amount for this budget",
      "type": "number"
    },
    "overEncumbrance": {
      "description": "The amount currently over-encumbered for this budget",
      "type": "number"
    },
    "overExpended": {
      "description": "Amount the budget is over expended.",
      "type": "number"
    },
    "fundId": {
      "description": "UUID of fund",
      "$ref": "../../common/schemas/uuid.json"
    },
    "fiscalYearId": {
      "description": "UUID of fiscal year",
      "$ref": "../../common/schemas/uuid.json"
    },
    "acqUnitIds": {
      "description": "acquisition unit ids associated with this budget",
      "type": "array",
      "items": {
        "$ref": "../../common/schemas/uuid.json"
      }
    },
    "tags": {
      "type": "object",
      "description": "arbitrary tags associated with this budget",
      "$ref": "../../../raml-util/schemas/tags.schema"
    },
    "metadata": {
      "type": "object",
      "$ref": "../../../raml-util/schemas/metadata.schema",
      "readonly": true
    },
    "initialAllocation": {
      "default": 0,
      "description": "The amount of the first allocation made to this budget",
      "type": "number"
    },
    "allocationTo": {
      "default": 0,
      "description": "The sum of all allocation transaction amounts made TO this budget not including the Initial allocation",
      "type": "number"
    },
    "allocationFrom": {
      "default": 0,
      "description": "The sum of all allocation transaction amounts made FROM this budget not including the Initial allocation",
      "type": "number"
    },
    "totalFunding": {
      "description": "budget.allocated + budget.netTransfers",
      "type": "number"
    },
    "cashBalance": {
      "description": "Total Funding minus Expended",
      "type": "number"
    }
  },
  "additionalProperties": false,
  "required": [
    "budgetStatus",
    "name",
    "fundId",
    "fiscalYearId"
  ]
}


Schema name: ledger_fiscal_year_rollover_budget_collection

Code Block
languagegroovy
themeMidnight
titleledger_fiscal_year_rollover_budget_collection
collapsetrue
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "description": "Contains a list of budgets generated during the rollover. Also budgets include all calculated mounts",
  "type": "object",
  "properties": {
    "budgets": {
      "description": "The list of fiscal year rollover errors",
      "type": "array",
      "items": {
        "$ref": "ledger_fiscal_year_rollover_budget.json"
      }
    },
    "totalRecords": {
      "description": "The number of records contained in this collection",
      "type": "integer",
      "minimum": 0
    }
  },
  "additionalProperties": false,
  "required": [
    "budgets",
    "totalRecords"
  ]
}

Create new DB Table "ledger_fiscal_year_rollover_budget" → schema.json

In the database will be stored records with type of "ledger_fiscal_year_rollover_budget"


Code Block
themeMidnight
titleledger_fiscal_year_rollover_budget
collapsetrue
    {
      "tableName": "ledger_fiscal_year_rollover_budget",
      "fromModuleVersion": "mod-finance-storage-8.1.7",
      "withMetadata": true,
      "index": [
        {
          "fieldName": "ledgerRolloverId",
          "tOps": "ADD",
          "caseSensitive": false,
          "removeAccents": true
        },
        {
          "fieldName": "fundId",
          "tOps": "ADD",
          "caseSensitive": false,
          "removeAccents": true
        },
        {
          "fieldName": "fiscalYearId"
        }
      ],
      "foreignKeys": [
        {
          "fieldName": "ledgerRolloverId",
          "targetTable": "ledger_fiscal_year_rollover",
          "tableAlias": "ledgerFYROBudget",
          "targetTableAlias": "ledgerFYRO",
          "tOps": "ADD"
        }
      ]
    }

Create ledger fiscal year rollover Logs schemas 

Schema name : ledger_fiscal_year_rollover_log

PropertyTypeRead OnlyDefaultRequiredNotes

ledgerRolloverId

UUID


trueNA

Y


startDatedate-timetrueNAYThis is "creratedDate" from rollover metadata. 
endDatedate-timetrueNA
This is "updatedDate" from rollover metadata. Should be provided only when "status" is "Success" or "Error". 
rolloverStatusramls/acq-models/mod-finance/schemas/rollover_status.jsontrueNAY
errorsLinkString or patterntrueNANIf possible Link should included environment URL : https://okapi-bugfest-lotus.int.aws.folio.org/finance/ledger-rollovers-errors?query=ledgerRolloverId=={id}fff-e-errors 
budgetsLinkString or patterntrueNANIf possible Link should included environment URL : https://okapi-bugfest-lotus.int.aws.folio.org/finance/ledger-rollovers-budgets?query=ledgerRolloverId=={id}fff-
ledgerRolloverTypeenum ["Preview", "Commit", "Rollback"]trueNAY


Schema name : ledger_fiscal_year_rollover_log_collection

Code Block
languagegroovy
themeMidnight
titleledger_fiscal_year_rollover_budget_collection
collapsetrue
{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "description": "Contains a list of rollover runs",
  "type": "object",
  "properties": {
    "logs": {
      "description": "The list of  rollover runs",
      "type": "array",
      "items": {
        "$ref": "ledger_fiscal_year_rollover_log.json"
      }
    },
    "totalRecords": {
      "description": "The number of records contained in this collection",
      "type": "integer",
      "minimum": 0
    }
  },
  "additionalProperties": false,
  "required": [
    "budgets",
    "totalRecords"
  ]
}

APIs

Ledger Rollover generated Budgets Business API

Method

Path

Request

Response

Description

Interface

Notes

POST
GET/finance/ledger-rollovers
/previewsramls/acq-models/mod-finance/schemas/
-budgets/{id}NA

ledger_fiscal_year_rollover_budget

GET budgets of the ledger fiscal year rollover by idfinance.ledger-rollovers-budgets.item.getStatus : 200
GET/finance/ledger-rollovers-budgetsNAledger_fiscal_year_rollover
.jsonpreview
_
ledger_fiscal_year_rolloverRun preview
budget_collectionGET budgets of the ledger fiscal year rolloversfinance.ledger-rollovers
.previews
-budgets.collection.
post
getStatus :
201
200

Ledger Rollover generated Budgets Storage API

Method

Path

Request

Response

Description

Interface

Notes

GET/finance/ledger-rollovers
/previews
-budgets/{id}NA
preview_

ledger_fiscal_year_rollover_budget

GET
preview
budgets of the ledger fiscal year rollover by idfinance.ledger-rollovers
.previews
-budgets.item.getStatus : 200
GET/finance/ledger-rollovers
/previews
-budgetsNA
preview_
ledger_fiscal_year_rollover_budget_collectionGET
previews
budgets of the ledger fiscal year rolloversfinance.ledger-rollovers
.previews
-budgets.collection.getStatus : 200
DELETE

Ledger Rollover Logs Business API

-storage/previewsDELETE  preview .previewsdelete 204

Method

Path

Request

Response

Description

Interface

Notes

GET/finance/ledger-rollovers-logs/{id}NANA

ledger_fiscal_year_rollover_log

GET logs of the ledger fiscal year rollover by idfinance.ledger-rollovers-logs.item.getStatus :

Preview Ledger Rollover Storage API

POSTpreview_

Method

Path

Request

Response

Description

Interface

Notes

200
GET/finance-storage/ledger-rollovers/previewsramls/acq-models/mod-finance/schemas/-logsNA

ledger_fiscal_year_rollover.json

preview

_

ledger_fiscal_year_rolloverPOST previews

log_collection

GET logs of the ledger fiscal year rolloversfinance-storage.ledger-rollovers.previews-logs.collection.postgetStatus : 201GET

/finance-storage/ledger-rollovers/previews/{id}

NA200

Design


Work Breakdown Structure


WorkCommentsStory~ Estimate
1
  1. Update schema "ledger_fiscal_year_
rolloverGET preview of the
  1. rollover" with new "action" field
  2. Define schema and table for preview rollover results → Define new schemas and table

 2.1. Create ledger fiscal year rollover

by idfinance-storage.ledger-rollovers.previews.item.getStatus : 200GET/finance-storage/ledger-rollovers/previewsNApreview_ledger_fiscal_year_rollover_collectionGET previews of the ledger fiscal year rolloversfinance-storage.ledger-rollovers.previews.collection.getStatus : 200DELETE/finance-storage/ledger-rollovers/previews/{id}NANADELETE previews of the ledger fiscal year rollover by idfinance-storage.ledger-rollovers.previews.item.deleteStatus : 204

Design

Work Breakdown Structure

WorkCommentsStory~ Estimate1
  1. Update schema "ledger_fiscal_year_rollover" with new "action" field
  2. Create duplicate of the Budget schema where all amounts are "persistent"
  3. Define schema and table for preview rollover results → Define schema and table for preview rollover results
acq-models, mod-finance-storage,

Budget schemas
 2.2. Create ledger fiscal year rollover Log schemas
 2.3. Define new table for storing generated budgets in scope of Rollover

acq-models, mod-finance-storage, mod-finance

Jira Legacy
serverSystem JIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODFISTO-319

1
2
  1. Define Storage API : Preview Ledger Rollover → Storage API : Ledger Rollover Budgets
  2. Implement API → Should be without any logic.
 mod-finance-storage

Jira Legacy
serverSystem JIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODFISTO-320

1
3
  1. Define and Implement Budgets API → Budgets Ledger Rollover Business API
  2. Define and Implement Logs API → Logs view should be built accordingly with ledger_fiscal_year_rollover_log Ledger Rollover Logs Business API
mod-finance2
  1. Define Storage API : Preview Ledger Rollover → Storage API : Preview Ledger Rollover
  2. Implement API → Should be without any logic.
  3. Business logic for POST will be implemented in scope of MODFISTO-318
 mod-finance-storage

Jira Legacy
serverSystem JiraJIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODFISTO-319

1

MODFIN-256

1
4
  1. Spike : Implement POC with storing preview result in the temporary tables and after that upload results in the preview rollover table

mod-finance-storage

All all open technical questions should be resolved

Jira Legacy
serverSystem JiraJIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODFISTO-320322

15
3
  • Define API : Preview Ledger Rollover → Business API : Preview Ledger Rollover
  • Implement API → Should be Proxy to storage layer without any logic
  • Sending email will be added in scope of MODFIN-2575
    1. Update ledger rollover SQL script with preview rollover logic and store results in the preview rollover table
    mod-finance-storage

    Jira Legacy
    serverSystem JiraJIRA
    serverId01505d01-b853-3c2e-90f1-ee9b165564fc
    keyMODFINMODFISTO-256323

    1
    46
    1. Spike : Implement POC with storing preview result in the temporary tables and after that upload results in the preview rollover table

    mod-finance-storage

    All all open technical questions should be resolved

    Jira Legacy
    serverSystem Jira
    serverId01505d01-b853-3c2e-90f1-ee9b165564fc
    keyMODFISTO-322

    55Implement logic for storing preview result in the temporary tables and after that upload results in the preview rollover table

    Implement Preview rollover flow and update budgets with all calculated amounts

    1. Steps should be run in scope of one DB transaction :

    1.1 Rollover script run (field LedgerFiscalYearRollover."action" = "Preview")
    1.2 Retrieve result of preview from table "ledger_fiscal_year_rollover_budget", where budgets amounts will be partially filled (MODFISTO-323)
    1.3. Go through all budgets and calculate remaining amounts → org.folio.utils.CalculationUtils#calculateBudgetSummaryFields
    1.4. Storing updated budgets in the rollover preview table → ledger_fiscal_year_rollover_budget

      2. Update delete rollover logic : org.folio.service.rollover.LedgerRolloverService#deleteRollover. Rollover record  deletion should happened together with rollover budgets in scope of one DB transaction

    mod-finance-storage

    Jira Legacy
    serverSystem JiraJIRA
    serverId01505d01-b853-3c2e-90f1-ee9b165564fc
    keyMODFISTO-323318


    6

    Implement Preview rollover flow and update budgets with all calculated amounts

    1. Steps should be run in scope of one DB transaction :
    1.1 Rollover script run (field LedgerFiscalYearRollover."action" = "Preview")
    1.2 Retrieve result of preview from table "preview_ledger_fiscal_year_rollover", where budgets amounts will be partially filled (MODFISTO-323)
    1.3. Go through all budgets and calculate remaining amounts → org.folio.utils.CalculationUtils#calculateBudgetSummaryFields
    1.4. Storing updated budgets in the rollover preview table → preview_ledger_fiscal_year_rollover7

    Replace unique index in the table "ledger_fiscal_year_rollover" with uniqueness logic in the code

    1. Remove index "ledgerId_fromFiscalYearId" from table "ledger_fiscal_year_rollover"
    2. Implement validation logic in the separate validation service:

    2.1 If incoming rollover is real with action "Commit", then rollover record should be unique for pair "ledgerId, fromFiscalYearId" (the same as in the index)

    2.2 If incoming rollover is real with action "Preview", then rollover record should not be unique for pair "ledgerId, fromFiscalYearId"  and it possible to have many previews for one ledger and same fiscal year.

    mod-finance-storage

    Jira Legacy
    serverSystem JiraJIRA
    serverId01505d01-b853-3c2e-90f1-ee9b165564fc
    keyMODFISTO-318327


    78
    1. Spike : To gain knowledge about mod-email and implement example with usage email sending
    mod-finance

    Jira Legacy
    serverSystem JiraJIRA
    serverId01505d01-b853-3c2e-90f1-ee9b165564fc
    keyMODFIN-258


    89
    1. Implement sending email and include it in the preview ledger rollover flow
    mod-finance

    Jira Legacy
    serverSystem JiraJIRA
    serverId01505d01-b853-3c2e-90f1-ee9b165564fc
    keyMODFIN-257


    ...