Purpose/Overview:
This will implement most of the API defined by MODFIN-320. The only thing that should not be implemented is the patch actions (this should be implemented later within https://folio-org.atlassian.net/browse/UXPROD-3666#icft=UXPROD-3666).
It should effectively replace the previous implementation using transaction summaries.

Requirements/Scope

• All operations should be executed in one database transaction.

• If one fails, it should have no effect on the data and an error should be returned.

• A lock should be used to update budgets to avoid parallel processing conflicts.

• Transaction summaries and temporary transactions should not be used.

• The current API should not be removed at this point.

• New unit tests should be written using the new API.

• New integration tests should already have been implemented within MODFISTO-458; update them if necessary.

• Update the metadata when updating transactions (creating it was already done in MODFISTO-458 but updating was missing).

• Create new stories to update integration tests and remove the API and tables in mod-finance and mod-finance-storage in a future release.

• Try this PR (Tests to demonstrate issues with parallel processing)

• More sanity checks:

  • make sure the metadata property is present for transactions to update

  • ensure all transactions to update exist, and transactions to create do no exist

  • encumbrance deletion: see validateDeletion() in mod-finance EncumbranceService (error DELETE_CONNECTED_TO_INVOICE)

  • credits and payments: see updateTransaction in CreditService and PaymentService in mod-finance (error 422 UPDATE_PAYMENT_TO_CANCEL_INVOICE and UPDATE_CREDIT_TO_CANCEL_INVOICE).

Approach
The current implementation should be reused as much as possible, but there will be large differences:

• There is currently no all-or-nothing mechanism to delete transactions.

• The temporary transactions are currently used for SQL queries together with existing budgets (see EncumbranceService.getSelectBudgetsQueryForUpdate()); this will have to be redesigned because we will no longer have temporary transactions.

• The different transaction types (encumbrances, payments etc) are currently processed separately - it would be nice to be able to process all types in one request, although in practice we currently only need to support one type at a time; if we only support one type at a time, we should check that all transactions are of the same type before processing.

• The current implementation only supports processing one order or one invoice at a time (the id is given by the summary id); we should support processing more than one order at a time if possible (there are cases where it would be useful). More than one invoice is currently not necessary but it could be useful in the future.

Acceptance criteria:

• New API implementation (except for patch actions)

• All finance, order, invoice and cross-modules integration tests pass

Implementation notes:

• The new implementation has 3 parts, in order:

  • Loading all data needed for calculations (using a holder class)

  • Calculations of everything that needs to be created or updated

  • Saving all data that needs to be saved

• As a consequence, most of the implementation (all the calculations) do not need to use futures. Also, performance is much improved.

• Calculations are done in order: creations, updates, patches, deletions. Encumbrances are processed last for each step. This order makes it possible to add encumbrances to update while creating pending payments, so the code to update encumbrances does not need to be duplicated in the pending payment service and the encumbrance service.

• There are validation checks along the way: before loading, after loading, and after calculations.

• Some validations differ from the previous implementation. In particular:

  • All the budgets related to the transactions are checked to be active.

  • Expenditure and encumbrance restrictions are checked systematically after calculations, and they take into account all of the changes in the batch, not just a single transaction at a time.

• The functionality provided by the cancel package has been moved inside the related batch transaction services at a low level: when a transaction is processed, there is a check to see if it was cancelled and the calculations take that into account. This change removes the code which was doing the same thing within the cancel package as elsewhere (like grouping by budget) and clarifies all changes that can happen for a given transaction type (it is easier to see if the cancel action does the opposite as the initial action).

• The functionality of the restriction package and other validation steps has been moved to BatchTransactionChecks. This makes sense because many checks are similar or identical between different transaction types. It is also convenient to have all the validations in the same place.

• Budgets are locked (with FOR UPDATE) when they are loaded. A single connection is used for all operations, and budgets are released at the end.

• The integration tests for parallel processing (initially for MODFISTO-260) show that this ticket resolves some previous issues with parallel processing.

• https://folio-org.atlassian.net/browse/MODFISTO-460 will take care of removing the old code and remove redundancies. For now the old API is still used by the encumbrance script (see https://folio-org.atlassian.net/browse/MODFISTO-462).

• In the old API there was a check in mod-finance when deleting encumbrances, to make sure they are not connected to an invoice (through pending payments). This check was integrated into mod-finance-storage with the new API, but they are just logging a warning because returning an error would break the encumbrance script. It was moved to mod-finance to return an error from there (the encumbrance script uses mod-finance-storage directly). See https://folio-org.atlassian.net/browse/MODORDERS-1043 for an example of a case where this error can be returned.