Folio issues with Data consistency

Owned by Vasily Gancharov
Last updated: Aug 26, 2020
2 min read

This page contains examples of issues with Data consistency in Folio. The issues could fall into one of the following categories:

  • There is no logic to make sure a data is consistent across a set of modules
  • There is a logic to perform data consistency, but it isn't reliable and can lead to inconsistent data in a set of modules

Cases discovered earlier

Spike: Distributed transaction use cases and solutions

Order and Invoice module cases

Approve invoice (mod-invoice)

  1. Fetch invoice (mod-invoice → mod-invoice-storage)

  2. Process prorated adjustment (mod-invoice → mod-invoice-storage)

2.1. Fetch invoice lines by invoice id (mod-invoice → mod-invoice-storage)
2.2. Persist invoice lines (mod-invoice → mod-invoice-storage)

3. Update invoice lines with encumbrance transactions (mod-finance → mod-finance-storage)

4. Check enough money in budget (mod-finance → mod-finance-storage)

5. Persist invoice lines (mod-invoice → mod-invoice-storage)

6. Check expense classes (mod-finance → mod-finance-storage)

7. Create/persist pending payment transaction (mod-finance → mod-finance-storage)

8. Create/persist voucher with voucher lines  (mod-invoice → mod-invoice-storage)

8.1. Fetch voucher lines by invoice id (mod-invoice → mod-invoice-storage)
8.2. Fetch invoice lines by invoice id (mod-invoice → mod-invoice-storage)
8.3. Persist voucher (mod-invoice → mod-invoice-storage)
8.4. Persist voucher lines (mod-invoice → mod-invoice-storage)

       9. Persist invoice with new status (mod-invoice → mod-invoice-storage)

Pay invoice (mod-invoice)

  1. Fetch invoice (mod-invoice → mod-invoice-storage)

  2. Process prorated adjustment (mod-invoice → mod-invoice-storage)

2.1. Fetch invoice lines by invoice id (mod-invoice → mod-invoice-storage)
2.2. Persist invoice lines (mod-invoice → mod-invoice-storage)

3. Handle payment and credits transactions (mod-finance → mod-finance-storage)

4. Persist invoice with new status (mod-invoice → mod-invoice-storage)

5. Update voucher (mod-invoice)

5.1. Fetch voucher lines by invoice id (mod-invoice → mod-invoice-storage)
5.2. Fetch invoice lines by invoice id (mod-invoice → mod-invoice-storage)
5.3. Persist voucher (mod-invoice → mod-invoice-storage)
5.4. Persist voucher lines (mod-invoice → mod-invoice-storage)

Open order : transition from pending to open status (mod-orders)

  1. Fetch order (mod-orders → mod-orders-storage)

  2. Fetch order lines (mod-orders → mod-orders-storage)

  3. Check locations and pieces consistency (mod-inventory → mod-inventory-storage)

  4. Handle purchase order lines (POL)3.1. Delete existing POL (mod-orders → mod-orders-storage)

4.2. Persist changes in POL (mod-orders → mod-orders-storage)
4.3. Release POL encumbrance transactions (mod-finance → mod-finance-storage)

5. Persist order with new status (mod-orders → mod-orders-storage)

Unopen order : transition from open to pending status (mod-orders)

  1. Fetch order (mod-orders → mod-orders-storage)

  2. Fetch POL encumbrance transactions (mod-finance → mod-finance-storage)

  3. Make/Persist encumbrances pending (mod-finance → mod-finance-storage)

  4. Update/Persist order transaction summary  (mod-finance → mod-finance-storage)

  5. Update/Persist transactions  (mod-finance → mod-finance-storage)

  6. Make/Persist POL pending (mod-orders → mod-orders-storage)

  7. Persist order with new status (mod-orders → mod-orders-storage)

Data redundancy in models

  1. Pair of RefNumber and RefType should be in consistence state between POL and invoice line.
  2. FundCode should be in consistence state between Fund record and pol.fundDistribution.code.
  3. VendorCode should be in consistence state between Organization record and purchaseOrder.vendorCode.