SPIKE: Closing declared lost loan (Lost and paid status) for set cost

Description

Purpose: Once the patron has paid for the item (or the associated fines have otherwise been taken care of - for example, if the library waives them), the loan can be closed.

The lost item fee policy will specify that either the actual cost of the item is charged or a set cost for a lost item is charged (in addition to a lost item processing fee). This story covers scenarios where the set cost is charged (automated).

Spike output:

  • Design document

Scenarios

  1. Scenario

    • Given a loan that has been declared lost with the following fees assigned:

      • no lost item processing fee

      • charge amount for item is set cost > 0

    • When the charge amount for item fee has been closed

    • Then:

      • close the loan

      • change item status to Lost and paid

  2. Scenario

    • Given a loan that has been declared lost with the following fees assigned:

      • some lost item processing fee > 0

      • charge amount for item is set cost = 0.00

    • When the lost item processing fee has been closed

    • Then:

      • close the loan

      • change item status to Lost and paid

  3. Scenario

    • Given a loan that has been declared lost with the following fees assigned:

      • some lost item processing fee > 0

      • some charge amount for item > 0

    • When both the lost item processing fee and the charge amount fee have been closed

    • Then:

      • close the loan

      • change item status to Lost and paid

Notes:

  • If the Actual cost fee has not been charged to the patron yet, but is expected according to the fee/fine policy, the loan should not be closed.

  • Whether or not an Actual cost fee is owed should be determined when the item is declared lost. If some aspect determining the circulation rule/lost item fee policy changes between when the item was declared lost and when the fees were charged or paid, adhere to the lost item fee policy in place when the item was declared lost.

  • Other fees or fines may exist on the loan that are not related to its being declared lost. These should not block the loan from closing once all lost item fees are paid.

Environment

None

Potential Workaround

None

Checklist

hide

TestRail: Results

Activity

Show:

Marc Johnson May 7, 2020 at 3:22 PM

while not ideal, I think hosting providers could come up with some sort of log monitoring to detect errors in pub sub modules (or any errors in general) and send an alert to interested parties.

Assuming that an error produces a log entry.

Some logs are next to impossible to use for visually diagnosing issues, so we (EBSCO) are likely getting to this stage anyway unless we revisit logging in FOLIO (which I am an advocate for, since, even with log monitoring tools, multi-tenant logs are still a problem).

I think this leads us into the bigger topic of operational observability, which I agree is an important topic that FOLIO likely needs to invest in at some point.

We currently have a similar situation with Okapi timer tasks where they can fail silently and remain undetected. For example, the session time out timer task was failing with a server side error for the first couple of weeks in the Chalmers deployment of Edelweiss due to a property type mismatch in the JSON config.

Indeed, the Okapi timer interface is used to implement scheduled tasks (some older tasks use RAML Module Builder's internal scheduler). I agree, this other kind of asynchronous process also has challenges with making folks aware of failures and compensating for them.

I just happened to notice a persistent 5xx error in a cloudwatch chart while I was looking into another error. The system appeared to work fine and no one seemed to notice that their circ sessions were not timing out automatically.

This is a key challenge, it is difficult for folks to notice that something didn't happen. Testing for this can be challenging (I discussed it briefly on the pull request to resolve this issue, as the defaulting approach is still hard to observe)

Matt Reno May 7, 2020 at 1:21 PM

, while not ideal, I think hosting providers could come up with some sort of log monitoring to detect errors in pub sub modules (or any errors in general) and send an alert to interested parties. Some logs are next to impossible to use for visually diagnosing issues, so we (EBSCO) are likely getting to this stage anyway unless we revisit logging in FOLIO (which I am an advocate for, since, even with log monitoring tools, multi-tenant logs are still a problem).

We currently have a similar situation with Okapi timer tasks where they can fail silently and remain undetected. For example, the session time out timer task was failing with a server side error for the first couple of weeks in the Chalmers deployment of Edelweiss due to a property type mismatch in the JSON config. I just happened to notice a persistent 5xx error in a cloudwatch chart while I was looking into another error. The system appeared to work fine and no one seemed to notice that their circ sessions were not timing out automatically.

Marc Johnson May 7, 2020 at 12:55 PM

This came up in another meeting and let us know that this is a known issue with PubSub and that people are already looking into how to solve it.

That is good to know, I was not aware that it was being actively progressed at the moment.

Please could you point me in the direction of this work.

Given that, I don't think lack of failure handling should prevent us from using PubSub for this feature. Are there any other reasons we shouldn't just use PubSub? If not, I think we can move forward.

As long as we are happy with the process being asynchronous, I am not aware of any reason not to proceed with pub-sub.

BTW, will be calling a meeting on the topic of failure handling in PubSub (with a few folks such as you, , and ).

Oh, I wasn't aware of this either. Either I missed both of these, or a lot has changed since I raised the topic yesterday in the Tech Leads meeting.

Cate Boerema May 7, 2020 at 12:40 PM

Thanks . This came up in another meeting and let us know that this is a known issue with PubSub and that people are already looking into how to solve it. Given that, I don't think lack of failure handling should prevent us from using PubSub for this feature. Are there any other reasons we shouldn't just use PubSub? If not, I think we can move forward.

BTW, will be calling a meeting on the topic of failure handling in PubSub (with a few folks such as you, Craig, Vince and Taras).

+

Marc Johnson May 7, 2020 at 12:04 PM

what is your preference?

Short Answer

I think the pub-sub based asynchronous process is best aligned with how a real world process could work and with the technical direction. I think FOLIO will need to address the handling of failures (in general) at some point.

Long Answer

Longer answer: I think an asynchronous process where the circulation reacts to a fine being closed is the closest alignment to how this process might work in reality (e.g. a accounts staff member closes the fine, and later, after being informed by accounts, a circulation staff member closes the loan, as two separate tasks).

presented two asynchronous options. I think the pub-sub option more closely aligns with both how the process could work in the real world and with the technical direction FOLIO is taking.

It sounds like the primary challenge with PubSub is handling of the (rare) case when something goes wrong and there is an error closing the loan.

Yes, the two most important aspects of this decision are:

  • how are (hopefully rare) errors handled

  • loans will not be closed as soon as the fee in closed (it might be very soon after)

As I understand it, this is a general issue with PubSub and other asynchronous approaches and the lack of solution hasn't prevented other teams from moving forward with such approaches.

Yes, FOLIO has already adopted some asynchronous processes with scheduling (request expiration, loan anonymization) and with pub-sub (data import, and I think automated patron blocks will use this).

The lack of agreed solutions to handling failure hasn't stopped these moving forward. I raised this with the Technical Leads yesterday and will be writing a document describing the challenges for further discussion.

Given that, I am not inclined to let it hold us back either. I think we can probably go into this knowing we are incurring tech debt that will need to be resolved later (not just for this feature, but for the project as a whole).

Agreed. I only wanted to make sure folks were aware of these challenges whilst making the decision. These aspects will need addressing at some point. I don't know who needs to sponsor investing in that (maybe the Technical Council?).

Done

Details

Assignee

Reporter

Priority

Sprint

Development Team

Prokopovych

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs
Created April 24, 2020 at 3:14 PM
Updated May 15, 2020 at 1:21 PM
Resolved May 11, 2020 at 2:11 PM
TestRail: Cases
TestRail: Runs