Fees/Fines (UXPROD-792)

[UXPROD-2495] Future Fees/Fines: Add staff member id to the fees/fines 'action' record for statistics reporting Created: 16/Jun/20  Updated: 12/Jan/23  Resolved: 12/Jan/23

Status: Closed
Project: UX Product
Components: Fees/Fines
Affects versions: None
Fix versions: None
Parent: Fees/Fines

Type: New Feature Priority: TBD
Reporter: Holly Mistlebauer Assignee: Unassigned
Resolution: Won't Do Votes: 3
Labels: Unassigned-from-Holly, feesfines, resourceaccess
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: JPEG File Add-staff-id-to-ff-action-record.jpg     PNG File Screen Shot 2020-06-16 at 7.43.31 AM.png     PNG File screenshot-1.png    
Epic Link: Fees/Fines
Front-End Confidence factor: Medium
Back End Estimate: Small < 3 days
Back End Estimator: Marc Johnson
Estimation Notes and Assumptions: Estimate only covers adding the change metadata. Old records will not include this information.
Development Team: Vega
Kiwi Planning Points (DO NOT CHANGE): 1
PO Rank: 0
Rank: Chicago (MVP Sum 2020): R4
Rank: Cornell (Full Sum 2021): R2
Rank: Duke (Full Sum 2021): R2
Rank: 5Colleges (Full Jul 2021): R2
Rank: GBV (MVP Sum 2020): R4
Rank: Grand Valley (Full Sum 2021): R1
Rank: MO State (MVP June 2020): R1
Rank: SHL (Dec 2021): R1
Rank: TAMU (MVP Jan 2021): R2
Rank: U of AL (MVP Oct 2020): R5

 Description   

Overview
The fee/fine 'action' record only includes the staff member name, not the staff member id. The name can change over time. For statistical purposes, the Chinese SIG has asked that staff member id be saved to the fee/fine 'action' record for consistency purposes. (Side note from Holly: I thought that the staff member id was being saved and that it would be translated to the name for display on Fee/Fine Details.)

Fee/fine 'action' records are written in many places, such as when...

  • manual fee/fine is charged
  • manual fee/fine is charged & paid now
  • fee/fine is paid
  • fee/fine is waived
  • fee/fine is cancelled
  • fee/fine is manually transferred
  • overdue fine is automatically charged for item returned late
  • 'Additional information for staff' is created on Fee/Fine Details page
  • lost item fees are automatically charged for a declared lost item
  • lost item fees are automatically refunded when declared lost item is returned
  • lost item fees are automatically refunded when declared lost item is renewed
  • fee/fine is manually refunded (currently in development--too late to change story)
  • lost item fees are suspended/refunded when declared lost item is claimed returned (currently in development--too late to change story)
  • lost item fees are automatically charged for an aged to lost item (aged to lost currently in development--too late to change story)
  • lost item fees are automatically refunded when aged to lost item is returned (aged to lost currently in development--too late to change story)
  • lost item fees are automatically refunded when aged to lost item is returned (aged to lost currently in development--too late to change story)
  • lost item fees are automatically suspended/refunded when aged to lost item is claimed returned (not in development yet--could add this requirement to user story)
  • fee/fine is automatically transferred (not in development yet--could add this requirement to user story)
    (there might be more that I haven't thought of...)

Attachments
Slack correspondence with Lucy Liu has been attached...



 Comments   
Comment by Holly Mistlebauer [ 21/Sep/20 ]

Marc Johnson: Hi Marc! Cate asked me to contact you to request a backend estimate for this feature. We want to update the fee/fine 'action' record to include staff member Username. Right now we save the actual name of the staff member. Given names can change, we need to add the Username to the record for reporting purposes.
Thanks, Holly
cc: Cate Boerema

Comment by Marc Johnson [ 22/Sep/20 ]

Holly Mistlebauer

Right now we save the actual name of the staff member. Given names can change, we need to add the Username to the record for reporting purposes.

Have the GDPR (or data privacy) folks been involved in these conversations? I believe in other places, there are some discussions about not storing personal information about users across the system.

Comment by Holly Mistlebauer [ 29/Sep/20 ]

Marc Johnson: I believe we are storing the name of the person who did the transaction elsewhere. Cate Boerema, do you know if this is true? Also, do the same GDPR rules apply to library staff doing their job?
P.S. We are storing the username at checkin, checkout and renewal.

Comment by Holly Mistlebauer [ 29/Sep/20 ]

Emma Boettcher: I noticed that the loan 'action' record has a 'Source' field but not a 'Created at' field like the fee/fine 'action' record does. When I checked in a book in Goldenrod, the 'Source' was set to 'folio, admin' (my login full name) but there is no mention of what desk the item was checked in at. I have two related questions:

  1. Did the RA SIG indicate that they don't care to know which desk the book was checked in or out at?
  2. Was the RA SIG concerned about recording the name of the person who did the transaction? Don't GDPR rules apply? Do you ever scrub the 'Source'?
  3. Do you store the username in the Loan 'action' record and convert it to the user's full name on the fly? Or is the user's full name stored?
  4. If you store the user's full name, has SHL approached you about storing the username instead/also?
    Thanks for your help with this...
    Best,
    Holly
Comment by Emma Boettcher [ 30/Sep/20 ]
    • Check out: it hasn't come up, interestingly. Probably worth mentioning, thanks for bringing it up.
    • Check in: the check in service point is recorded on the loan
  1. It's come up - not so much GDPR but labor agreements. However, recording the user responsible for an action is baked in to a lot of different interactions in FOLIO, including the record last updated toggle that appears on all kinds of records:

    Patty and I spoke about this at the circ PO meeting a while ago and UXPROD-2467 Open was created as a result.
  2. No, I believe just the user ID is stored
  3. No one's come to me about that - who or what is SHL?
Comment by twliu [ 01/Oct/20 ]

Emma BoettcherSHL is Shanghai Library.
Maybe some libraries use full name as user ID. But Shanghai Library (SHL) is using a combination of letters, numbers, underscore, dash, and Chinese characters (in very special cases) as user ID.
User ID won't change once it's created. So SHL wants to use it to get consistent and reliable statistics.
Thank you all for your attention to this!

Comment by Holly Mistlebauer [ 02/Oct/20 ]

Emma Boettcher: If the Loan actions already have the user id stored you are all set.

Comment by Holly Mistlebauer [ 02/Oct/20 ]

Cate Boerema: How should we proceed with this feature? twliu just asked me via Slack if this will be part of Q3 2020.

Comment by Cate Boerema (Inactive) [ 05/Oct/20 ]

twliu just asked me via Slack if this will be part of Q3 2020.

Holly Mistlebauer, we are already doing the core module backend releases for Q3 2020. It is too late to get this feature into the release. If you want to move this forward, you will need to (1) get estimates on this UXPROD and (2) get the stories written (3) give it a high PO ranking.

Comment by Holly Mistlebauer [ 05/Oct/20 ]

Cate Boerema: When I asked Marc to estimate this, he raised concerns about GDPR (data privacy) issues. We already store the staff member name for fees/fines, this would just be a switch to the staff member id. We store the staff member id with almost everything that is done, so I am not sure why this is a concern. What do you think?

Our community as a whole does not view this as a high priority. You had mentioned wanting to do this for SHL. It will require at least 18 back-end user stories to save the staff member's id instead of their name and some front-end stories to use the id to obtain the staff member's name for display. I can't justify a high PO Rank for this feature given that it will mean something else of importance won't get done.

Comment by twliu [ 05/Oct/20 ]

Holly Mistlebauer Of all the round IV implementers, three voted R1 and three - R2. I think this indicated that the community viewed this issue as a high priority.
Shanghai reported this as early as June. In our Slack communications, you mentioned that it could be included in the Q3 release. It will be really disappointing to SHL if this can't be done before the trial run starts this coming December.

Comment by Cate Boerema (Inactive) [ 06/Oct/20 ]

When I asked Marc to estimate this, he raised concerns about GDPR (data privacy) issues. We already store the staff member name for fees/fines, this would just be a switch to the staff member id. We store the staff member id with almost everything that is done, so I am not sure why this is a concern. What do you think?

Holly Mistlebauer, I don't see why this would be a GDPR issue (or any more of GDPR issue than we already have)

It will require at least 18 back-end user stories to save the staff member's id instead of their name and some front-end stories to use the id to obtain the staff member's name for display

Yikes - is that really true? I'd like to see the estimates on this and get Marc Johnsons take on the number of stories needed. I would have thought you could just add a staff ID property to the fee/fine actions table.

I can't justify a high PO Rank for this feature given that it will mean something else of importance won't get done.

That's fair - but we need estimates before we know what kind of tradeoff we are talking about.

Comment by Marc Johnson [ 06/Oct/20 ]

Holly Mistlebauer Cate Boerema twliu

The fee/fine 'action' record only includes the staff member name, not the staff member id.

We want to update the fee/fine 'action' record to include staff member Username

Maybe some libraries use full name as user ID. But Shanghai Library (SHL) is using a combination of letters, numbers, underscore, dash, and Chinese characters (in very special cases) as user ID. User ID won't change once it's created.

I think I'm a little confused by what we are intending to store for this feature. The title of feature refers to an id yet the comments above refer to username. I think what twliu is referring to is the username not the ID, as the latter is generated by the system and should not include the characters being referred to.

From what I can tell from the current code, the system stores the ID of the user. Whilst this uses an unconventional structure, this approach is consistent with other parts of the system. It does not appear that we store the name.

Are we proposing to store the ID and username but not the name?

Our community as a whole does not view this as a high priority. You had mentioned wanting to do this for SHL.

Of all the round IV implementers, three voted R1 and three - R2. I think this indicated that the community viewed this issue as a high priority. Shanghai reported this as early as June. In our Slack communications, you mentioned that it could be included in the Q3 release. It will be really disappointing to SHL if this can't be done before the trial run starts this coming December.

What decision are we trying to make, whether to include this feature in development targeted for 2021 R1?

It will require at least 18 back-end user stories to save the staff member's id instead of their name and some front-end stories to use the id to obtain the staff member's name for display. I can't justify a high PO Rank for this feature given that it will mean something else of importance won't get done.

Yikes - is that really true? I'd like to see the estimates on this and get Marc Johnsons take on the number of stories needed. I would have thought you could just add a staff ID property to the fee/fine actions table.

Where does this prediction of 18 stories come from?

I haven't done significant analysis into this feature and I'm not sure I properly understand it. If we intend for each client to provide these details when performing an action, then it would require every client to change, which might result in that kind of scope. This approach could also allow for incorrect, inconsistent or disingenuous information to be provided and might affect the refactoring of the APIs that Vega have been recently doing.

When I asked Marc to estimate this, he raised concerns about GDPR (data privacy) issues. We already store the staff member name for fees/fines, this would just be a switch to the staff member id. We store the staff member id with almost everything that is done, so I am not sure why this is a concern. What do you think?

I don't see why this would be a GDPR issue (or any more of GDPR issue than we already have)

I asked the GDPR question because I had recently encountered features that referred to the possibility of needing to remove personal details from records or to disable storing the ID / username of the user that created / changed a record. I'm ok with excluding such concerns from the scope of this feature (the design choice made for this work, might affect implementation of these features in the future).

Comment by twliu [ 09/Oct/20 ]

Marc JohnsonThank you for asking. Is there a UUID for library staff? If so, the SHL wants to add this UUID to the database. I just got the clarification from Shanghai that It's ok if it doesn't display on the frontend.

Comment by Marc Johnson [ 09/Oct/20 ]

twliu

Is there a UUID for library staff?

I'm not sure I understand your question. I'll try to reflect my understanding and you can help me understand if that is useful or answers your question.

My understanding is that library staff are represented as users. Like most records in FOLIO, these have an id property which should be a UUID.

From what I can tell from the current code, mod-feesfines stores the ID of the user that performed the action (though this information is provided by the client and so could be different to the user referenced by the token for the logged in user).

If the requirement is that the user who performed the action (assuming we trust the client) is recorded, then I believe the current behaviour already meets that need.

Holly Mistlebauer Cate Boerema Zak Burke Bohdan Suprun Please do correct me if my understanding is incorrect.

bq If so, the SHL wants to add this UUID to the database. I just got the clarification from Shanghai that It's ok if it doesn't display on the frontend.

Which database are you referring to?

Comment by Zak Burke [ 09/Oct/20 ]

Marc Johnson, my understanding of Holly Mistlebauer's initial request is that SHL wants to store a persistent user identifier with fee-fine actions. I would expect that to be the UUID of the user who performs the action. As others have noted, given that we already display the name of that user, I don't understand why any new issues related to GDPR would come into play.

My understanding of the problem is that shape of the data returned by /feefineactions is insufficient because its source property only contains a name, not a UUID. For example:

{
  "feefineactions" : [ {
    "dateAction" : "2020-10-09T16:48:48.774+0000",
    "typeAction" : "Misc fee/fine",
    "comments" : "",
    "notify" : false,
    "amountAction" : 12.34,
    "balance" : 12.34,
    "transactionInformation" : "-",
    "createdAt" : "c4c90014-c8c9-4ade-8f24-b5e313319f4b",
    "source" : "ADMINISTRATOR, DIKU",
    "accountId" : "7d9efacf-a59f-4263-a5b6-2e3c9127ac5f",
    "userId" : "6f36265e-722a-490a-b436-806e63af2ea7",
    "id" : "567fed9e-29c4-465f-a43c-a4bacb3e7ad4"
  } ],
  "totalRecords" : 1
}

Note that source is simply a string and that no persistent identifier for "ADMINISTRATOR, DIKU", is attached to this record. If DIKU were "Smith, Kate" and she changes her name to "Witters, Kate" or "Smith, Karl", or "Smith, Katherine" even just "Smith, Cate" because somebody typed it in wrong the first time, it would prevent the correct collection of statistical data where a single user (e.g. with UUID 1234) is the "source", regardless of whether "Cate" or "Kate" (or ...) is the name attached to UUID 1234.

Comment by Gang Zhou [ 10/Oct/20 ]

Marc Johnson twliu Yes,it is the scenario that Zak Burke described.

So far the “feefineactions” have only "source" as staff name, but the staff name may be modified and the staff UUID is exact and permanent.
We want to trace the payments of specific staff by staff UUID , then we hope the staff UUID will be contained in "feefineactions".

Comment by Zak Burke [ 10/Oct/20 ]

Looking at this again, it strikes me that the shape of the response has additional problems:

  1. there is no currency field to indicate the units of the transaction
  2. dateAction would be better expressed as metadata.createdDate to be consistent with other services
  3. createdAt contains the UUID for a service point and therefore should be renamed to servicePointId both to be clear about what it represents and to remove ambiguity with other created... fields that refer to timestamps
  4. typeAction, like source, contains the string value of a related field instead of its UUID (a "fee/fine", not, in fact a "fee/fine action" which is the name of this endpoint) and therefore will cause problems with statistics and i18n; i.e., this field should have both a different name and a different value

I'm not really sure what to do with this information but it makes me think we should take a high-level look at these endpoints to make sure any future development is consistent with existing project conventions with regard to naming conventions and data storage, and that we it anticipates the kinds of needs expressed here, i.e. for relational data to be stored by UUID or by copying over the entire related object as an object (e.g. source should contain an object representing a user record, or should be named sourceUserId and contain the UUID of a user record).

Comment by Marc Johnson [ 12/Oct/20 ]

Zak Burke Thank you for sense checking my thoughts.

my understanding of Holly Mistlebauer's initial request is that SHL wants to store a persistent user identifier with fee-fine actions. I would expect that to be the UUID of the user who performs the action.

That was what I thought, however the first comment on the issue suggests otherwise.

I'm hoping that Holly Mistlebauer and twliu can confirm whether it is the username or the id of the user that is intended to be stored.

My understanding of the problem is that shape of the data returned by /feefineactions is insufficient because its source property only contains a name, not a UUID

That is entirely up to the client. Any client could put the UUID of the user in that existing property if it wanted to.

I think I may have got a bit distracted and confused by the userId property. Is that the user who performed the action or the user who is responsible for payment of the account the action relates to?

As others have noted, given that we already display the name of that user, I don't understand why any new issues related to GDPR would come into play.

I regret raising the GDPR question (I've cited why I raised it) and have already tried to suggest we disregard it from the scope of this work :-/

it strikes me that the shape of the response has additional problems

I agree. I don't know how much of this is within the scope of Vega's design work that they have been doing. Holly Mistlebauer Darcy Branchini do you know if restructuring the fee fine APIs in general is part of the scope, or was it only adding the new APIs for specific processes?

I'm not really sure what to do with this information but it makes me think we should take a high-level look at these endpoints to make sure any future development is consistent with existing project conventions with regard to naming conventions and data storage

I agree.

Within the scope of this issue, I'm trying to figure out if the requirements would be met by introducing the standard change metadata. If folks want the ID of the user that created the action, then that might be sufficient. And source can be removed (when it's convenient to make that breaking change).

(I'm trying not to get too deep into expressing my frustrating and confusion whenever I try to understand this particular area of FOLIO, as the API does not seem to reflect the domain language folks use)

Comment by Marc Johnson [ 12/Oct/20 ]

Having done a little bit of further investigation into this and I'm even more confused. I have at least confirmed that my hopes about the userId were entirely wrong.

I just tried making a payment an account via the UI.

That generated the following request

POST https://folio-snapshot-okapi.dev.folio.org/accounts/ad2cf071-cc57-4b3a-b2d0-fb3ab50f5f49/pay
{
  "amount":"30",
  "paymentMethod":"Cash",
  "notifyPatron":false,
  "comments":"",
  "servicePointId":"c4c90014-c8c9-4ade-8f24-b5e313319f4b",
  "userName":"ADMINISTRATOR, DIKU",
  "transactionInfo":"-"
}

That uses a property called userName for the formatted name of the user. It does not refer to a user at all. (It also refers to payment methods directly by their name)

The following action was created:

{
    "dateAction" : "2020-10-12T11:58:54.305+0000",
    "typeAction" : "Paid partially",
    "comments" : "",
    "notify" : false,
    "amountAction" : 30.0,
    "balance" : 70.0,
    "transactionInformation" : "-",
    "createdAt" : "c4c90014-c8c9-4ade-8f24-b5e313319f4b",
    "source" : "ADMINISTRATOR, DIKU",
    "paymentMethod" : "Cash",
    "accountId" : "ad2cf071-cc57-4b3a-b2d0-fb3ab50f5f49",
    "userId" : "86c9f455-a685-45d0-9d01-5943a1ba7e5b",
    "id" : "23005755-6495-4f46-8807-8a4020ca6aa4"
  }

This suggests that the userName of the payment maps to the source (this discrepancy in naming might increase the confusion in this area) and it appears the userId is derived from the account (as it is the user the original fee / fine was issued to) and isn't included in the payment action itself.

Alas, this likely means that there could well be an impact to the newly added APIs depending upon how we decide to address this.

Comment by twliu [ 14/Oct/20 ]

Marc JohnsonZak BurkeHolly MistlebauerCate BoeremaSo what's the conclusion here? How are we going to solve SHL's problem?

Comment by Marc Johnson [ 14/Oct/20 ]

twliu

So what's the conclusion here? How are we going to solve SHL's problem?

Can you confirm that it is the user ID that SHL needs storing?

Would it be sufficient to capture the regular change metadata (the metadata property on other records) that has the user ID of the user which created the record and which last updated it?

It should be noted that I believe some of these actions are created by the system in background tasks (Bohdan Suprun please confirm that mod-circulation creates actions in timer endpoints) and so there is no user associated with those.

Comment by twliu [ 14/Oct/20 ]

Marc JohnsonWhat other records do you mean? Would you mind clarifying it? Thank you~

Comment by Marc Johnson [ 14/Oct/20 ]

twliu

What other records do you mean?

FOLIO has a fairly common pattern for recording the user that created or updated a record. By other records I was referring to most of the other kinds of records e.g. items, instances, loans, requests etc.

The record types in mod-feesfines are somewhat unusually structured. I was trying to understand if the goal of this is to record the ID of the user that created the action then it is likely that we can use a standard approach to do that.

Comment by Gang Zhou [ 15/Oct/20 ]

Marc Johnson Marc, didi you mean the fields createdByUserId and updatedByUserId?

Comment by Marc Johnson [ 15/Oct/20 ]

twliu

did you mean the fields createdByUserId and updatedByUserId?

Yes, I mean the standard metadata object which contains those properties. I think Zak Burke suggested it above.

"metadata" : {
      "createdDate" : "2020-10-15T05:55:06.025+0000",
      "createdByUserId" : "18b824f0-4946-5e3f-b95d-15e010d991ad",
      "updatedDate" : "2020-10-15T05:55:06.025+0000",
      "updatedByUserId" : "18b824f0-4946-5e3f-b95d-15e010d991ad"
    }
Comment by twliu [ 20/Oct/20 ]

Marc Johnson, thank you for clarifying this! It is an acceptable solution.

Holly Mistlebauer SHL just confirmed that this ticket could be closed.

Marc JohnsonZak BurkeCate BoeremaHolly MistlebauerThank you all for your attention to this!

Comment by Marc Johnson [ 20/Oct/20 ]

twliu

SHL just confirmed that this ticket could be closed.

Are you referring to this issue ( UXPROD-2495 Closed )? If so, does that mean this is no longer needed?

Comment by twliu [ 20/Oct/20 ]

Marc JohnsonAre you referring to this issue ( UXPROD-2495 Closed )? If so, does that mean this is no longer needed?

I mean SHL confirmed they could use createdByUserId and updatedByUserId to get the stats they need.

Generated at Fri Feb 09 00:24:27 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.