[FOLIO-556] align PG and mattype endpoints by using UUIDs Created: 12/Apr/17  Updated: 12/Nov/18  Resolved: 27/Apr/17

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P3
Reporter: Jakub Skoczen Assignee: shale99
Resolution: Done Votes: 0
Labels: sprint12, sprint13
Remaining Estimate: Not Specified
Time Spent: 1 hour, 30 minutes
Original estimate: Not Specified

Issue links:
Blocks
blocks STRIPES-287 "item types" should be "material type... Closed
blocks UIU-32 patron group filter should use the va... Closed
Cloners
is cloned by METADATA-57 Use UUID for Refering to Material Types Closed
Relates
relates to CIRC-3 Depend upon item-storage v2.0 interface Closed
relates to METADATA-58 Use Material Types Referred to by UUI... Closed
relates to FOLIO-535 agree on the approach for joining/fil... Open
Sprint:

 Description   

As decided in FOLIO-535 Open , any kind of "controlled vocabulary" endpoint should follow a similar scheme.

CRUDing

GET|POST /cv-items/
GET|PUT|DELETE /cv-items/:uuid

Associations

In the "parent" record (like User or Item) we would include raw UUID/s directly in the associated record, using a scalar UUID field for many-to-one relations (e.g MT) or an array of UUIDs for the many-to-many relations (e.g PG):

{
  "username": "shale99",
  "patron_groups": ["123-456-789-123", "345-678-910-111"]
}

The PUT/POST on the record with proper UUID value should allow for updating associations. To get all users/items/etc with a given PG/MT/etc association we will use sarching/filtering.

Searching with CQL

Search/filter for items with a CV-item should be supported directly on the resource:

lastName="Doe" AND (patron_group == "123-456-789" OR patron_group == "112-113-114")

when constructing facet-like clickable filters on the left-hand side the UI will lookup all CV values (the assumption is that the list is short) and create name-to-uuid map for handling filtering.

The above sheme should be used for all newly constructed CV-like endpoints.



 Comments   
Comment by Mike Taylor [ 12/Apr/17 ]

I see from the example above that username survives as a sort of auxiliary ID. Good.

Returning list-valued fields like ["123-456-789-123", "345-678-910-111"] – is that something RMB-based modules like mod-users can do, or will it require the involvement of a BL module?

(Lastly, a precision on the CQL: I avoid the any relation since it requires parsing the term itself, to break it on spaces. So it's better to use patron_group=("123-456-789" or "112-113-114").)

Comment by Jakub Skoczen [ 12/Apr/17 ]

Mike Taylor yes, I didn't get into the details of using "username" as an identifier across modules, I am aware there is an issue but this doesn't aim to solve it.

In terms of returning list and scalar fields with UUIDs directly in the object – yes the storage module should be able to do it since so far we use the CVs intra module only. I don't think we need to involve the BL module.

Comment by Mike Taylor [ 12/Apr/17 ]

OK: so mod-users will do the join on users-to-PGs association table to get the list of PG IDs for each user; but it won't go on to get the corresponding PG names. Is that right? It seems ... clumsy. Would it be hard for mod-users to also consult the PG table that (I assume) it is response for, and return to the client a list of PG objects instead of PG IDs?

Comment by Jakub Skoczen [ 12/Apr/17 ]

It would be easy to perform the join, yes. But I don't think we should "just do it".

Searching for actual names is subject to tokenization and language rules, UUID based matching is much simpler and more precise so I would still advocate the UI/client used that (and so it needs to lookup the map).

In terms of performing the join to include more data in the parent object - the objects behind UUIDs might be more complex than just a single value – multi-language labels, descriptions, notes, etc. I think we should have a bit more control about what gets joined in, but I wouldn't like that discussion to block the simple case.

Comment by Mike Taylor [ 12/Apr/17 ]

Ah, cross-purposes, sorry. I wasn't suggesting we do the join for searching (i.e. search on user by patron-group name), only for retrieval (user records contain a list of patron-group objects).

Comment by Wayne Schneider [ 17/Apr/17 ]

It looks like shale99 has made this change in the inventory-storage module of mod-metadata (see https://folio-org.atlassian.net/browse/STRIPES-287?focusedCommentId=148729&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel). Will this also affect mod-circulation and the inventory module of mod-metadata?

Comment by shale99 [ 18/Apr/17 ]

the two apis have been aligned

uuid based crud api for both MT and PG
adding / updating / removing an MT / PG occurs directly in the item / user objects (jsons) respectively by placing the desired uuid in the material type / patron group fields

please let me know of any issues / comments

thanks

Comment by Mike Taylor [ 18/Apr/17 ]

As noted in STRIPES-187 Closed , for material types we are waiting for a storage-module API change to be propagated into the corresponding BL module and for the resulting new API to be made available via a new folio-auth-backend VM. Do we need similar steps regarding patron-groups, too?

AFAIK, both MTs and PGs are provided by mod-users, and so may well both end up being mediated by bl-users – is that correct? If so, then UIU-32 Closed will also need to wait for that process to complete.

Comment by shale99 [ 19/Apr/17 ]

There are two modules affected by this change Kurt Nordstrom / Marc Johnson please correct me if i am wrong.
1. inventory
2. mod-users-bl

will start with (2) - right now the the expectation is that when a user is saved, if that user belongs to a patron_group - the patron_group field should be populated with a valid uuid (this is validated on the server side prior to updating / saving the user). does this negate the need for querying the /groups api from within the mod-users-bl?

(1) marc, i havent really coded in groovy before, but i am only seeing problems with the material types in the tests? would that be correct? (ItemCollectionExamples.groovy)

Comment by Marc Johnson [ 19/Apr/17 ]

I am less familiar with patron groups, so I will focus on the circulation / inventory aspects of this change. I don't believe material types are used by the users business logic module.

The impact on the inventory module depends upon whether the change to refer to a external material type record by UUID is also made in an item at this level. shale99 is it your understanding that this change would lead to the item in the inventory business logic module going from something like:

"materialType": {
      "type": "object",
      "properties": {
        "name": {
          "type": "string"
        }
      },

to something like:

    "materialTypeId": {
      "type": "string"
    },

Or would the business logic module do some of the mapping, e.g.

"materialType": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string"
        },
        "name": {
          "type": "string"
        }
      },

Hugs,

Marc

Comment by shale99 [ 19/Apr/17 ]

so i think that question, at least for me, ties into the i18n convo.
are the uuids keys to translations - for example 1234-5678-999 = Book (en) or sefer (he)? so that the UI can process this via an i18n bundle - or is the server responsible for providing a key + lang in -> label out API.
i really prefer the UI working with a bundle honestly.
if the UI wants more info about an MT - whatever that may be in the future (for editing , etc...) then it can use the MT API to retrieve the full record.
but that is my take on it, not sure what was finalized with this

Comment by Mike Taylor [ 19/Apr/17 ]

I don't believe material types are used by the users business logic module.

Yep, my dumb mistake. Ignore that earlier assertion

Comment by Mike Taylor [ 19/Apr/17 ]

More generally, I see there is going to be some way to go before this API is fully designed, let alone implemented an included in a folio-backend-auth VM. So for now I am going to leave the relevant UI-side issues ( UIU-32 Closed and STRIPES-287 Closed ) blocked, and work on other things. (Sorry, Cate Boerema – I know you're keen to see this stuff working in demos, but that's how it has to be. Can't build on a foundation that doesn't exist yet.)

Comment by shale99 [ 19/Apr/17 ]

just reading over FOLIO-535 Open and it seems that the requirement is to do a language specific join by the BL module and return that. Can someone confirm i am not hallucinating?
if that is the case - then it would seem that the BL modules will run a join between the MT / PG UUIDs within the Items / Users and be returning language specific values.
so Marc Johnson if this is indeed the case - it would seem like the original schema stays?


"materialType": {
      "type": "object",
      "properties": {
        "name": {
          "type": "string"
        }
      },
Comment by Marc Johnson [ 19/Apr/17 ]

Thanks shale99 I took a look at that, based predominantly on this comment I think I shall augment the inventory business logic module item schema to look like:

"materialType": {
      "type": "object",
      "properties": {
        "id": {
          "type": "string"
        },
        "name": {
          "type": "string"
        }
      },

In order to preserve as much compatibility as possible and offer a basic textual representation for display purposes, whilst also adding in the explicit id. This will still effectively be incompatible as the client will need to provide the ID of the material type when creating or updating items.

I am not going to do anything with reference to the special CQL indexes at this point, so the filtering implementation will need to change, particularly because I believe to do this the business logic modules would need to interpret and translate the CQL before passing it to the storage module.

Hugs

Comment by shale99 [ 20/Apr/17 ]

guys,
should we close this issue and open 2 separate issues
1. migrating inventory - to use latest MT endpoint
2. migrating mod-users-bl - to use latest PG endpoint

Comment by Mike Taylor [ 20/Apr/17 ]

That would certainly suit me, as then I could link my dependent issues in a more precise way.

Comment by Cate Boerema (Inactive) [ 25/Apr/17 ]

Assigning this to sprint 13, as it's a blocker to UIU-32 Closed which is in sprint 13.

Comment by Mike Taylor [ 26/Apr/17 ]

Marc Johnson, now that you have closed METADATA-57 Closed and METADATA-58 Closed , what is the status of this one?

Comment by Marc Johnson [ 26/Apr/17 ]

Mike Taylor

Given this issue was created before we started moving issues for each module (or intermediary group of projects) into their own JIRA projects, I look at this as either represents the group of changes (in which case it should only be closed when all of the work has been done across the five modules affected) or it represents the decision to make the change (and hence should be closed if issues have been created for all of the changes). Maybe Adam Dickmeiss can advise on what he thinks is the most appropriate for this issue, following the discussion we had yesterday.

As for the patron groups changes, maybe shale99 or Kurt Nordstrom can advise on which issues cover these (I had a quick look but wasn't sure).

I closed the issues as I released these changes yesterday (in that they are tagged in github) in order for them to be prepared for inclusion in the black boxes (I imagine Wayne Schneider will likely raise a new issue for updating them). I believe they are sufficient in order to update the UI to use the UUID way of referring to material types. Given the change in compatibility, we may want to manage how people migrate to the new blackbox until the UI is updated too.

Hugs

Comment by Mike Taylor [ 26/Apr/17 ]

Thanks, Marc.

No need to change the VM upgrade process at the moment, as material-types and patron-groups are presently pretty much just broken. So they will just get better, not worse.

Comment by Cate Boerema (Inactive) [ 27/Apr/17 ]

Per folio meeting, this is done.

Generated at Thu Feb 08 23:06:40 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.