isBoundWith is not published to kafka

Description

Overview: isBoundWith is not published to kafka, making it unavailable to be read and re-published by mod-search.

Details: There is a discrepancy in the data returned by mod-inventory and mod-search for the same instances; mod-search results do not/cannot include the isBoundWith attribute because mod-inventory-storage does not publish it to the kafka topic mod-search consumes.

Interested parties: ,

Environment

None

Potential Workaround

None

has to be done before

Checklist

hide

TestRail: Results

Activity

Show:

Marc JohnsonJanuary 25, 2022 at 5:04 PM
Edited

Closing as a duplicate following advice from  that the Falcon team later created a separate issue, for this work that makes this issue redundant.

Holly MistlebauerDecember 15, 2021 at 8:53 PM

: Will you set up the meeting recommended by ? It appears this will need to wait until January. Thanks...

Marc JohnsonDecember 15, 2021 at 4:57 PM
Edited

Given the uncertainty about how this work should proceed, should I move this issue into draft status?

Sure, or block it.

Kyle FelkerDecember 15, 2021 at 3:26 PM

Given the uncertainty about how this work should proceed, should I move this issue into draft status?

Marc JohnsonDecember 15, 2021 at 10:48 AM

Thank you for sharing your understanding of the situation, especially following the conversations with

I'll try to share my understanding of the decisions that have been made and how that leads to the way I think the work is intended to be done.

Relevant Decisions

Record Oriented Change Notifications

It was decided during the original search using ES work that FOLIO would adopt record oriented change notification messages (see inventory domain events section of linked document).

This has later been codified in the Kafka usage guidance.

I believe this decision was made by the Falcon team and EPAM SAs (and later supported by the TC, at least in the context of a PoC / exceptional use).

Bound-with Design

It was decided that the bound with relationship between items would be modelled in inventory using separate records.

I believe this was done in order to avoid making changes to the primary record types (instance, holdings and item) involved.

I was involved in the working group that made this decision, alongside some SMEs, and folks from the Thor team who later went on to do the work.

Response to Comments

The way in which I think Marc and I intended to approach this was to create a new kafka topic for changes made in the bound-with-parts endpoint, which is were bound-with relationships are actually tracked

This is based upon the decisions made above. The messages published to Kafka are expected to represent the record structure in storage. Hence, when information is introduced in a new record type (bound-with-parts), then a new topic and new messages have to be introduced.

The isBoundwith field cannot currently be published, as it is generated on-the-fly by the module when relevant records are called-so there is literally nothing there to publish.

I think that is a question for , as I believe he made the decision that the derivation would not be stored. I imagine it was done in order to not change the existing model (and couple this into the primary records).

Changes to bound-with data should really be the trigger for updating such a field.

Yes, if we chose to derive a new property on the instance, then this will need to be updated when the bound-with records change. It will also need to be checked / derived for any change that could affect the relationships e.g. move an item, move holdings and replacements (inc bulk) etc.

I think @marc has some concerns about this approach, so I'm posting this here, where relevant folks can see/discuss.

I'm trying to respect the prior decisions:

  • change notification messages are based upon record types

  • the bound-with-parts record was introduced in order to not affect existing records

  • the derivation of whether an instance has bound-with items or not, is a business logic consideration

We could add derived data in the instance record and change the instance change notifications. That means we are embedding the bound-with model into the existing record types (in storage at least), which I believe the SMEs and Thor team actively chose not to do.

Next Steps

to get this work moving along, would it then be helpful if I set up a meeting with you guys and Niels Erik?

There needs to be collective agreement on how this work is expected to be done across the teams (Thor, Falcon, Prokopovych) involved.

I suggest those conversations involve the relevant POs (who represent the SMEs) and , , , myself and possibly one of the EPAM SAs e.g.

Unfortunately, the Log4j work is likely to take up all of the remaining time before the Christmas / New Year holidays that folks are taking (and maybe afterwards too), so this work will like need to wait till after that is completed.

cc:

Duplicate

Details

Assignee

Reporter

Priority

Sprint

Development Team

Prokopovych

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created September 20, 2021 at 12:11 PM
Updated January 25, 2022 at 5:04 PM
Resolved January 25, 2022 at 5:02 PM
TestRail: Cases
TestRail: Runs