Integrate mod-inventory with mod-pubsub

RCA Group

None

Description

Motivation

Allow mod-inventory to act as publisher/subscriber of events related to data-import process.

Steps

1. Based on data-import events described in matrix define which events will be published by mod-inventory modules and to which events it will be subscribed.
2. Define necessary EventDescriptors in a MessagingDescriptor file (see documentation) for events that the module publishes: "CREATED_INVENTORY_INSTANCE", "CREATED_HOLDINGS_RECORD", "CREATED_ITEM_RECORD", "UPDATED_INVENTORY_INSTANCE", "DATA_IMPORT_ERROR".
3. Define module's subscription on the event "CREATED_SRS_MARC_BIB_RECORD" in a MessagingDescriptor file.
4. Register module in pub-sub (see documentation).

Results

mod-inventory is registered in mod-pubsub as publisher/subscriber of events related to operations on Instances, Holdings and Items.

Environment

None

Potential Workaround

None

Checklist

hide

TestRail: Results

Activity

Show:

Marc JohnsonJanuary 28, 2020 at 2:05 PM
Edited

Thanks for your response

Would it be helpful to set up a mtg with (Folijet team lead), (architect) and maybe and to go over your questions?

I'm happy to meet with folks, if we feel it could be useful. I don't really know how to answer that, it depends upon what involvement folks want me to have in this work going forward?

provided some thoughts already on the pull request.

In order for me to contribute useful feedback on this work I need to understand the goal and process better. I asked the questions to try to understand where this work is going, and what impact that has on the responsibilities for inventory.

If so, I'm happy to set that up.

Thank you for the offer.

It seems like we should get these questions sorted and you up to speed on PubSub before we try to plan the larger developer meeting on PubSub.

I don't know how important it is for my questions to be answered before this meeting is arranged.

The first set of questions are around process design and modelling responsibilities in FOLIO, and not really specific to pub sub. My interest in understanding that part is primarily to understand the design of the import process and what responsibilities inventory has in that process.

The second set of questions are related to technical standards and governance, and could be considered out of scope for an initial meeting describing pub sub. I would hope that other folks have similar questions to this as a follow up to that conversation, but they might not.

is keen for Core Functional to start to use pub sub soon to fix some issues with synchronising state changes between inventory and circulation. I think it could be useful to know whether FOLIO intends to have a standard for event definition before the team starts that work in earnest.

The one thing I can say about tense in the naming conventions - I think that tense is present tense if the event being announced is to do something (e.g. create this or that) but past tense if the event being announced is that something was done (e.g. created this or that)

This might be where I have a different conceptual understanding (this has come up before in other conversations, it comes from my experience of the distributed domain driven design community).

I think of an event as representing that something has already happened (an immutable fact) and hence always are expressed in the past tense, e.g. order has shipped.

Whereas, I think of the present (or future) tense as the intent to change or do something e.g. ship order and think of these as commands rather than events.

It may well be that FOLIO has chosen to use a different definition of event, and I need to get used to that, to reduce my confusion.

cc:

Ann-Marie BreauxJanuary 27, 2020 at 7:47 PM

Hi Would it be helpful to set up a mtg with (Folijet team lead), (architect) and maybe and to go over your questions? If so, I'm happy to set that up.

It seems like we should get these questions sorted and you up to speed on PubSub before we try to plan the larger developer meeting on PubSub.

The one thing I can say about tense in the naming conventions - I think that tense is present tense if the event being announced is to do something (e.g. create this or that) but past tense if the event being announced is that something was done (e.g. created this or that)

Thanks,
A-M

Marc JohnsonJanuary 27, 2020 at 4:14 PM

Whilst trying to review the pull request for this issue, it became clear that I had insufficient understanding of both the intended behaviour or process being implemented and the work decomposition strategy to adequately review this change.

Given that (and no reviews by other code owners), I suggest that review / approval responsibilities for this work moves to folks with more awareness of the intended output.

Whilst reviewing this work, it raised a bunch of questions that I felt I needed to raise, to gain a better understanding of the broader context of these changes. Especially as it forms the start of significant changes to data import, inventory and how FOLIO modules integrate together.

As usual, feel free to disregard these questions or feedback if you do not feel that they are useful. (I don't really know who owns this process or the technical design for it, so I'm being rather vague about my audience for these questions)

Import Process

I am trying to understand the intended changes to the import process better, at the moment, my main route to doing that is through inference based upon the events defined in the work for this current issue.

This change defines the following events:

  • CREATED_INVENTORY_INSTANCE

  • UPDATED_INVENTORY_INSTANCE

  • CREATED_SRS_MARC_BIB_RECORD

  • DATA_IMPORT_ERROR

Responsibility for mapping MARC to instances

My understanding is that, prior to this, mod-data-import / mod-source-record-manager was responsible for converting MARC records into FOLIO instance records and creating them in inventory.

My interpretation of this change is that mod-inventory is going to react to CREATED_SRS_MARC_BIB_RECORD events. That suggests that mod-inventory will be responsible for converting a MARC record into an instance, rather than data import, which currently does that conversion.

Is this a sensible understanding? If so, what does that mean for action profiles, which are defined as part of data import and not inventory?

Instance Events

What is/are the intended reaction(s) to an instance being created or updated? Which part(s) of the system will perform that reaction?

Data Import Error Event

If I am following this change correctly, mod-inventory is responsible for publishing a data import error. Is that a reasonable interpretation? If so, under what circumstances should inventory publish a data import error? It seems odd to me that inventory would publish an event owned by data import which, as I understand it, is a distinct and separate contexts within FOLIO.

Event Naming Guidance

Prior to this work being started, I (possibly prematurely) raised the topic of event naming. I created [FOLIO-2408] for the purpose of that discussion. I personally think this topic is important in order to aid consistency across FOLIO and hopefully help with folks attempting to understand the system.

I am bringing it back up in the context of this work because, by defining the above events, it potentially establishes a de facto standard for event names:

  • All capitalised

  • Words separated by unnderscores

  • Past tense verb at the beginning

  • Un-versioned (no version information in name)

  • Global (no partitioning into namespaces)

Is this the intended standard for naming events in FOLIO (if it is, then I think it should be explicitly documented)?

Beyond that, I think it could be worthwhile establishing whether events are domain centric e.g. inventory or module centric e.g. source-record-storage or SRS (and what kinds of modules we expect to publish events in FOLIO)?

cc:

Ann-Marie BreauxJanuary 23, 2020 at 5:19 PM

Yes, looks like it's FOLIO-2432. I'll get with you next week (after WOLFcon) to discuss arranging the meeting. Ideally folks will do some homework before the meeting, so that we don't have to start from scratch. There's a lot of technical documentation outlining the basis.

Marc JohnsonJanuary 23, 2020 at 4:35 PM
Edited

I've raised [FOLIO-2432] (thanks for the correction, ) for introducing publish / subscribe to the broader developer community, so it can be discussed separately to this issue.

Done

Details

Assignee

Reporter

Priority

Story Points

Sprint

Development Team

Folijet

Fix versions

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created December 19, 2019 at 10:10 AM
Updated March 9, 2020 at 1:35 PM
Resolved February 11, 2020 at 8:26 AM
TestRail: Cases
TestRail: Runs