2021-10-05 - Sys Ops & Management SIG

Owned by Ingolf Kuss
Last updated: Nov 01, 2021 by jroot

SysOps SIG NFRs Initiative & Integrations Work Group Meeting

Date

Attendees

Ingolf Kuss

Ian Walls

jroot

Philip Robinson

Wayne Schneider

Robert Douglas

jpnelson

Chris Rutledge (second half)

...

Goals

  • Push forward our NFRs initiative / Define and describe top 3 NFR requirements that cause most pain and need to be worked on
  • Integrations initative: Collect integrations from more Implementers, group and prioritize.

Discussion items

TimeItemWhoNotes



Note Taking: Ian ; Ingolf

FOLIO infrastructure / NFRsIngolf

Next Steps // ToDos // What we should focus on 

I.) Define 1 - 3 NFR requirements

  • Work on 1 NFR per release, or span across multiple releases.
  • Start with end user Pain Points; these will get the biggest attention:  Check Out Performance ; no root causes determined, yet. That is a case for examining the domain boundaries. You need to store too much. It requires careful consideration. It results in a monolithic chunk. Do we really want that ? You might want to have an ERM only system without inventory.
  • The data import problem => srs has relational database schemas; most data import ... talks directly to Kafka; issues with Kafka in multitenancy environments. Wayne: that work is done. You can use one Kafka across multiple Kafka instances and also the tenant id as part of the ...

  • Summary from our discussion 10/01:





    • I.a)) Enlarge Domain Boundaries.
      • More modules require more communication.
      • The more modules you have, the more round trips you have.
      • Domain boundaries will probably have the  biggest impact and benefits to the platform overall — and take the most time to implement.
      • Identify several modules that need to be merged together.

Is this in the scope of SysOps, or more for SMEs/POs/SIGs? It is between SMEs and architects.

Circulation may be a good example of this; so much information about loans, inventory and patrons has to be consulted to perform circulation actions.  Can some of those calls be reduced?  Users shouldn't require circulation.




    • I.b)) Get rid of the distinction between business logic modules and storage modules.
      • fully embrace the microservices architecture / need to reconcile this concept with merging BL and storage modules.
      • define operations where eventual consistency is fine
      • "Get rid of storage modules" 

Is this Domain Boundaries in another form? Yes. Can I run a big module independently of the rest of these things ?




    • II.) Change JSON schema to a relational database schema
      • The amount of processing getting data in and out of JSON fields is insane.
      • JSON was a compromise for full-text search.
      • Search is now driven by ES.
      • Keep JSON schema for communication, scrub it for storage.
      • This will facilitate patching.

mod-source-record-storage has been re-engineered to be more relational to improve performance


FOLIO Integrations

Status of the work of the Integrations Working Group

  • Question to Paula about having us in the Implementers Group  => no response yet; Oct 5th Implementers has been cancelled
  • Integrations Pain Points Tab Folio Source Data and Integrations

Goal: rank integration pain points... maybe not ready to do that yet today.  SSO seems like a possible high priority.

Much work has been done on INN-Reach. Much is apart from the capacity planning process.

One Option is: Hire EBSCO, IndexData or some other vendor. E.g. only very few institutions have remote storage; not main stream modules. Other examples: CAIASOFT, Dematic.  Capacity Planning is only one possible group to work with; we're empowered to push our own institutional needs (provided we provide the necessary resources to make it happen). Cap Planning cannot provide resources for the entire project.

The FOLIO module mod-remote-storage abstracts the need for particular vendors to manage remote storage. The edge modules specify the integrations for particular vendors edge-caiasoft, edge-dematic. The vendors pour their resources in to get the development done.


Topics for future meetings
  • Migration scripts between FOLIO instances; e.g. circulation rule migration


Zoom Chat


Wayne Schneider: Perhaps data normalization is a better term than integrity in this context
Ian Walls: How much does it 'cost' to spin up a module in terms of cluster resources?  How much of that is overhead, and how much is meaningful logic?
Wayne Schneider: That’s a great question, Ian. Of course, it depends (in large part on the data set).
That is, if you’re trying to operate on a lot of data at once, you need more heap. If you’re operating on one thing at a time (e.g. circulation), not so much.
Root, Jason M: Indeed there are some memory requirements that are determined by the module developers. I consider those the “minimum requirements” for memory at least.
Ian Walls: The topic of "making FOLIO easier to install" has been brought up in all 3 Councils several times in the last few weeks.   So it's getting airtime
Philip Robinson: That’s good to hear, Ian - I knew TC was on it of course but didn’t know about the others

Action items

  • Type your task here, using "@" to assign to a user and "//" to select a due date
  • Get NFRs on the Roadmap
  • jroot will ask Paula Sullenger to have us speak up in the Implementers Group.
  • All: Pull together a list of existing JIRAs (UXPRODs, Epics) for these top NFR requirements. Post it in the Chat. Edit these JIRAs / Add more JIRAs (UXPROD, stories).