2023-06-09 Sys Ops & Management SIG Agenda and Meeting notes

2023/06/02 Meeting Notes:

 Our WolfCon Session

Tentative title:
  "Platform Decisions for FOLIO Self-Deployment on Kubernetes"

•  Schematic Installation of FOLIO using Helm Charts (Tobi)
• Building a Collaborative Repository - not in the title, but as part of the talk
• Collaborative Repository for FOLIO Setup
• CA/CI-Pipelines
• Self Hosting Services /  Infrastructure agnostic
• Requirements to the infrastructure (for a cluster) / what decisions need to be made ?
•  AWS or self-hosted cluster ?
• a minimum for a highly-available FOLIO instance
• What configurations will be needed if you operate FOLIO in a cluster and not on a single VM ?
    K8s-Cluster; of more remote interest: Kubespray, Ingress, Storage, Backup
• Tobias: a common repository for Helm Charts / Infrastructure / Planning and Setup
•  Starting point - Helm Charts - then you end up with ... ... these are the obstacles
• What is that thing doing ? Which infrastructure do you need ?

----------------------------------------------------

Does anyone here use helm charts on public cloud infrastructure?

Colorado (Marc Stacy), Cornell was doing it for a while

folio-org/folio-helm: Helm charts modules repository (github.com)

https://github.com/folio-org/folio-helm-v2/

Experiences and Recommendations from Sys Ops for self-hosting FOLIO

Session Title: Platform Decisions for FOLIO Self-Deployment on Kubernetes

Session Description: This session is meant for systems administrators who plan to self-deploy FOLIO in a general Kubernetes environment (private cloud or public cloud). We will share experiences and give recommendations and insights for self-hosting FOLIO. We will address aspects of needed infrastructure and a usable common repository for Helm Charts.

Followed by a short discussion

K8s is out of scope.

..............................................................

Tobias: We should indicate "SysOps SIG" in the names column, as other presenters do.

Other (proposed) sessions of potential interest for SysOps:

Architectural Roadmap - Organizer: Tod Olson

FOLIO and Next Level AuthN/Z - Craig McNally

Applications and the Management Thereof - Craig McNally

Hosting Metadb - best practices - Axel Dörrer

Using the FOLIO Migration Tools - Kyle Banerjee

Metadb 1.0: an open-source data platform for analytics - Nassib Nassar

FOLIO goes LibraryNetworX - Kirstin Kemner-Heek

FOLIO Release Management Process: How does this Work? - Kristin Martin

Software testing: How to design a software testing process for everyone - Wilhelmina Randtke

Brainstorming Workflows in FOLIO - Jeremy Huff

Current State of App and Module Evaluation by the PC and TC - Jeremy Huff

... and others !

Ingolf: Using mod-oai-pmh to harvest the title data from FOLIO, importing into VuFind. This will harvest only the title data (inventory instances). For a full-fledged Discovery system, you would also need information about locations and items, about availability and a users's loans.

 Stewart: We are using Black Light with FOLIO. edge-rtac is enriching the OAI-PMH calls. RTAC (harvests) MARC21-Holdings. We ended up doing it thru GraphQL. We are harvesting every 30 Minutes. Doing 5,000 calls in 30 Minutes.
  Stewart: It is not in DC;  rtac adds availibilty information into MARC Local fields and is moving that around.
                  We have ~ 8 Mio records; the first OAI-PMH Harvest took 90 hours.
                  OAI-PMH best practices page - speed claim - we weren't able to hit that.
                  OAI-PMH gives you hrid, but you need a uuid.
 Stewart and Uwe are in the Discovery group #discovery. Much has been discussed there.
 Jeremy: We are using Black Light .
   

The new consortia functionality (mod-consortia, ui-consortia): Will installers who do not need it be able to succesfully separate it out ?

Ingolf: ECS introduces changes in the user login mechanism. mod-login and mod-authtoken and other modules have already been changed in the current main branch. [UXPROD-4092] ECS user login mechanism - FOLIO Issue Tracker

Ingolf / Julian: The changes have induced security concerns. These concerns have been reported to the Technical Council. Nevertheless, the TC has accepted the new funtionalities. 

Ingolf / Julian. These security concerns pertain to the consortium single-sign-on procedure. Tenant checks have been disabled. See the discussion led in the TC. https://folio-org.atlassian.net/wiki/display/TC/2023-06-05+-+Consortia+Tenant+Checks . This discussion will be continued.

Ingolf: The concerns about the new consortia features have been expressed in a Slack Thread of the Product Council. It is this Thread: https://folio-project.slack.com/archives/CTYQZ7PF1/p1686053665347659

Ingolf reads the main parts of this Thread to the group. Can SysOps weigh in ?

Stewart : We are in 2 consortia: One within our State; One with Stanford. Also Ivy+
Ingolf: mod-consortia won't work for institutions who are a member of more than one consortium.

Stewart:   When using Primo in Alma, you also have problems with more than one consortium.

Ingolf: Ann-Marie Breaux and the Folijet team will test these new functionalities in all scenarios (consortia tenants, no consortium). We agree that the new features must be backward compatible, i.e. allowing for institutions that do not need it to disable them or not use them.

Ingolf: Another topic we will face is online update between a central union catalogue and FOLIO as the local library system. We will have Alma NZ as the central union catalogue, will need interfaces in FOLIO and Alma.

Stewart: We are in a consortium where the other members are on Alma. We were on Alma ourselves, before FOLIO. They are all connected to the same Discovery and exchange data via the Discovery. I am using a software that does the conversions based on MARC21 data exchange. He shows the software repository and configurations for his consortium: trln/marc-to-argot: Tools for converting MARC records to shared TRLN ingest format (github.com) . It converts MARC input to Argot, TRLN Discovery's shared ingest format. The institutional specifications are in lib/data/<inst>/.

• Overview: List of Integrations - System Operations and Management SIG - FOLIO Wiki
• Discovery: Discovery Integration Subgroup - System Operations and Management SIG - FOLIO Wiki  → partially covered today