2023-06-02 Sys Ops & Management SIG Agenda and Meeting notes

The OLF folks want a draft list of session ideas by the first week of June, so less than one month from now.  You can see last year's FOLIO sessions here.  So, what technical sessions should we have this year?

Let's collect ideas for SysOps sessions.

Meeting Notes 2023-05-12

WolfCon 2023 planning form August 22 - 24.

Attendance in person: Tod Olson jroot 

Maccabee Levine has volunteered to wrangle together the technical FOLIO session proposals. 

-------------------------------

Suggestions:  A FOLIO install session (suggested by Ian Walls ?). Relevant to SysOps.  Florian Kreft is investigating. Tobias Stumpp jroot 

Probably not a big on-person audience of system operators. => Participate in other meetings of technical nature.

Managing deployment. Most are self-deploying. Helm chart completeness vs single tenant installs vs error prone due to many back-end modules.

One project: building a world-wide repository for Helm charts for FOLIO K8s deployment. Maybe not in github. Maybe together with dev team members ?

-----

Allow for a simpler basic install: Define Apps as collections of modules. Mike Taylor gave a presentation last year.

Deployment by Apps.  Not only important for new installers. Easier to automate end-to-end testing. Make testing easier. Enabeling future development.

Maybe the list of supported technologies ?

Sys Ops shall network with other technical folks on WolfCon session.

---------------------------

What needs to be addressed most urgently ?

What are the most important problems:  documentation in docs.folio.org.  Many decisions that you have to make for your infrastructure.  Decisions about AWS storage , minio, elasticsearch installation ,...     a mixture of different guides is online

Contribute back our experiences.

Database tuning properly. Migration runs.  SRS imports failed after setup of Metadb. Critical non-folio dependencies need to be better documented.     Sessions with reporting people ?   The guides mention PSQL version 12. It would be beneficial to switch to newer PSQL versions to increase performance.  

Linköping University planning go live last week of July.

Other topics:

• Integrations

• Discovery integration

----------------------

Clarify: Remote meeting rooms (technical equipment: PC with Zoom, microphone, beamer, camera). 

2023-05-19

From Data Migration Group: potential WolfCon session:

• High level guidance on how to set up database, resourcing, settings, best practices, Kafka, Elasticsearch (SysOps might run with this one).
• Session/panel of recent migrators? UChicago, Davidson (June ’23)
• Would be good for EBSCO to do a session on their tools / Maybe Ingolf to demo migration using EBSCO's migration tools.

For SysOps maybe an overview of what has changed / what work has been going on in this last year, since the last WolfCon ?

Pick up "hot topics" from last WolfCon (cf. 2022-09-07 Meeting notes - Technical Council - FOLIO Wiki)

Ideas:

• Support of OpenSearch vs. Elasticsearch (ES)
  • an incompatibility with the license of some version of ES 7
  • currently, mod-search supports both ES 8 and OpenSearch
• Platform minimal
  • [FOLIO-3253] SPIKE - Investigate minimal FOLIO platform - FOLIO Issue Tracker
  • Changes in authentication and gateway (Okapi). e.g. [STCOR-650] remove service-point handling - FOLIO Issue Tracker
  • A tag to see all tickets together: [MODUSERBL-155] Spike: Investigate current implementation for using optional interfaces - FOLIO Issue Tracker
  • postgresql and Okapi are required, Kafka and Elasticsearch not at this point.
  • A script by Adam Dickmeiss: https://github.com/adamdickmeiss/folio-local-run
  • [FOLIO-1730] Create folio/minimal Vagrant box - FOLIO Issue Tracker
  • platform minimal contains the fundamental building blocks to build a FOLIO system.
• Blue/green deployments
• Kafka messaging improvements
  • Kafka messaging for tenant collections. 

    Reduce Number of Partitions in Kafka By Using Single Topics Instead of Topics For Each Tenant. PUBLIC REVIEW | Tenant Collection Kafka Topics by okolawole-ebsco · Pull Request #8 · folio-org/rfcs (github.com)  - ongoing work

• API technical debt

Other topics that have been discussed in the TC in the last 12 month:

• controlling AWS hosting costs  . E.g. the discussion on 2023-03-01 Meeting notes - Technical Council - FOLIO Wiki
• approvement of mod-consortia TCR 25/26. A new app to manage consortia tenants & users. Adding cross-tenant functionalities to FOLIO.
• other new modules, technically reviewed by the TC: mod-oa TCR-6, adding open access functionalities (not accepted; relies on Groovy/Grails/Gradle → added to list of supported technologies); translation app TCR-9 (not accepted), ui-marc-authorities TCR-12 (accepted), ui-calendar / mod-calendar (TCR-16/17) (approved)
• mod-setting TCR-24. A centralized vs. distributed approach towards . Fixing security problems. Approved. folio-docs/customization.md at main · MikeTaylor/folio-docs · GitHub . A TC subgroup run by Florian Gleixner . 2023-05-17 : the deprecation statement in mod-configuration will be updated to reflect the fact that mod-settings is an alternative but recommends storing sensitive information within the module.
• technology changes: java 17, postgres 14, SpringBoot v3.0 (Spring Boot 2.7.* - OSS supported ends Nov 2023), ...   Maintaining List of  Officially Supported Technologies - Technical Council - FOLIO Wiki
• Breaking Changes (a TC subgroup). The group has been working on an RFC for defining breaking vs. non-breaking changes.
• Evaluation process for new FOLIO modules; working document: https://docs.google.com/document/d/1VtQXj5EgWGFUusKDbaLQTX5dSBZgwv_XlffBXJTrBAI/edit   TC may be covering this at WolfCon
•  DR-000032 - Splitting Database Read/Write Traffics  -  this has only been tested on AWS RDS and how database nodes synchronize data is outside of the scope of this work. There is very little downside if other modules don't adopt this approach. What are the downsides if other modules do adopt this approach? Martin Tran suggested that one downside is that RMB creates separate connection pools for read and write connections, which could use more memory in the database. Read more: 2023-02-01 Meeting notes - Technical Council - FOLIO Wiki

----------------------------

Meeting Notes:

Considered most important by Sys Ops:

Kafka issues

Platform minimal core

cost controlling measures

supported technologies