/
2025-05-07 Sys Ops & Management SIG Agenda and Meeting Notes

2025-05-07 Sys Ops & Management SIG Agenda and Meeting Notes

Translator

Date and time

12 EST

https://openlibraryfoundation.zoom.us/j/591934220?pwd=dXhuVFZoSllHU09qamZoZzZiTWhmQT09

Topics


Attendees

TimeItemWhoNotes

Welcome


Ramsons CSP-1 Installation Experiences from Ingolf

Upgrade Experiences



Ramsons Installation and Upgrade Experiences

1 Problems with mod-reporting replacing mod-ldp

Installation with Okapi's install endpoint. Dependecy resolution does not work. The system still expects mod-ldp to be present and installation fails with "Some modules cannot be topological sorted: mod-ldp-1.1.3". 

Ingolf eventually took out mod-reporting (and folio_ldp) from the platform.

There might be a way to solve this by de-activation of mod-ldp in Quesnelia, and then deploying mod-reporting like a new model.

Wayne has posted a prescription to migrate configurations from mod-ldp (in Quesnelia) to mod-reporting (Ramsons) to the sys-ops Slack channel.


2 Errors in the Launch Descriptor of Launch mod-search-4.0.11 

Non-Integer values provided where it should be Integers. This leads to failure of startup of mod-search. Launch Descriptor had to be edited manually to fix.


3 Memory problems with mod-users-bl / Problems to login with RTR

mod-users-bl failed "Terminating due to java.lang.OutOfMemoryError: Java heap space" upon a try to log in the first time to the new Ramsons system.

This disappeared after adding JAVA_OPTIONS: -Xmx2048m and also deleting (undeploying) all Quesnelia modules on the server before log in.

For Deployment with Okapi, also "Memory" in the Launch Descriptor has to be increased to 2 GB (without manual changes, it only takes up 500 MB – not enough).

mod-users-bl seem to need more memory (in Ramsons) than previously.


4 Asynchronous migration for mod-inventory-storage required. This took 19 minutes on Ingolf's server.




30Eureka variable naming conventions ; naming consistensy
  • Kevin Day created a Jira Issue: KEYCLOAK-53 - Getting issue details... STATUS  
    • work has been done for this issue. Pull requests have been made. Status is "In Refinement".
  • consistency of environment variables across multiple and within modules
  • observed inconsistencies among variable naming
    • variables having different names in different modules, but which express the same quantity
    • inconsistent naming (KC_, KEYCLOAK_, ...)
  • This is relevant for special modules like Kong, KeyCloak, FQM, etc..."
  • cf. Tech Council Discussion 2025-04-28 Meeting notes

Question to System Admins:

Where are the pain points ? What problems have system admins encountered ?

ENV variable. Kafka topics pull that value in. Elasticsearch index also. ES requires a reindex with the new value. Unexpected dependency of the ENV variable value. Re-Indexing takes a lot of time for bigger libraries. For a multitenant system, setting ENV is required.

ENV = quesnelia

Maybe the ENV variable should be named better. Maybe it should be only for Kafka. It is not good to use the ENV variable for two purposes.

Maybe the purpose it that you could have multiple instances with the same tenant ID → ask the developers.

Upgrades get broken if you don't change the value of ENV between releases on a muti-tenant system.


env variables with dots: e.g.

okapi.url , tenant.url, kong.url  , am.url
that is inconsistent to environment variables naming schemes of docker containers. In Linux, this can cause errors.
Allowed standards: Uppercase letters, digits, underscore

We recommend following the IEEE standards: https://pubs.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap08.html


ERM modules take very long to spin up (in Quesnelia).

Many modules have to be adapted for more memory. mod-agreements needed to raise memory exteremely for 15 tenants; using gokb.

Many out-of-memory events and heap-space events.


For mod-fqm-manager, there are lowercase variables with dots.

Snapshot environments. The sample data are old and are erased from day to day.





30Airflow applicationsShelley Doljack
  •  Different workflow processes that use folio APIs
  •  Different observability tools that Stanford is using with hosting folio


Report about the Work of the Eureka Early Adopters Group

(optional)

Ingolf is attending the weekly group meetings since April 8, but he/his organisation is not an early adopter. He is taking on a listener's role.



Chat Log

Day, Kevin  an  Alle 18:23
Consider this Readme:

okapi.url
kong.url
KONG_ADMIN_URL

Florian Kreft (LRZ)  an  Alle 18:30
https://pubs.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap08.html

Day, Kevin 18:31
>The name space of environment variable names containing lowercase letters is reserved for applications









Topics for next meetings
  • Problems with RTR that Ingolf and colleagues encountered

users sometimes get kicked out unexpectedly, earlier than the expiration period. Ingolf has captured log files for this failure.

  • Next meeting in 4 weeks

Action items

  • Type your task here, using "@" to assign to a user and "//" to select a due date

Related content