2026-04-15 Sys Ops & Management SIG Agenda and Meeting Notes

WolfCon Session Proposals

all

Submission Deadline: Wednesday, March 25th at 11:59 PM ET 

• has been extended to April 15th.

https://site.pheedloop.com/portal/event/EVEUQNAUXTGPI/submission/CALGJ08YMTMG9AM/application#start

“We encourage you to submit your proposals regardless of whether you are sure you can attend in person at this moment. We will ensure that in-person, virtual, and hybrid presentations can happen successfully.”

Previous Notes:

WolfCon
More structured - short talks
depict infrastructure requirements

Meeing Notes today:

Jason will submit a talk today. Wayne will be there in person.

FOLIO Eureka Installation Issues

Mahrad, Ingolf, Shelley; all

Stephanie Spratt from Moebius. They have moved to Sunflower on Eureka.

Permissions were in Users App in Okapi => Roles, Capabilities and Capability Sets in Eureka. These live in mod-settings. Stephanie shows an App of mod-settings in action. It looks like “permissions” in the Users app, but the Capabilities have no names ! They invent names for the Capabilities. You can get a list of descriptions, what the capabilities do.

Kong routing: in Sunflower, only one version per module. Also for Kafka, different versions of modules don't know what message they should pick up. Both issues will be resolved in Trillium. Jason: It will be backported to Sunflower.

Kong web interface is on a different port.
KONG_ADMIN_URL is set in Eureka common secret. You can port-forward to that.

Ingolf: in mgr-tenant-entitlements set APPLICATION_KONG_ENABLED = false. Otherwise, Kong might not find all routes.
Magnus: I have found a number of strange env vars while reading the change logs for the releases.

Magnus: We will go to Ramsons on the Weekend, on an Okapi Platform.
There is no way to upgrade in Eureka, yet.

Shelley: We went from Okapi Ramsons to Eureka Sunflower.

[minio]
Shelley: minio is in an archived state. The container Image is archived.
Mahrad: They changed the name to AI store. You have to build the image yourself.

Florian G.: I think you have to buy a license. There a some alternatives, but they are not really ready for production. Until then, we will wait and stay on an old minio version.
CSI Storage Classes. "Ruk-Set" it's a CEPH cluster inside kubernetes. It is not worth to run this inside Kubernetes, just for a few data. It's a monster.

[Question of upgrading a Kubernetes Node]

[Mahrad:] Removing and adding virtual machines. Adding a Node and removing the old one will make you headaches.
Preparing the nodes outside the cluster and then adding it is better.

Jason: You have to upgrade every tenant in your environment. That fix is going to come out soon and will be backported to Sunflower. The issues with multi-version enviromnents were with Kafka. In Okapi you can just upgrade straightforwardly.

Jason: The recommended migration path is to upgrade to Okapi Sunflower, then migrate to Eureka Sunflower. Sunflower is the new cutover-release. Do permissions and roles migration there.

Shelley: We ended up with duplicated timers. We did Ramsons Okapi to Ramsons Eureka. And then Sunflower Eureka.

Currently, you have to drop Kong and Keycloak tables and recreate everything. In Okapi, it was all stored in the FOLIO database.
In Eureka, you have to recreate (users) after the upgrade.
Then, you have to re-entitle everything.

Jason: The test environment tenant IDs and the system user passwords were the same across all environments. We would change it after the restore. The users want to use the same logins across test, dev and whatever environment.
The only problem is the secure store. Stand up an empty environment. Then drop it and import it from your dump.
You don't want to override the secure store.
Shelley: I want to be able to copy the data.
Jasons: I have my own repo with my hand created entitlement files. I don't know a better way to do this.

Open Topics

Next Meeting

In two weeks: 29 April.

Chat Log

Sie 18:24
 in mgr-tenant-entitlements set  APPLICATION_KONG_ENABLED = false. Otherwise, Kong might not find all routes.

Sie 18:38
Im am using 4.0.0-SNAPSHOT.199 for mgr-tenant-entitlements.

Shelley Doljack 18:50
https://s3.amazonaws.com/foliodocs/api/mgr-tenant-entitlements/s/mgr-tenant-entitlements.html#tag/entitlement/operation/applyState

Shelley Doljack 18:56
We have a ceph cluster running outside our k8s cluster.

Sie 18:58
VMware, not Citrix

Shelley Doljack 18:59
We just put in new worker nodes in our k8s cluster and we cordoned the old worker nodes while the new nodes got added and then restarted all our folio deployments to get them on the new nodes.

Topics for next meetings