2020-01-17 - System Operations and Management SIG Agenda and Notes
- Ingolf Kuss
Date
Attendees
- Johannes Drexl
- Hkaplanian
- jroot
- Former user (Deleted)
- Anton Emelianov (Deactivated)
- Brandon Tharp
- jackie.gottlieb@duke.edu (Deactivated)
- Catherine Smith
- spampell
Goals
Discussion items
| Time | Item | Who | Notes |
|---|---|---|---|
| Edelweiss Release | Released January Mon 13th Discuss our first experiences with it Notes: Edelweiss has been released, and there was already a hotfix. Harry: Did anyone load some data, yet ? Jason is running on 24 Cores with 64 GB of memory. Import of bib records: Jason did 1.7 Mio in 6 hours Jason runs 7 Okapis Jo: We ar working around the issues with perfomance Harry: But 4 Mio records is not a lot of data. Jason at least quadrupled the heap memory from Q3.2 to Q4. Taras Spashchenko has created recommendations for the postgres database configuration. We will stop using postgres indexes for searches. The project will move to elasticsearch. That's the final recommendation (Jakub etc.) Jo: postgres itself is not the problem. Jo has a postgres cluster that ingestst 5 GB of data per day. This is not a problem. Harry + Steve : Postgres itself is a great tool. But we are using postgres for something that it was not built to be used for. We will use some other tool for that. Cassandra and Mongo, for example, are better suited for our purposes. Jason: Folio only supports postgres 11, but we need postgres 12. Texas A&M is doing all on premise, they are not using AWS. They are looking at a crunchy postgres solution. There are enterprise solutions for that. Jo: We should also try to not containerize. Use 1. Kubernetes 2. postgres running on hardware Jason: We are seeing the same performance. Using postgress with ingress load balancing. We are not seeing a performance difference. If we run it on NFS, then it is far slower. Brandon: Crunchy is all open source. You can pay for support or run it on your own. Steve: Crunchy is a certified stack for open source tools. A&M will pay them for engineering support. The tools are all free and open source. Jason: Chris Creswell runs a postgres server. Also Dale in Chicago. Jason is running databases on two separate hosts. This speeds up the responsiveness of our system. Hong-Wei Ji is the lead person for database setup at EBSCO. Database upgrades Jason documented the upgrading steps from Q3.2 to Q4: Texas A&M University Libraries experiences Harry is targeting Feb 4th for the Chalmers upgrade. So far they have been able to do it multiple times with no data loss. Mark Veksler manages the team who does this. Cornell is on the verge of going live with ERM on Folio. Robbie is working on that. He is deploying locally using AWS. Maybe they hope to announce at WolfCon that they have gone live. | ||
| Preparations for WolfCon | WolfCon Schedule: https://wolfcon2020a.sched.com/ https://wolfcon2020a.sched.com/overview/type/SO Have we prepared all meetings ? What do we still need to do to make the meetings become successful ? Notes: Pretty much all sessions will happen on Wednesday; not sure if that was on intention. Data migration is on Thursday and Friday. On Thursday at 2 PM someone from Chalmers is presenting on OpenAthens integration. They have to allow walk-ins, because they are state-funded. So, they implemented OpenAthens which can connect to the Campus SSO and in addition uses its own database. |