Page Comparison

Released January Mon 13th

Discuss our first experiences with it

Notes:

Edelweiss has been released, and there was already a hotfix.

Harry: Did anyone load some data, yet ?

Jason is running on 24 Cores with 64 GB of memory.

Import of bib records: Jason did 1.7 Mio in 6 hours

Jason runs 7 Okapis

Jo: We ar working around the issues with perfomance

Harry: But 4 Mio records is not a lot of data.

Jason at least quadrupled the heap memory from Q3.2 to Q4.

Taras Spashchenko has created recommendations for the postgres database configuration.

We will stop using postgres indexes for searches. The project will move to elasticsearch. That's the final recommendation (Jakub etc.)

Jo: postgres itself is not the problem. Jo has a postgres cluster that ingestst 5 GB of data per day. This is not a problem.

Harry + Phil Steve : Postgres itself is a great tool. But we are using postgres for something that it was not built to be used for. We will use some other tool for that. Cassandra and Mongo, for example, are better suited for our purposes.

Jason: Folio only supports postgres 11, but we need postgres 12.

Texas A&M is doing all on premise, they are not using AWS. They are looking at a crunchy postgres solution. There are enterprise solutions for that.

Jo: We should also try to not containerize. Use 1. Kubernetes 2. postgres running on hardware

Jason: We are seeing the same performance. Using postgress with ingress load balancing. We are not seeing a performance difference. If we run it on NFS, then it is far slower.

Brandon: Crunchy is all open source. You can pay for support or run it on your own.

Steve: Crunchy is a certified stack for open source tools. A&M will pay them for engineering support. The tools are all free and open source.

Jason: Chris Creswell runs a postgres server. Also Dale in Chicago. Jason is running databases on two separate hosts. This speeds up the responsiveness of our system.

Hong-Wei Ji is the lead person for database setup at EBSCO.