2020-01-17 - System Operations and Management SIG Agenda and Notes

Released January Mon 13th

Discuss our first experiences with it

Notes:

Edelweiss has been released, and there was already a hotfix.

Harry: Did anyone load some data, yet ?

Jason is running on 24 Cores with 64 GB of memory.

Import of bib records: Jason did 1.7 Mio in 6 hours

Jason runs 7 Okapis

Jo: We ar working around the issues with perfomance

Harry: But 4 Mio records is not a lot of data.

Jason at least quadrupled the heap memory from Q3.2 to Q4.

Taras Spashchenko has created recommendations for the postgres database configuration.

We will stop using postgres indexes for searches. The project will move to elasticsearch. That's the final recommendation (Jakub etc.)

Jo: postgres itself is not the problem. Jo has a postgres cluster that ingestst 5 GB of data per day. This is not a problem.

Harry + Phil : Postgres itself is a great tool. But we are using postgres for something that it was not built to be used for. We will use some other tool for that. Cassandra and Mongo are better suited for our purposes.