Hive off database to a separate server

This is a prescription how to turn a Single Server into a "Double Server". A Double Server is a cluster of two servers: One server runs everything except for the database (thus, Okapi, the Webserver (nginx), Stripes and the modules). The other server is a postgres database host which runs Folio's central database system (with the databases okapi and folio).

I. Set Up the database server

Prescription for Ubuntu 20.04.1 LTS


#1. Update the apt cache

sudo apt-get update

#2. Special Treatment for Ubuntu-20 (Thanks to https://askubuntu.com/questions/1261614/ubuntu-20-04-libssl-so-1-0-0-cannot-open-shared-object-file-no-such-file-or-d)

Edit the source list

sudo nano /etc/apt/sources.list

Add the following line:

deb http://security.ubuntu.com/ubuntu xenial-security main

sudo apt update
sudo apt install libssl1.0.0

sudo apt-get install libpq5

 #3. Import the PostgreSQL signing key, add the PostgreSQL apt repository, install PostgreSQL
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo add-apt-repository "deb http://apt.postgresql.org/pub/repos/apt/ xenial-pgdg main"
sudo apt-get update
sudo apt-get -y install postgresql-10 postgresql-client-10 postgresql-contrib-10 libpq-dev

#4. Start the cluster

sudo systemctl start postgresql@10-main

# 5. Configure PostgreSQL to listen on all interfaces and allow connections from all addresses (to allow Docker connections)

 Edit file /etc/postgresql/10/main/postgresql.conf to add line listen_addresses = '*' in the "Connection Settings" section

 Edit file /etc/postgresql/10/main/pg_hba.conf to add line host all all 0.0.0.0/0 md5

 Restart PostgreSQL with command sudo systemctl restart postgresql

--- the following can only be done once the database has been dumped on the main server --> see the prescription there ---

#6. Load database contents from the main server
#6.1 Load roles :
psql -d postgres -f roles.psql
sudo su -c psql postgres postgres
\dg    - list roles
\q

#6.2 Create databaes

CREATE DATABASE okapi WITH OWNER okapi;

CREATE DATABASE folio WITH OWNER folio;


#6.3 THIS STEP ONLY NEEDS TO BE DONE, instead of 6.2, if the databases already exist:

#6.3 Re-create databases

First delete sessions, which might still act on the database:
sudo su -c psql postgres postgres
\l (list databases)
UPDATE pg_database SET datallowconn = 'false' WHERE datname = 'okapi';
SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = 'okapi';
drop database okapi;
CREATE DATABASE okapi WITH OWNER okapi;
\q

And the same for the database folio:

sudo su -c psql postgres postgres

UPDATE pg_database SET datallowconn = 'false' WHERE datname = 'folio';

SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = 'folio';

drop database folio;

CREATE DATABASE folio WITH OWNER folio;

\q

#6.4 Load data from main server into the databases okapi and folio:
psql okapi < okapi.psql

psql folio < folio.psql


II. Changes on the main server, which runs Okapi

II.A    Change name of the main server (in Folio system)

If also the name of the main server changed, this needs to be done first on the main server:

#A.1 Re-Build Stripes package

- Change this line
  okapi: { 'url':'https://folio-hbz1.hbz-nrw.de/okapi', 'tenant':'diku' },
  in ~/platform-complete/stripes.config.js to use the new server's name.

  Re-build Stripes package:

  NODE_ENV=production yarn build output

#A.2 Re-Configure Webserver to serve the stripes packet :

 copy key and certificate for the new server name to /etc/nginx .

 concatenate the certificate and the certificate chain to a certificate bundle :

     cat cert-11065238610709564929899284079.pem chain.txt > cert_bundle.crt

  vim /etc/nginx/sites-available/stripes
  Change two lines:
    server_name  folio-hbz1.hbz-nrw.de;
    ssl_certificate_key folio-hbz1.hbz-nrw.de-key.no_enc.pem;

#A.2 Re-Start Webserver

  cd /etc/nginx

  cp -p sites-available/stripes sites-enabled/stripes

  Re-start nginx: sudo systemctl restart nginx

II.B  Use database of the new database server

  • #B.1 Dump all roles on the source database

    pg_dumpall -g > roles.psql

    Dump Database contents

    pg_dump okapi > okapi.psql

    pg_dump folio > folio.psql


  • Now first do step #6. of "Set Up the database server"  to import the database contents into the new database host.

Now connect Okapi to the new database host:

 - #B.2  Make new database host know to Okapi (for the database okapi):

  vim /etc/folio/okapi/okapi.conf
     host="193.30.112.85"
     postgres_host="10.9.2.61"
     okapiurl="http://193.30.112.85:9130"


 - #B.3   Make database information (for connecting the modules to the database)  known to the Okapi environment (POST to /env) :

  curl -w '\n' -D - -X POST -H "Content-Type: application/json" -d "{\"name\":\"DB_HOST\",\"value\":\"10.9.2.61\"}" http://localhost:9130/_/env

#B.4 (Optional ? ) Clean up unused docker containers:

  sudo su

  docker system prune

  docker image prune -a

  systemctl stop docker.service

  systemctl start docker.service


 - #B.5 Re-start Okapi

    sudo systemctl daemon-reload

    sudo systemctl restart okapi

#This finally re-deploys all modules, connects them to the new database host and activates them for the tenant (really ?)

#If that didn't work, do also this:

curl -w '\n' -D - -X POST -H "Content-type: application/json" -d @/usr/folio/platform-complete/okapi-install.json http://localhost:9130/_/proxy/tenants/diku/install?deploy=true\&preRelease=false

and
curl -w '\n' -D - -X POST -H "Content-type: application/json" -d @/usr/folio/platform-complete/stripes-install.json http://localhost:9130/_/proxy/tenants/diku/install?preRelease=false