Example of OKAPI to EUREKA migration overview

1 Overview
2 Phase 0 - Prerequisites:
3 Phase 1 - Components:
4 Phase 2 - Management components:
5 Phase 3 - Discovery and Apps registration:
6 Phase 4 - Tenants:
7 Phase 5 - Entitle:
8 Phase 6 - Admin user:
9 Phase 7 - Users & roles migrations:
10 Phase 8 - Post-migration activities:
11 Phase 9 - Users validation:
12 Phase 10 - Hyper care:

Overview

This EXAMPLE explains migration from OKAPI to EUREKA on Folio Rancher (Kubernetes) infrastructure.

Purpose of document: Show overall steps required to perform migration, actual PRODUCTION steps may differ. THIS IS NOT A STEP-BY-STEP RUNBOOK.

Strongly suggestion to follow DB dumping approach and testing all of the migration's steps on test environment.
PLEASE TEST ALL OF THE THINGS BEFORE PRODUCTION MIGRATION.

THIS IS ONLY DEMONSTRATED EXAMPLE FOR MIGRATION THAT WAS DONE AT RANCHER ENVIRONMENT.

This guide outlines the steps for migrating from OKAPI to EUREKA within the Community Rancher environment. Please note that depending on the specific technology stack used in your own environment, additional steps may be necessary to complete the migration.

Please update this guide to reflect any changes specific to your environment. Clearly document any differences from this process, along with the additional steps taken to address them.

***For production migrations: You perform all actions at your own risk***

Example assumes that existing OKAPI environment is being hosted on Kubernetes platform, either in self-hosted mode (On-Premises) or on AWS Cloud platform, all of the below activities referencing Kubernetes based environment(s) specifically for RAMSONS release (folio-org/platform-lsp at Ramsons-release).

Assumptions:

Assuming that the target infrastructure already has below components deployed in newly created namespace:

new k8s namespace created: kubectl create namespace {{YourNamespaceName}}
Kafka deployed in {{YourNamespaceName}}
(Elastic||Open)-Search deployed in {{YourNamespaceName}}
PostgreSQL deployed in {{YourNamespaceName}}
Vault (in case of self-hosted variant) in {{YourNamespaceName}}
Minio (in case of self-hosted variant) || S3 buckets for all required BE (Backend modules) in case of AWS setup deployed in {{YourNamespaceName}}

Prerequisites:

ingress controller is installed on target cluster (nginx, traefik, AWS ALB controller etc.)
“kubectl” utility installed and target cluster configured with required context, to execute “kubectl” commands
helm installed and configured
Existing DB (PostgreSQL) instance\server MUST be at least version 16.1.0 or higher (if not, please perform upgrade, prior to starting to work on migration)
Dump\Snapshot existing env’s DB (before\after upgrade to v16.1.0 or higher)
Update existing environment to Ramsons (at least)
Once update process is complete, take another backup of the DB with Ramsons version
Save existing tenants structure from: https://{{okapiUrl}}/_/proxy/tenants (example of RBF env), it will be required later on

Phase 0 - Prerequisites:

build migration testing environment for migration purposes.

restore dumped DB(OKAPI based env) on DB instance in {{YourNamespaceName}}

create required Kubernetes secrets

minio
eureka-edge
eureka-common
db-credentials
kong-credentials
kafka-credentials
vault-credentials
keycloak-credentials
opensearch-credentials

Examples of secrets: Example of k8s secrets - Folio Development Teams - FOLIO Wiki

NOTICE: All secrets MUST match all related values used for components deployment i.e. KAFKA_HOST must be pointing to existing kafka broker and so on.

Phase 1 - Components:

Required deployments:

Keycloak → README.md, example of k8s deployment || required version
Kong → README.md, example of k8s deployment || required version
Kafka (if not deployed already) → example of k8s deployment
Vault (if self-hosted approach is used, if env is hosted on AWS Cloud, please skip this deployment) → example of k8s deployment

Required configurations:

DB and DB role creation for Keycloak
DB and DB role creation for Kong

P.S. Below DBs is mandatory requirement. MUST BE pre-created, before Kong and Keycloak deployments.

CREATE DATABASE kong;
CREATE USER kong PASSWORD 'secretDBpassword'; <---- ALL OF THESE SETTINGS SHOULD BE REFLECTING ACTUAL CONFIGURATION FOR KONG AND KEYCLOAK.
ALTER DATABASE kong OWNER TO kong;
ALTER DATABASE kong SET search_path TO public;
REVOKE CREATE ON SCHEMA public FROM public;
GRANT ALL ON SCHEMA public TO kong;
GRANT USAGE ON SCHEMA public TO kong;
CREATE DATABASE keycloak;
CREATE USER keycloak PASSWORD 'secretDBpassword';
ALTER DATABASE keycloak OWNER TO keycloak;
ALTER DATABASE keycloak SET search_path TO public;
REVOKE CREATE ON SCHEMA public FROM public;
GRANT ALL ON SCHEMA public TO keycloak;
GRANT USAGE ON SCHEMA public TO keycloak;

This is an example of how DB structure should look like:

folio - BE modules DB
keycloak - dedicated DB for KC
kong - dedicated DB for kong

Phase 2 - Management components:

Required deployments:

mgr-applications → README.md, example of k8s deployment || required version
mgr-tenants → README.md, example of k8s deployment || required version
mgr-tenant-entitlements → README.md, example of k8s deployment || required version

Phase 3 - Discovery and Apps registration:

Up to this step, all mgr-* components, Kong, Keycloak, Kafka, Vault etc. should be deployed, verified and tested.

*** In the example, port 8082 referencing Sidecar’s port, depending on your setup port number MUST BE replaced with appropriate value.

**** Each Backend module MUST have 2nd container (sidecar) aka multi-container pod(s).

**** Edge modules DO NOT require sidecar container, hence deployed without it.

Sample discovery for 1 edge and 1 BE modules:

[
  {
      "location": "http://edge-sip2", <-- Notice that edge-* modules DO NOT HAVE sidecar's port specified
      "id": "edge-sip2-3.5.0",
      "name": "edge-sip2",
      "version": "3.5.0"
  },
  {
      "location": "http://mod-agreements:8082", <-- Notice that ALL Backend modules must be pointed to Sidecar's port, in this case: port number 8082
      "id": "mod-agreements-7.3.0",
      "name": "mod-agreements",
      "version": "7.3.0"
  }
]

All required modules versions are available inside apps, you can download all applications specified in: platform-lsp/install-applications.json at Ramsons-release · folio-org/platform-lsp via links below (save all files, they will be needed later on):

app-platform-minimal-1.0.39
app-erm-usage-1.0.3
app-reading-room-1.0.0 (currently not available) | related ticket: [MODRR-35] release re-genarate required - FOLIO Jira
app-dcb-1.0.10
app-platform-complete-1.1.70
app-edge-complete-1.0.8
app-edge-locate-1.0.3
app-consortia-manager-1.0.0
app-requests-mediated-1.1.9
app-consortia-1.1.3
app-requests-mediated-ui-1.0.3
app-requests-ecs-1.0.13

For discovery preparation you may perform building it manually or use below simple bash script.

Steps to follow inside Linux shell (jq utility MUST BE already installed, if not please install it):

mkdir -p /tmp/discovery
cd /tmp/discovery && touch test.sh (open test.sh file and paste below script, then save and exit, it could be done via VIM, Nano or any other Linux file editor)
chmod +x test.sh
download all application-descriptors inside the same directory /tmp/discovery
execute script ./test.sh (save this script, it will be required for future release upgrades, new installations and so on)

#!/bin/bash

OUTPUT_FILE="discovery.json"

TMP_MODULES=$(mktemp)

echo "[]" > "$TMP_MODULES"

for file in *.json; do
  if [ -f "$file" ]; then
    jq '.modules' "$file" | jq -s 'add' >> "$TMP_MODULES.tmp"
    jq -s 'add' "$TMP_MODULES" "$TMP_MODULES.tmp" > "$TMP_MODULES.merged"
    mv "$TMP_MODULES.merged" "$TMP_MODULES"
    rm "$TMP_MODULES.tmp"
  fi
done

jq '{
  discovery: map({
    location: "http://\(.name)\(if (.name | startswith("edge-")) then "" else ":8082" end)",
    id: "\(.name)-\(.version)",
    name: .name,
    version: .version
  })
}' "$TMP_MODULES" > "$OUTPUT_FILE"

echo "Discovery JSON saved to $OUTPUT_FILE"

rm "$TMP_MODULES"

inside the same directory, “discovery.json” file will be placed:

Perform registration of prepared “Discovery”: Example

If by some reason, inside your list of modules, below entities presented, please remove them completely:

*** mod-login, mod-login-saml, mod-authtoken & okapi modules MUST BE EXCLUDED from deployment

EUREKA platform does not require 4 modules listed above, please do not deploy those modules.

Perform all modules deployment via Helm, Script(s) or any other automation which is being used: Example || required Sidecar version

Perform Apps registration (with content of previously downloaded files): Example

Phase 4 - Tenants:

OKAPI tenant:

EUREKA tenant:

As you can see from the pictures above, ID of tenant from OKAPI based env MUST BE equal to NAME of EUREKA tenant. Names MUST BE IDENTICAL with IDs. This is MANDATORY requirement, to preserve existing DB schemas with data.

Create all required tenants, list was taken previously: in Prerequisites steps.

Example how to create tenant: Link

Phase 5 - Entitle:

Using previously downloaded application descriptors, perform entitle operation: Example

Phase 6 - Admin user:

Add user(s): Example

Phase 7 - Users & roles migrations:

Perform users' migration: Users Keycloak API (can take a while, please monitor it, DO NOT START roles migration until completion)
Perform roles migration: Mod Roles Keycloak API (can take a while, please monitor it)
Perform required capabilities and capabilities sets assignment: Example

P\S Once users' migration is complete, they should be visible in Keycloak Admin UI

NOTE: Migration happens ONLY for users with permissions assigned, if user(s) didn’t have permission(s) assigned in previous OKAPI installation, migration for user(s) will be skipped.

Phase 8 - Post-migration activities:

Build UI for NonECS and ECS (central) tenants, if present. Example

Configuration files: location, all required variables MUST BE replaced with values matching migrated environment.

Reindex: the following manual approach can be implemented: Example or simply clone ELK\OS indices: Documentation

Cloning indices much faster operation comparing with standard reindexing.

P\S It’s actively being used on Folio Rancher environments: Example of Groovy code

Phase 9 - Users validation:

Users' migration performs user accounts migration ONLY, meaning credentials(passwords) are not migrating, this means that for each of the migrated users, password should be reset via Keycloak Admin UI:

or any scripted approaches. Best practice is to inform all users upfront about upcoming migration and set their passwords equal to their username, something like: jsmith\jsmith and enforce password change during 1st logon activity. ***NOT RECOMMENDED***

Phase 10 - Hyper care:

As usual after any migration hyper care period should be planned. Users may face any unknown issue(s). Most of the issue(s) will be related to the capabilities and capabilities sets .i.e inability to perform any desired operation(s). This kind of issue(s) always resolving by assigning appropriate roles\capabilities\capabilities sets.

Kubernetes Example Deployment - Folio Development Teams - FOLIO Wiki