Ramsons (R2 2024) Eureka Bugfest environment preparation plan

Owned by Denis
Last updated: about 5 hours ago by Maksym Sinichenkom
5 min read

Pre-Ramsons Eureka Bugfest environments

Standalone tenants (diku):

Non-ECS (folio|folio)

Consortia tenants(ECS)1+10+1:

ECS (ecs_admin|admin)

Ramsons Eureka Bugfest environments

Standalone tenants (diku):

Non-ECS (folio|folio)

Consortia tenants(ECS)1+10+1:

ECS (ecs_admin|admin)

  • tbd

Applications

  1. app-platform-complete: https://github.com/folio-org/app-platform-complete/tree/EUREKA-414

  2. app-platform-minimal: https://github.com/folio-org/app-platform-minimal/tree/EUREKA-414

  3. app-consortia: https://github.com/folio-org/app-consortia/tree/EUREKA-414

  4. app-dcb: https://github.com/folio-org/app-dcb/tree/EUREKA-414

  5. app-edge-complete: https://github.com/folio-org/app-edge-complete/tree/EUREKA-414

  6. app-erm-usage: https://github.com/folio-org/app-erm-usage/tree/EUREKA-414

  7. app-marc-migrations: https://github.com/folio-org/app-marc-migrations

Extra configuration notes

Edge user used by module edge-inventory need to have following permissions to be able to obtain authority data, authority source record data and marc record

data-export.download-record.item.get

inventory-storage.authorities.item.get

inventory-storage.authorities.collection.get

source-storage.records.get

 

Environment preparation plan

Date

Action

Status

Description/Comments

Ticket

Responsible

Notes

Date

Action

Status

Description/Comments

Ticket

Responsible

Notes

1

Oct 26, 2024 - Oct 30, 2024

Prepare pre-Ramsons BF environment (on Quesnelia SP-5)

List of Eureka applications:

app-consortia, app-dcb, app-edge-complete, app-erm-usage, app-platform-complete, app-platform-minimal

 

DONE

Сreate infrastructure from snapshot, adjust cloned db, deploy modules, configure admin token lifespan (set 60 min), copy params(x15), create tenant(x15), enable apps( fs090002, fs090003, load data from repos. OR okapi), user & roles migration(x15), update admin role, update passwords, create UI (x4) (keycloak configuration), create consortia (for all members). Reindex. Smoke test/validation. Revert token lifespan according to documentation. SP5, Quesnelia release branch.

Create admin user with login/password ‘folio/folio’ for stansdalone tenants (should have all capabilities). Create ecs_admin\admin

Note: pre-Bugfest will not have app-erm-usage because of module descriptor validation issue (fixed in Ramsons: link). Impact: while migration of users with erm-usage permissions, the users will not have erm-capabilities. Expect that if in Ramsons Eureka has application 'app-erm-usage', capabilities will be available and FSE Platform team/QA/PO will assign those manually OR some migration will be executed (to be clarified with Eureka dev team)

Done: Consortia tenants - pending the fix: EUREKASUP-8

US1324154

FSE Platform

 

2

Oct 28, 2024

Communication to teams who typically work with the infrastructure components to avoid any changes w/o approval from Kitfox/FSE Platform

DONE

 

NA

FSE Platform

 

3

Oct 30, 2024

Finalize set of Applications for Ramsons Eureka-based Bugfest and a list of modules

DONE

Expectation is that the environment might contain modules that will be targeted for inclusion in FOLIO platform after Ramsons release (as for several modules the work is In Progress)

US13270953270

FSE Platform

 

4

 

Prepare sFTP access to pre-Bugfest env

 

https://bugfest-quesnelia.int.aws.folio.org/settings/invoice/batch-group-configuration similar to BF-792

 

image-20241028-105026.png

 

<>

FSE Platform

 

5

by Oct 30, 2024

Prepare scripts (pipelines) for new modules

DONE

https://github.com/folio-org/mod-circulation-bff

 

US1326339

FSE Platform

 

6

Oct 30, 2024

Notify QA leads, Product Owners that Pre-BF environment is ready for data validation

DONE

By default, after migration to Eureka platform the password of the user is equal to username (case sensitive: ex. JohnDoe / JohnDoe) and need to be reset.

Please note, on the Eureka Ramsons pre-Bugfest environment the users that were migrated from Quesnelia Bugfest, do not have a capability to reset their passwords. This is a known issue regarding capability and it is fixed in Ramsons ( https://folio-org.atlassian.net/browse/MODUSERSKC-52 )

na

Release Manager

 

7

Oct 31, 2024 - Nov 1, 2024

Verify capabilities for existing users

N/A

Denis to work with Artem and Oleg K.

 

QA/PO

 

8

Nov 1, 2024

Review release notes with Kitfox team and discuss deployment approach, branch creation, adding env. variables

DONE

 

US1326337

FSE Platform/Kitfox/FSE Ops

 

9

Oct 25, 2024 -

Create/modify automation to support Ramsons' reindex

in progress

 

US1326353

FSE Platform

 

10

 

Create a data snapshot of Pre-Ramsons set up

DONE

Please inform Kitfox once available

US1328411

FSE Platform

 

11

 

[OPTIONAL] Update Eureka Bugfest wrapper to handle management modules update

 

 

<>

FSE Platform

discussion with Eureka/Kitfox

12

Nov 2, 2024

Prepare master branch for platform-complete + hosting.configuration with Ramsons modules versions

DONE

 

 

Kitfox

 

13

 

non-platform-complete modules

list of modules from Ramsons+

(releases from branches), they will not be a part of regular install.json

in progress

  • mod-requests-mediated

  • folio_requests-mediated

  • mod-tlr

  • mod-circulation-bff

<>

FSE Platform

will they be a part of platform-complete install.json?

Denis - check frequency of releases? Decide deployment approach

14

 

Prepare/Generate/Validate Applications for Ramsons release

Applications:

  1. app-consortia

  2. app-dcb

  3. app-edge-complete

  4. app-erm-usage

  5. app-platform-complete

  6. app-platform-minimal

  7. app-marc-migrations

DONE

 

US1328418

FSE Platform

 

15

Nov 3, 2024

Update existing/Create new BF cluster from DB snapshot with all migrations& run adjust-cloned-db pipeline

DONE

Updated existing cluster US1324154: Create pre-bugfest environment for Ramsons release

N/A

FSE Platform

 

16

 

Required (read release notes more mod-data-export)

DONE

similar to BF-807

 

FSE Platform

 

17

 

EBSCO edge passwords should be the same (done with Kitfox)

DONE

similar to BF-808

 

FSE Platform

 

18

 

DI slicing should be validated

DONE

similar to BF-804

 

FSE Platform

 

19

 

Deploy Ramsons Applications to BF cluster

DONE

 

 

FSE Platform

 

20

 

Reset password links

DONE

similar to BF-781

 

FSE Platform

Check with Dmytro (do we need to set invalid links or not?)

21

Nov 3, 2024 - Nov 6, 2024

Migrate Pre-BF env to Ramsons (+ any manual process from Q to R)

in progress

Ramsons (R2 2024) Required manual migrations

US1328729

FSE Platform

 

22

Entitle Ramsons Applications

DONEfor non-ECS
blocked for ECS

 

 

FSE Platform

blocker: EUREKASUP-11

23

[IF NEW CLUSTER]

Create ECS/non-ECS tenant + MARC MIGRATIONS + run reindex + create UI. [IF OLD CLUSTER] update tenant DNS

N/A

 

 

FSE Platform

create DNS for tenants, not tenants themself

24

[IF NEW CLUSTER]

Add the default roles and users

N/A

 

 

FSE Platform

 

25

mod-entities-links migration

Done

Ramsons (R2 2024) Required manual migrations

 

 

 

26

LCCN migrations

DONE

Script to update tenant mapping rules to include mapping for Canceled LCCN, Date type, Subject Source and Subject Type

Instruction: Login into i-0231a7b0d96d4f8ba instance, devQa, us-east-1, navigate inside “/home/ec2-user“. Here you will find jar file, instructions.txt file and bunch of config files for different tenants.

aqa - fs09000003 - done
image-20241105-171105.png
plus - fs09000002 - done
image-20241105-171241.png

fs09000000 - done
image-20241105-171348.png

 

FSE Platform

Quick migration

27

mod-inventory-storage migration

Done for non-ecs

validated for non-ecs

in progress in progress for ECS

Instruction here
Validation:

SELECT 1 FROM <tenant_id>_mod_inventory_storage.instance WHERE jsonb -> 'publicationPeriod' IS NOT NULL LIMIT 1;

 

FSE Platform

long migration

28

Execute marc-migrations

will be performed later - dev team not ready

mod-marc-migrations will be used more like external tool now. For Okapi-based deployments, this module can be deployed in the usual way, with no additional changes needed. 

For Eureka-based deployments, the Spitfire team has created an application to facilitate module deployment:

https://github.com/folio-org/app-marc-migrations

In the R release, default mapping rules have changed, and remapping is now required for inventory authorities and instances. 

 

FSE Platform

 

29

Roles migration

DONEfor non-ECS

in progress for ECS

 

 

FSE Platform

q

30

UI creation

DONE for non-ECS
to do for ECS

 

 

 

FSE Platform

q

31

Verify (update admin) capabilities for existing users.

DONE for non-ECS
to do for ECS

 

 

FSE Platform

q

32

Perform Instance reindex

DONE for non-ECS
to do for ECS

jenkins job

 

FSE Platform

 

33

Perform authority + location reindex

DONE for non-ECS
to do for ECS

jenkins job

 

FSE Platform

l

34

Validate environment (some testing)

DONE for non-ECS
to do for ECS

 

 

FSE Platform

 

35

Disable deprecated permissions.

N/A

similar to BF-798

 

FSE Platform

 

36

SMTP - check that emails are working

DONEfor non-ECS

similar to BF-796

 

FSE Platform

 

37

DOCUMENTATION

IN PROGRESS

similar to BF-780

 

 

 

38

Deployment to Ramsons BF env is competed

 

 

 

FSE Platform

 

39

 

[JUST CHECK]

Verify that modules configuration is updated in FSE branch. Update app versions.

DONE

 

 

FSE Platform/Kitfox

 

40

 

Update tenant DNS

DONE for non-ECS
to do for ECS

 

 

 

 

41

 

Data set is ready. Create Ramsons DB snapshot

to do

 

 

FSE Platform

 

42

 

Prepare and share links to Ramsons Eureka BF logs

 

Check with Dmytro

How to get logs from Q bugfest environment (Logs are updated every 15 minutes)Couple of ways to access logs:

<>

 

FSE Platform

 

43

evening Nov 6, 2024 - morning Nov 7, 2024

Notify POs/QAs that Eureka Ramsons BF is up and running, and available for sanity testing

 

 

 

Release Manager

 

44

Nov 7, 2024 - Nov 8, 2024

POs check the Bugfest env

 

 

 

POs

 

45

Nov 7, 2024 - Nov 8, 2024

Smoke test - Manual & Automated

 

Results:

 

@Oleg Kurdyayev

 

46

Nov 8, 2024

High Availability verification

 

Verify new modules:

mod-reading-room
mod-record-specifications
ui-reading-room

 

Dev Teams

 

47

by Nov 22, 2024

DB Schema comparison

 

Denis to work with FSE Ops

US1327373

FSE Operations

 

48

by Nov 22, 2024

Re-index time measurement

 

Denis to work with FSE Ops

US1327378

FSE Operations

 

49

by Nov 22, 2024

Migration from Quesnelia to Ramsons time measurement 

 

Denis to work with FSE Ops

US1327378

FSE Operations

 

50

 

Daily updates (morning/evening) of BF env during bugfest and bugfix periods

 

 

<>

FSE Platform

 

51

 

Troubleshoot Eureka Ramsons BF environment issues (review issues from https://issues.folio.org/projects/BF project with label ‘eureka-bugfest_R2.2024’)

 

 

<>

FSE Platform

TBD

52

 

Maintain slack notifications in following channels: #eureka-bugfest-notify

 

 

<>

FSE Platform

TBD