Ramsons (R2 2024) Eureka Bugfest environment preparation plan
- Denis
- Maksym Sinichenkom
- Artem Akimov
Pre-Ramsons Eureka Bugfest environments
Standalone tenants (diku):
Non-ECS (folio|folio)
bugfest - https://pre-eureka-bugfest-ramsons.int.aws.folio.org/
bugfest-aqa - https://pre-eureka-bugfest-ramsons-aqa.int.aws.folio.org/
bugfest-plus - https://pre-eureka-bugfest-ramsons-plus.int.aws.folio.org/
Consortia tenants(ECS)1+10+1:
ECS (ecs_admin|admin)
Ramsons Eureka Bugfest environments
Standalone tenants (diku):
Non-ECS (folio|folio)
bugfest-aqa - https://eureka-bugfest-ramsons-aqa.int.aws.folio.org/
bugfest-plus - https://eureka-bugfest-ramsons-plus.int.aws.folio.org/
Consortia tenants(ECS)1+10+1:
ECS (ecs_admin|admin)
Updates schedule
Bugfest on Eureka is updated two times per day (in case there are new releases from the teams)
first update: 3:00 am Eastern time / 7:00 UTC
second update: 11:00 am Eastern time / 15:00 UTC
Notifications on the updates/work in progress will be posted in #eureka-bugfest-notify channel in Slack
Logs
Couple of ways to access logs:
aws s3 ls s3://int-folio-bugfest-logs/ramsons-eureka/ --no-sign-request --recursive --human-readable --summarize
If you want to use the UI, then go to https://int-folio-bugfest-logs.s3.amazonaws.com/ and search for the module based on timestamp and attach the <key> to the URL - You can download logs that way but there is a max limit of 1000 objects that get displayed from UI
Applications
app-platform-complete: https://github.com/folio-org/app-platform-complete/tree/EUREKA-414
app-platform-minimal: https://github.com/folio-org/app-platform-minimal/tree/EUREKA-414
app-consortia: https://github.com/folio-org/app-consortia/tree/EUREKA-414
app-dcb: https://github.com/folio-org/app-dcb/tree/EUREKA-414
app-edge-complete: https://github.com/folio-org/app-edge-complete/tree/EUREKA-414
app-erm-usage: https://github.com/folio-org/app-erm-usage/tree/EUREKA-414
app-marc-migrations: GitHub - folio-org/app-marc-migrations: Application descriptor repository for app-marc-migrations.
Extra configuration notes
Edge user used by module edge-inventory need to have following permissions to be able to obtain authority data, authority source record data and marc record
data-export.download-record.item.get
inventory-storage.authorities.item.get
inventory-storage.authorities.collection.get
source-storage.records.get
Environment preparation plan
Date | Action | Status | Description/Comments | Ticket | Responsible | Notes | |
|---|---|---|---|---|---|---|---|
| 1 | Oct 26, 2024 - Oct 30, 2024 | Prepare pre-Ramsons BF environment (on Quesnelia SP-5) List of Eureka applications: app-consortia, app-dcb, app-edge-complete, app-erm-usage, app-platform-complete, app-platform-minimal
| DONE | Сreate infrastructure from snapshot, adjust cloned db, deploy modules, configure admin token lifespan (set 60 min), copy params(x15), create tenant(x15), enable apps( fs090002, fs090003, load data from repos. OR okapi), user & roles migration(x15), update admin role, update passwords, create UI (x4) (keycloak configuration), create consortia (for all members). Reindex. Smoke test/validation. Revert token lifespan according to documentation. SP5, Quesnelia release branch. Create admin user with login/password ‘folio/folio’ for stansdalone tenants (should have all capabilities). Create ecs_admin\admin Note: pre-Bugfest will not have app-erm-usage because of module descriptor validation issue (fixed in Ramsons: link). Impact: while migration of users with erm-usage permissions, the users will not have erm-capabilities. Expect that if in Ramsons Eureka has application 'app-erm-usage', capabilities will be available and FSE Platform team/QA/PO will assign those manually OR some migration will be executed (to be clarified with Eureka dev team) Done: Consortia tenants - pending the fix: EUREKASUP-8 | FSE Platform |
| |
| 2 | Oct 28, 2024 | Communication to teams who typically work with the infrastructure components to avoid any changes w/o approval from Kitfox/FSE Platform | DONE |
| NA | FSE Platform |
|
| 3 | Oct 30, 2024 | Finalize set of Applications for Ramsons Eureka-based Bugfest and a list of modules | DONE | Expectation is that the environment might contain modules that will be targeted for inclusion in FOLIO platform after Ramsons release (as for several modules the work is In Progress) | US13270953270 | FSE Platform |
|
| 4 |
| Prepare sFTP access to pre-Bugfest env |
| https://bugfest-quesnelia.int.aws.folio.org/settings/invoice/batch-group-configuration similar to BF-792
| <> | FSE Platform |
|
| 5 | by Oct 30, 2024 | Prepare scripts (pipelines) for new modules | DONE | GitHub - folio-org/mod-circulation-bff: Circulation backend-for-frontend module
| FSE Platform |
| |
| 6 | Oct 30, 2024 | Notify QA leads, Product Owners that Pre-BF environment is ready for data validation | DONE | By default, after migration to Eureka platform the password of the user is equal to username (case sensitive: ex. JohnDoe / JohnDoe) and need to be reset. Please note, on the Eureka Ramsons pre-Bugfest environment the users that were migrated from Quesnelia Bugfest, do not have a capability to reset their passwords. This is a known issue regarding capability and it is fixed in Ramsons ( MODUSERSKC-52: _self endpoint doesn't return permissions associated with capabilitySetsClosed ) | na | Release Manager |
|
| 7 | Oct 31, 2024 - Nov 1, 2024 | Verify capabilities for existing users | N/A | Denis to work with Artem and Oleg K. |
| QA/PO |
|
| 8 | Nov 1, 2024 | Review release notes with Kitfox team and discuss deployment approach, branch creation, adding env. variables | DONE |
| FSE Platform/Kitfox/FSE Ops |
| |
| 9 | Oct 25, 2024 - | Create/modify automation to support Ramsons' reindex | DOne |
| FSE Platform |
| |
| 10 |
| Create a data snapshot of Pre-Ramsons set up | DONE | Please inform Kitfox once available | FSE Platform |
| |
| 11 |
| [OPTIONAL] Update Eureka Bugfest wrapper to handle management modules update | DONE |
| <> | FSE Platform | discussion with Eureka/Kitfox |
| 12 | Nov 2, 2024 | Prepare master branch for platform-complete + hosting.configuration with Ramsons modules versions | DONE |
|
| Kitfox |
|
| 13 |
| non-platform-complete modules list of modules from Ramsons+ (releases from branches), they will not be a part of regular install.json | done |
| <> | FSE Platform | will they be a part of platform-complete install.json? Denis - check frequency of releases? Decide deployment approach |
| 14 |
| Prepare/Generate/Validate Applications for Ramsons release Applications:
| DONE |
| FSE Platform |
| |
| 15 | Nov 3, 2024 | Update existing/Create new BF cluster from DB snapshot with all migrations& run adjust-cloned-db pipeline | DONE | Updated existing cluster US1324154: Create pre-bugfest environment for Ramsons release | N/A | FSE Platform |
|
| 16 |
| Required (read release notes more mod-data-export) | DONE | similar to BF-807 |
| FSE Platform |
|
| 17 |
| EBSCO edge passwords should be the same (done with Kitfox) | DONE | similar to BF-808 |
| FSE Platform |
|
| 18 |
| DI slicing should be validated | DONE | similar to BF-804 |
| FSE Platform |
|
| 19 |
| Deploy Ramsons Applications to BF cluster | DONE |
|
| FSE Platform |
|
| 20 |
| Reset password links | DONE | similar to BF-781 |
| FSE Platform | Check with Dmytro (do we need to set invalid links or not?) |
| 21 | Nov 3, 2024 - Nov 6, 2024 | Migrate Pre-BF env to Ramsons (+ any manual process from Q to R) | DONE | FSE Platform |
| ||
| 22 | Entitle Ramsons Applications | DONEfor non-ECS |
|
| FSE Platform | blocker: EUREKASUP-11 | |
| 23 | [IF NEW CLUSTER] Create ECS/non-ECS tenant + MARC MIGRATIONS + run reindex + create UI. [IF OLD CLUSTER] update tenant DNS | N/A |
|
| FSE Platform | create DNS for tenants, not tenants themself | |
| 24 | [IF NEW CLUSTER] Add the default roles and users | N/A |
|
| FSE Platform |
| |
| 25 | mod-entities-links migration | Done Done for ECS | Ramsons (R2 2024) Required manual migrations - script removed from page. used prev version SET search_path TO <tenantId>_mod_entities_links;
-- Find deleted records in authority table; index avoids two full table scans
CREATE INDEX IF NOT EXISTS authority_deleted_idx ON authority(deleted) WHERE deleted;
-- Move to archive
INSERT INTO authority_archive SELECT * FROM authority WHERE deleted = true ON CONFLICT DO NOTHING;
-- Delete deleted
DELETE FROM authority WHERE deleted = true;
-- Drop index
DROP INDEX authority_deleted_idx; |
|
|
| |
| 26 | LCCN migrations | DONE for non-ecs DONE for ECS | Instruction: Login into i-0231a7b0d96d4f8ba instance, devQa, us-east-1, navigate inside “/home/ec2-user“. Here you will find jar file, instructions.txt file and bunch of config files for different tenants. aqa - fs09000003 - done fs09000000 - done |
| FSE Platform | Quick migration | |
| 27 | mod-inventory-storage migration | Done for non-ecs validated for non-ecs Done for ECS | Instruction here SELECT 1 FROM <tenant_id>_mod_inventory_storage.instance WHERE jsonb -> 'publicationPeriod' IS NOT NULL LIMIT 1; |
| FSE Platform | long migration | |
| 28 | Execute marc-migrations | will be performed later - dev team not ready | mod-marc-migrations will be used more like external tool now. For Okapi-based deployments, this module can be deployed in the usual way, with no additional changes needed. For Eureka-based deployments, the Spitfire team has created an application to facilitate module deployment: GitHub - folio-org/app-marc-migrations: Application descriptor repository for app-marc-migrations. In the R release, default mapping rules have changed, and remapping is now required for inventory authorities and instances. |
| FSE Platform | long migration | |
| 29 | Roles migration | DONEfor non-ECS DONE for ECS |
|
| FSE Platform | q | |
| 30 | UI creation | DONE for non-ECS
|
|
| FSE Platform | q | |
| 31 | Verify (update admin) capabilities for existing users. | DONE for non-ECS |
|
| FSE Platform | q | |
| 32 | Perform Instance reindex | DONE for non-ECS | jenkins job |
| FSE Platform |
| |
| 33 | Perform authority + location reindex | DONE for non-ECS | jenkins job |
| FSE Platform | l | |
| 34 | Validate environment (some testing) | DONE for non-ECS | used basic autotest |
| FSE Platform |
| |
| 35 | Disable deprecated permissions. | N/A | similar to BF-798 |
| FSE Platform |
| |
| 36 | SMTP - check that emails are working | DONEfor non-ECS | similar to BF-796 |
| FSE Platform |
| |
| 37 | DOCUMENTATION | IN PROGRESS | similar to BF-780 |
|
|
| |
| 38 | Deployment to Ramsons BF env is competed | done |
|
| FSE Platform |
| |
| 39 |
| [JUST CHECK] Verify that modules configuration is updated in FSE branch. Update app versions. | DONE |
|
| FSE Platform/Kitfox |
|
| 40 |
| Update tenant DNS | DONE for non-ECS |
|
| FSE Platform |
|
| 41 |
| Load reference/sample data to all tenants | DONEfor non-ECS |
|
| FSE Platform |
|
| 42 |
| Data set is ready. Create Ramsons DB snapshot | done |
|
| FSE Platform |
|
| 43 |
| Prepare and share links to Ramsons Eureka BF logs | to do | Check with Dmytro How to get logs from Q bugfest environment (Logs are updated every 15 minutes)Couple of ways to access logs: <> |
| FSE Platform |
|
| 44 | evening Nov 6, 2024 - morning Nov 7, 2024 | Notify POs/QAs that Eureka Ramsons BF is up and running, and available for sanity testing | unknown |
|
| Release Manager |
|
| 45 | Nov 7, 2024 - Nov 8, 2024 | POs check the Bugfest env |
|
|
| POs |
|
| 46 | Nov 7, 2024 - Nov 8, 2024 | Smoke test - Manual & Automated |
| Results: |
| @Oleg Kurdyayev |
|
| 47 | Nov 8, 2024 | High Availability verification |
| Verify new modules: mod-reading-room mod-record-specifications ui-reading-room |
| Dev Teams |
|
| 48 | by Nov 22, 2024 | DB Schema comparison |
| Denis to work with FSE Ops | FSE Operations |
| |
| 49 | by Nov 22, 2024 | Re-index time measurement |
| Denis to work with FSE Ops | FSE Operations |
| |
| 50 | by Nov 22, 2024 | Migration from Quesnelia to Ramsons time measurement |
| Denis to work with FSE Ops | FSE Operations |
| |
| 51 |
| Daily updates (morning/evening) of BF env during bugfest and bugfix periods |
|
| <> | FSE Platform |
|
| 52 |
| Troubleshoot Eureka Ramsons BF environment issues (review issues from https://issues.folio.org/projects/BF project with label ‘eureka-bugfest_R2.2024’) |
|
| <> | FSE Platform | TBD |
| 53 |
| Maintain slack notifications in following channels: #eureka-bugfest-notify |
|
| <> | FSE Platform | TBD |
