Quesnelia (R1 2024) - Bugfest env preparation - Issues/preparation we needed to resolve on environment

General recomendations and platform-complete Issues

Issue 1: edge-oai-pmh

Need to create clean environment with new unik edge keys for all tenants. Faced a lot of essues related to edge keys.
https://folio-org.atlassian.net/browse/BF-682
https://folio-org.atlassian.net/browse/BF-732
https://folio-org.atlassian.net/browse/BF-707

Issue 2: Cloudfront and ALB configuration
While creating new ALB for Quesnelia environment faced with AWS limit of ALB creation on FSE AWS account.
Had to extend limit by contacting AWS support

Issue 3: Tenant creation parameters:
For each tenat creation use:

tenantParameters=loadReference%3dtrue%2cloadSample%3dfalse

Review of Release Notes

We reviewed the release notes for the Poppy release and adjusted our configs. Here are the changes made to specific modules:

mod-consortia
Set metaspace size
{ "name": "JAVA_OPTS", "value": "-XX:MetaspaceSize=512m -XX:MaxMetaspaceSize=1024m -Xmx2048m" }
mod-agreements:
Correct S3 path
{ "name": "AWS_URL", "value": "https://s3.amazonaws.com" }
mod-data-export:
Chsnge all AWS related variables from AWS_* to S3_*

{ "name": "S3_REGION", "value": "$REGION" }, { "name": "S3_BUCKET", "value": "data-export-$DOMAIN_PREFIX-$REGION-$ACCOUNT_NAME-tenant" }, { "name": "S3_URL", "value": "https://s3.amazonaws.com" }, { "name": "S3_IS_AWS", "value": "true" },

Deployment of Modules

We encountered various issues during module deployment. Here are the details and their resolutions:

mod-data-import:
Tables overfulled with data
https://folio-org.atlassian.net/browse/BF-735
Resolution
SELECT id, jsonb, creation_date, created_by
    FROM fs09000000_mod_data_import.upload_definitions
    WHERE jsonb->> 'status' :: text in ('NEW', 'IN_PROGRESS', 'LOADED')
    ORDER BY creation_date DESC;
mod-service-interaction, mod-serials-management, mod-agreements:
This resolution aplicable for mod-service-interaction, mod-serials-management, mod-agreements issues
Delete next data from federation_lock table

95d8d734-b027-421f-96cc-baa4b21ea8d1	tenant:cs00000int_0009	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
b73805df-aef6-49ce-ac99-4ec4c3a36f18	tenant:cs00000int_0010	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
6446e4be-020f-43b7-a5ba-c6cea25d536c	tenant:cs00000int_0011	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
b2d01573-cad8-4245-ba76-bf3d567bacfa	tenant:cs00000int	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
e7c2f601-ffe9-4653-92aa-02f71cc8f9bb	tenant:fs09000000	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
c6cde6dc-3d6b-49aa-9974-2c351395158b	tenant:seeding	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
2f5cb100-c7c3-434d-9516-8cc89d5e7b0d	tenant:cs00000int_0001	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
e298b1f8-076f-4b41-8ec6-4cf79226ece2	tenant:cs00000int_0002	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
e49f09b8-f19f-44a8-ab71-0ac3a71fced6	tenant:cs00000int_0003	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
967d56cc-3dde-485c-8309-7841798ae079	tenant:cs00000int_0004	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
0c6255db-6f25-4b82-bef4-1dcc0ce37f4b	tenant:cs00000int_0005	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
0837cbcf-49d2-405c-a0b1-70243580abd0	tenant:cs00000int_0006	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
4cd78b89-0bdb-4c7e-9829-a006cddb53c1	tenant:cs00000int_0007	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7
28e35002-6279-4a2d-9a2c-49219437d9c8	tenant:cs00000int_0008	7.0.4	f59c6d1a-5133-4572-8f87-c96c2e3b00a7

Wait 5-10 minutes for mod-agreements to setup after deployment

mod-consortia: Excluded deployment for non-consortia tenants.

Other Issues

Socket closed for ”eHoldins”
https://folio-org.atlassian.net/browse/BF-692
Resolution:
Fixed by Bienenvolk team in https://folio-org.atlassian.net/browse/ERM-3190
Missed instance-relationship-types reference data ECS tenants
Resolution:
https://folio-org.atlassian.net/browse/BF-688
https://folio-org.atlassian.net/browse/MODINVSTOR-1138
Unable to unlink authority from MARC bib's field
Resolution:
https://folio-org.atlassian.net/browse/BF-672
Kafka consolidation
Was done to reduce the number of topics load on the Kafka cluster.
1) List of kafka modules

[ { "id": "mod-audit", "action": "enable" }, { "id": "mod-entities-link", "action": "enable" }, { "id": "mod-orders-storage", "action": "enable" }, { "id": "mod-invoice", "action": "enable" }, { "id": "mod-data-export-spring", "action": "enable" }, { "id": "mod-quick-marc", "action": "enable" }, { "id": "mod-inventory", "action": "enable" }, { "id": "mod-search", "action": "enable" }, { "id": "mod-data-import", "action": "enable" }, { "id": "mod-remote-storage", "action": "enable" }, { "id": "mod-pubsub", "action": "enable" }, { "id": "mod-data-export-worker", "action": "enable" }, { "id": "mod-source-record-manager", "action": "enable" }, { "id": "mod-inventory-storage", "action": "enable" }, { "id": "mod-source-record-storage", "action": "enable" }, { "id": "mod-circulation", "action": "enable" }, { "id": "mod-invoice", "action": "enable" }, { "id": "mod-dcb", "action": "enable" }, { "id": "mod-consortia", "action": "enable" }, { "id": "mod-users", "action": "enable" } ]

2) Added variables to each kafaka related module.

{ "name": "KAFKA_PRODUCER_TENANT_COLLECTION", "value": "ALL" }, { "name": "ENV", "value": "$CLUSTER" },
RTR Implementation
RTR wa enabled on Quesnelia environment
https://folio-org.atlassian.net/browse/BF-664

Quesnelia environment general info

Tenant	Name	URL

Tenant	Name	URL
fs09000000	Integration	https://bugfest-quesnelia.int.aws.folio.org/
fs09000002	Plus	https://bugfest-quesnelia-plus.int.aws.folio.org/
fs09000003	AQA	https://bugfest-quesnelia-aqa.int.aws.folio.org/
cs00000int	Consortium	https://bugfest-quesnelia-consortium.int.aws.folio.org/

Modules list for NON ECS tenants

[
    {
        "id": "edge-caiasoft-2.2.2"
    },
    {
        "id": "edge-connexion-1.3.0"
    },
    {
        "id": "edge-courses-1.4.1"
    },
    {
        "id": "edge-dcb-1.1.2"
    },
    {
        "id": "edge-dematic-2.2.2"
    },
    {
        "id": "edge-fqm-2.0.1"
    },
    {
        "id": "edge-ncip-1.10.0"
    },
    {
        "id": "edge-oai-pmh-2.9.1"
    },
    {
        "id": "edge-orders-3.0.2"
    },
    {
        "id": "edge-patron-5.1.1"
    },
    {
        "id": "edge-rtac-2.7.2"
    },
    {
        "id": "edge-sip2-3.2.4"
    },
    {
        "id": "folio_acquisition-units-5.1.0"
    },
    {
        "id": "folio_agreements-11.0.2"
    },
    {
        "id": "folio_bulk-edit-4.1.4"
    },
    {
        "id": "folio_calendar-11.0.1"
    },
    {
        "id": "folio_checkin-9.1.1"
    },
    {
        "id": "folio_checkout-10.1.0"
    },
    {
        "id": "folio_circulation-9.1.0"
    },
    {
        "id": "folio_circulation-log-4.1.0"
    },
    {
        "id": "folio_courses-6.0.3"
    },
    {
        "id": "folio_dashboard-6.0.0"
    },
    {
        "id": "folio_data-export-6.1.4"
    },
    {
        "id": "folio_data-import-7.1.8"
    },
    {
        "id": "folio_developer-8.0.1"
    },
    {
        "id": "folio_eholdings-9.1.1"
    },
    {
        "id": "folio_erm-comparisons-7.0.0"
    },
    {
        "id": "folio_erm-usage-9.0.0"
    },
    {
        "id": "folio_export-manager-3.1.2"
    },
    {
        "id": "folio_finance-6.0.0"
    },
    {
        "id": "folio_gobi-settings-3.1.0"
    },
    {
        "id": "folio_handler-stripes-registry-2.2.0"
    },
    {
        "id": "folio_inventory-11.0.4"
    },
    {
        "id": "folio_invoice-6.0.3"
    },
    {
        "id": "folio_licenses-11.0.1"
    },
    {
        "id": "folio_lists-3.0.1"
    },
    {
        "id": "folio_local-kb-admin-8.0.1"
    },
    {
        "id": "folio_marc-authorities-5.0.1"
    },
    {
        "id": "folio_myprofile-9.1.0"
    },
    {
        "id": "folio_notes-9.1.0"
    },
    {
        "id": "folio_orders-6.0.4"
    },
    {
        "id": "folio_organizations-5.1.1"
    },
    {
        "id": "folio_plugin-bursar-export-4.0.2"
    },
    {
        "id": "folio_plugin-create-inventory-records-4.1.0"
    },
    {
        "id": "folio_plugin-eusage-reports-3.1.0"
    },
    {
        "id": "folio_plugin-find-agreement-11.0.1"
    },
    {
        "id": "folio_plugin-find-authority-4.0.0"
    },
    {
        "id": "folio_plugin-find-contact-5.1.0"
    },
    {
        "id": "folio_plugin-find-eresource-7.0.1"
    },
    {
        "id": "folio_plugin-find-erm-usage-data-provider-6.1.0"
    },
    {
        "id": "folio_plugin-find-fund-3.1.0"
    },
    {
        "id": "folio_plugin-find-import-profile-7.1.0"
    },
    {
        "id": "folio_plugin-find-instance-7.1.1"
    },
    {
        "id": "folio_plugin-find-interface-5.1.0"
    },
    {
        "id": "folio_plugin-find-license-11.0.1"
    },
    {
        "id": "folio_plugin-find-organization-5.1.1"
    },
    {
        "id": "folio_plugin-find-package-title-6.1.0"
    },
    {
        "id": "folio_plugin-find-po-line-5.1.1"
    },
    {
        "id": "folio_plugin-find-user-7.1.1"
    },
    {
        "id": "folio_plugin-query-builder-1.1.5"
    },
    {
        "id": "folio_quick-marc-8.0.1"
    },
    {
        "id": "folio_receiving-5.0.4"
    },
    {
        "id": "folio_remote-storage-5.1.0"
    },
    {
        "id": "folio_requests-9.1.1"
    },
    {
        "id": "folio_serials-management-1.0.4"
    },
    {
        "id": "folio_service-interaction-3.0.1"
    },
    {
        "id": "folio_servicepoints-7.1.0"
    },
    {
        "id": "folio_stripes-authority-components-4.0.1"
    },
    {
        "id": "folio_stripes-core-10.1.1"
    },
    {
        "id": "folio_stripes-marc-components-1.0.1"
    },
    {
        "id": "folio_stripes-smart-components-9.1.3"
    },
    {
        "id": "folio_tags-8.1.0"
    },
    {
        "id": "folio_tenant-settings-8.1.0"
    },
    {
        "id": "folio_users-10.1.1"
    },
    {
        "id": "mod-agreements-7.0.5"
    },
    {
        "id": "mod-audit-2.9.0"
    },
    {
        "id": "mod-authtoken-2.15.1"
    },
    {
        "id": "mod-batch-print-1.1.0"
    },
    {
        "id": "mod-bulk-operations-2.0.2"
    },
    {
        "id": "mod-calendar-3.1.0"
    },
    {
        "id": "mod-circulation-24.2.5"
    },
    {
        "id": "mod-circulation-item-1.0.0"
    },
    {
        "id": "mod-circulation-storage-17.2.1"
    },
    {
        "id": "mod-configuration-5.10.0"
    },
    {
        "id": "mod-copycat-1.6.0"
    },
    {
        "id": "mod-courses-1.4.10"
    },
    {
        "id": "mod-data-export-5.0.4"
    },
    {
        "id": "mod-data-export-spring-3.2.2"
    },
    {
        "id": "mod-data-export-worker-3.2.4"
    },
    {
        "id": "mod-data-import-3.1.1"
    },
    {
        "id": "mod-dcb-1.1.0"
    },
    {
        "id": "mod-di-converter-storage-2.2.2"
    },
    {
        "id": "mod-ebsconet-2.2.0"
    },
    {
        "id": "mod-email-1.17.0"
    },
    {
        "id": "mod-entities-links-3.0.1"
    },
    {
        "id": "mod-erm-usage-4.8.0"
    },
    {
        "id": "mod-erm-usage-harvester-4.5.0"
    },
    {
        "id": "mod-eusage-reports-2.1.1"
    },
    {
        "id": "mod-event-config-2.7.1"
    },
    {
        "id": "mod-feesfines-19.1.0"
    },
    {
        "id": "mod-finance-4.9.0"
    },
    {
        "id": "mod-finance-storage-8.6.1"
    },
    {
        "id": "mod-fqm-manager-2.0.4"
    },
    {
        "id": "mod-gobi-2.8.1"
    },
    {
        "id": "mod-graphql-1.12.1"
    },
    {
        "id": "mod-inventory-20.2.6"
    },
    {
        "id": "mod-inventory-storage-27.1.3"
    },
    {
        "id": "mod-inventory-update-3.3.1"
    },
    {
        "id": "mod-invoice-5.8.2"
    },
    {
        "id": "mod-invoice-storage-5.8.1"
    },
    {
        "id": "mod-kb-ebsco-java-4.0.0"
    },
    {
        "id": "mod-licenses-6.0.2"
    },
    {
        "id": "mod-lists-2.0.6"
    },
    {
        "id": "mod-login-7.11.1"
    },
    {
        "id": "mod-login-saml-2.8.1"
    },
    {
        "id": "mod-ncip-1.14.5"
    },
    {
        "id": "mod-notes-5.2.0"
    },
    {
        "id": "mod-notify-3.2.0"
    },
    {
        "id": "mod-oai-pmh-3.13.2"
    },
    {
        "id": "mod-orders-12.8.8"
    },
    {
        "id": "mod-orders-storage-13.7.2"
    },
    {
        "id": "mod-organizations-1.9.2"
    },
    {
        "id": "mod-organizations-storage-4.7.0"
    },
    {
        "id": "mod-password-validator-3.2.0"
    },
    {
        "id": "mod-patron-6.1.0"
    },
    {
        "id": "mod-patron-blocks-1.10.0"
    },
    {
        "id": "mod-permissions-6.5.0"
    },
    {
        "id": "mod-pubsub-2.13.1"
    },
    {
        "id": "mod-quick-marc-5.1.1"
    },
    {
        "id": "mod-remote-storage-3.2.0"
    },
    {
        "id": "mod-rtac-3.6.0"
    },
    {
        "id": "mod-search-3.2.6"
    },
    {
        "id": "mod-sender-1.12.0"
    },
    {
        "id": "mod-serials-management-1.0.3"
    },
    {
        "id": "mod-service-interaction-4.0.2"
    },
    {
        "id": "mod-settings-1.0.3"
    },
    {
        "id": "mod-source-record-manager-3.8.5"
    },
    {
        "id": "mod-source-record-storage-5.8.5"
    },
    {
        "id": "mod-tags-2.2.0"
    },
    {
        "id": "mod-template-engine-1.20.0"
    },
    {
        "id": "mod-user-import-3.8.0"
    },
    {
        "id": "mod-users-19.3.1"
    },
    {
        "id": "mod-users-bl-7.7.3"
    },
    {
        "id": "okapi-5.3.0"
    }
]