Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

mod-source-record-manager unable to create Kafka topics if tenant ID doesn't match \w{4,}

Description

NOTE: Extent of affected libraries is not clear. Currently 2 Index Data clients, no EBSCO or ByWater clients; unclear if any other hosted or on prem libraries

This seems to mean that tenant IDs with less than 4 consecutive word characters will not work with data import.

To replicate using a recent folio/snapshot Vagrant box:

  1. Create a tenant with an ID of 4 or more characters (e.g. test)

  2. Enable the same set of modules as used by the diku tenant in the Vagrant VM

  3. Create an admin user (e.g. using the bootstrap-superuser.pl script)

  4. Create a UI instance for interacting with the backend (e.g. by cloning platform-complete and running yarn start --tenant test --port 3001)

  5. Import a simple MARC file using the default profile. It should work.

Then:

  1. Create a tenant with an ID of less than 4 characters (e.g. tst)

  2. Enable modules and create admin user and UI instance as above

  3. Import a simple MARC file using the default profile. It will fail silently.

kafka-topics.sh --list shows:

folio.Default.diku.DI_COMPLETED folio.Default.diku.DI_INVENTORY_INSTANCE_CREATED_READY_FOR_POST_PROCESSING folio.Default.diku.DI_PARSED_RECORDS_CHUNK_SAVED folio.Default.diku.DI_RAW_RECORDS_CHUNK_PARSED folio.Default.diku.DI_RAW_RECORDS_CHUNK_READ folio.Default.diku.DI_SRS_MARC_BIB_INSTANCE_HRID_SET folio.Default.diku.DI_SRS_MARC_BIB_RECORD_CREATED folio.Default.test.DI_COMPLETED folio.Default.test.DI_INVENTORY_INSTANCE_CREATED_READY_FOR_POST_PROCESSING folio.Default.test.DI_PARSED_RECORDS_CHUNK_SAVED folio.Default.test.DI_RAW_RECORDS_CHUNK_PARSED folio.Default.test.DI_RAW_RECORDS_CHUNK_READ folio.Default.test.DI_SRS_MARC_BIB_INSTANCE_HRID_SET folio.Default.test.DI_SRS_MARC_BIB_RECORD_CREATED folio.Default.tst.DI_RAW_RECORDS_CHUNK_READ

Note that only one of the topics required for data import is created for the tst tenant. Log from mod-source-record-manager is attached.

I suspect the actual problem is in a shared library created from mod-pubsub in the KafkaTopicNameHelper class, but it manifests in this case in mod-source-record-manager. I suspect that this will need to be fixed in other modules, as well.

This bug is present in the Iris release and prevents tenants that don't match the pattern from being able to upgrade from Honeysuckle with a fully working system.

Additional info from : Regarding the limitation introduced by this regular expression check – it should be lifted completely and allow any valid FOLIO tenant ID e.g there should be no length limits (1 char names should be accepted) or limits wrt to specific characters used. If there are limitations wrt Kafka topic naming, the tenant ID should be encoded rather than carry these limitations to the tenant ID.

CSP Request Details

None

CSP Rejection Details

None

Potential Workaround

None

Attachments

1
  • 08 Jun 2021, 04:23 PM

blocks

defines

Checklist

hide

TestRail: Results

Activity

Show:

Ann-Marie Breaux June 24, 2021 at 7:52 AM

Hi - perfect! That's great to hear. I'm declaring this officially done.

Wayne Schneider June 23, 2021 at 4:46 PM

I performed two tests:

  1. Upgrade a Honeysuckle tenant

  2. Upgrade an existing Iris tenant (with broken DI) to the new versions of all DI modules

In both cases, Data Import works correctly and the appropriate Kafka topics are created. Thank you!

Ann-Marie Breaux June 22, 2021 at 7:06 PM

Checked with He'll update to the new releases and see if this resolves the issue for tenants IDs with fewer than 4 characters.

Ann-Marie Breaux June 21, 2021 at 4:25 AM

Hi Is there a way that you could review this in Iris Bugfest? The Bugfest tenants have tenant IDs longer than 4 characters, so I'm not sure what is the best way to test this.

cc:

Oleksii Petrenko June 18, 2021 at 11:40 AM

Please verify at Iris bugfest

Done

Details

Assignee

Reporter

Labels

Priority

Story Points

Sprint

Development Team

Folijet

Parent

Fix versions

Release

R1 2021 Hot FIx #2

RCA Group

Incomplete/missing requirements

CSP Approved

Yes

Affected Institution

Lehigh
Simmons

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs
Created June 8, 2021 at 4:23 PM
Updated July 1, 2021 at 9:23 AM
Resolved June 17, 2021 at 9:37 AM
TestRail: Cases
TestRail: Runs