Will focus on MODDATAIMP-419 instead

, Just in case my previous comment was missed, please consider closing this issue and instead focusing efforts on MODDATIMP-419 which is about handling a single import run of 10k+ records that includes a match profile and updates the MARC Bib.

I added results yesterday based on failed tests against the Iris reference environment.

Grooming: needs review and checking on Iris; then decide if any updates are needed

In this week's grooming, let's review this, plus the various bugfixes/changes related to PTF work, and link up the Jiras that we think would make concurrent imports testable in Iris bugfest.

Are you asking me to continue the steps as outlined in MODDATIMP-419 on Bugfest? It appears to be down (or "kaput") right now.

I just want to reiterate that these two tickets are related. We prefer to focus everyone's efforts on optimizing Data Import's ability to handle job executions of increasing size (10 - 15k records in this example) MODDATIMP-419 instead of having to split the file into multiples of <3k records and submit parrallel job executions (MODDATAIMP-424).

Hi The way that PTF tests are shaping up, it looks like Iris will be able to handle 50K without seizing up. Let's leave open for now; if you could try testing in Bugfest in the next couple of weeks (preferably toward the end of a US workday), then we'll figure out if anything else needs doing. We've definitely had success with large files in Bugfest, but not consistently and not with significant additional work happening in other parts of the system.

I'm going to block this for now, and we'll wait and see if any other work is needed. Does that sound OK with you?

Thanks for all your help,

Thanks for attaching the files – I agree with making sure that a single import works successfully for Iris and agree this can be closed

I've added the files as requested.

Initial Data Import Run: (SRS MARC Bib and Instance creation using default DI profile):

Follow up Data Import Run: (Overlay using our own set of DI profiles):

I also provided the full 10,300 processed records in a single file:

It will be tough to fully recreate the conditions for this test without our DI profiles. I can share them here, but honestly my vote would be to not dwell on the past (Honeysuckle) and instead close this issue so we can focus on standing up a stable Iris environment that can be used for testing. and – do you agree?

We don't actually want to split the processed MARC into multiple files and parallel DI job executions. We prefer to submit a single file, but Honeysuckle can't handle this. Please see our latest results running our preferred process against Iris Bugfest (incomplete) and snapshot-load (success) in MODDATAIMP-419.