Data import - Reliability & Performance testing

About this document: Several teams are working on improving data import reliability. From institutions, we need

Job profiles (including match/action/field mapping profiles)
Environment that a hosting provider can run tests (if applicable)
Files to use for testing
Expected time to complete job
Who submitted this information and from which institution

Goals

Measure reliability and performance during low and high peak activity periods
Measure against competitors benchmarks
Measure goal of handling a large record file of 100,000
Define clear and concise FOLIO benchmarks

Job profile name	Expected outcome of import	Environment	Precondition(s) does another profile need to be run beforehand? do we need to edit the data in some way?	File to use for testing please specify number of records in file.	Can this same file be processed over and over? (Y/N)	Reporting institution/reporter
Default - Create instance and SRS MARC Bib	A MARC bib and connected FOLIO instance is created Available in real-time for search via Inventory app	Orchid Bugfest	N/A	4900 records (.mrc) 100,000 records_1 (.mrc) 100,000 records_2 (.mrc) 238,000 records (.mrc)	Yes
Default - Create instance and SRS MARC Bib	A MARC bib and connected FOLIO instance is created Available in real-time for search via Inventory app	Orchid Bugfest	N/A	~400 records (marcxml)	Yes
Default - Create instance and SRS MARC Bib	A MARC bib and connected FOLIO instance is created Available in real-time for search via Inventory app	Orchid Bugfest	N/A	vanity_fair (json) NOTE - I cannot get this record to even load.
Default - Create SRS MARC Authority	A MARC authority is created Available in real-time for search via MARC authority app	Orchid Bugfest	N/A	1000 records file (.mrc) 5000 records file (.mrc) 10,000 records file (.mrc) 22,788 records file (.mrc) 50,000 records file (.mrc) 198,885 records file (.mrc) 265,000 records file (.mrc)	Yes
Default - Create SRS MARC Authority	A MARC authority is created Available in real-time for search via MARC authority app	Orchid Bugfest		2_meetingname records (marcxml) 2_geographic records (marcxml)
Default - Create SRS MARC Authority	A MARC authority is created Available in real-time for search via MARC authority app	Orchid Bugfest		name_title (json) apple_corporate (json) NOTE - I cannot even load these records.
Update a MARC authority record
Import Harrassowitz invoice	Create invoices Available in real-time for search via Invoices app	Orchid bugfest		Harrassowitz_folder_1
Import Hein subscription invoice	Create invoices Available in real-time for search via Invoices app	Orchid bugfest		Hein_folder_1
TBD	Import orders in MARC	Orchid bugfest		5_records (.mrc) ~10_records (mrc)
Create MARC holdings	Creates MARC holdings and connected FOLIO holdings record Available in real-time via Inventory app		Must create a MARC bib record
FCBLT Default Inventory UPDATE SRS Record ONLY Using MATCH 999 ff i to 999 ff i (No Override of Protected Fields)	Matches incoming 999 ff $i to existing 999 ff $i On match updates the marc srs bib On no match, there's no action	Nolana production	N/A	Note: these files are always 1,000 records, and processing ranges from 2-20 minutes per file. See screenshot for varying duration. 1000 records (File 1) 1000 records (File 2) 1000 records (File 4) 1000 records (File 5) 1000 records (File 6) 1000 records (File 7) 1000 records (File 8) 1000 records (File 9) 1000 records (File 3) (Completes with errors - see screenshot) 8 errors with message: javax.ws.rs.BadRequestException: Date when processing started is not set, expected snapshot status is PARSING_IN_PROGRESS, actual - FILE_UPLOADED or system time Reloaded those 8 records with no errors 34,550 records (Large file)	Yes	5 Colleges
FCBLT Default Inventory UPDATE HOLDINGS Electronic Access ALL Fields (Deletes and Replaces ALL FIELDS) * [MAKE SURE TO HAVE ADDed the $f for URL relationship to your MRC]*	updating the holdings electronic access field in the holdings record. Each file has 600 records and the match is to the holdings hrid	Nolana Production		Note: these files have 600 records and times vary from 1-20+ minutes (see screenshot). 600 records (File 1) 600 records (File 2) 600 records (File 3) 600 records (File 4) 600 records (File 5) 600 records (File 6) 600 records (File 7) 600 records (File 8) 19,396 records (Large file)		5 Colleges
quickMARC 2023-10-04 Authority files	Use to Create/Update Authority records			Please add link to each file or let me know and I will give you access to a shared file.