Trillium - Performance Improvements and Changes in Existing Behavior

1 Observed Performance Improvements
- 1.1 Instances FOLIO & MARC
- 1.2 Holdings
- 1.3 Items
- 1.4 Users
2 Observed Behavior Changes
3 Related JIRAs:
4 Implementation details
5 Query
6 Identifiers
- 6.1 UUIDs
  - 6.1.1 non-ECS
  - 6.1.2 ECS
- 6.2 non-UUIDs
  - 6.2.1 non-ECS
  - 6.2.2 ECS

The behavior changes described on this page stem from the Bulk Edit performance improvement work completed under https://folio-org.atlassian.net/browse/UXPROD-4990 This work was aimed at making Bulk Edit more reliable and scalable for large jobs by changing how bulk edit operations are processed behind the scenes. As a result, some aspects of job execution and timing now behave differently than in earlier releases. The following sections describe these changes and the reasons for them.

Observed Performance Improvements

Performance listed here based on manual testing in FOLIO environment, populated with bugfest dataset. Separate PTF reports are listed in https://folio-org.atlassian.net/wiki/x/E4BRbg

Instances FOLIO & MARC

Identifiers	Records	Matching preview
File with UUIDs Search in Inventory (instance.type in [two-dimentional moving image, unspecified, notated music])	95,124	0:19:11
File with HRIDs the same records as in the file with UUIDs	95,124	0:16:18

Notes:

In Central tenant upload Instances with UUIDs takes a bit longer than upload with HRIDs

Holdings

Identifiers	Records	Matching preview
File with UUIDs 100K-hold-uuid-college.csv	99,986	0:02:35
File with HRIDs the same records as in the file with UUIDs	99,886	0:10:03

Items

Identifiers	Records	Matching preview
File with UUIDs 100K-item-uuid-college.csv	99,994	0:02:09
File with HRIDs the same records as in the file with UUIDs	99,994	0:33:51
File with UUIDs 100K-item-uuid-mix-college-univer-school-special.csv	99,995	0:01:06
File with HRIDs the same records as in the file with UUIDs	99,995	0:28:52

Users

Details to be added

Observed Behavior Changes

Details to be added

Related JIRAs:

Identifiers Tab - for jobs triggered with files containing UUIDs

Significant performance improvements (except MARC Instances) https://folio-org.atlassian.net/browse/MODBULKOPS-346
- In the ECS environment, Central tenant upload Instances with UUIDs takes a bit longer than upload with HRIDs MODBULKOPS-346
No warnings are generated for invalid data when matching records https://folio-org.atlassian.net/browse/MODBULKOPS-640
Error reporting changes:
- In the ECS environment, when the user lacks the required affiliation, affected records are reported as “Not match found.” https://folio-org.atlassian.net/browse/MODBULKOPS-627

No changes for bulk edit jobs triggered with other types of identifiers.

To be done:

improve performance for Items (other types of identifiers) https://folio-org.atlassian.net/browse/MODBULKOPS-604
make non-unique identifiers trigger error notification https://folio-org.atlassian.net/browse/MODBULKOPS-642
display correct number of records on matching preview https://folio-org.atlassian.net/browse/MODBULKOPS-603

Query Tab

Significant performance improvements (except MARC Instances) MODBULKOPS-551
No warnings are generated for invalid data when matching records https://folio-org.atlassian.net/browse/MODBULKOPS-640
Non‑unique identifiers do not trigger error notifications https://folio-org.atlassian.net/browse/MODBULKOPS-641
- In the ECS environment, Duplicates across tenants error is not reported and records go to bulk edit and could be updated successfully MODBULKOPS-636
Generic filename for the file with errors on matching records https://folio-org.atlassian.net/browse/MODBULKOPS-638