Inventory import - Trillium - Notes on functionality
New app: Inventory import
Inventory import ui-inventory-import is a new FOLIO approved Stripes UI module providing an administrative interface by communicating with mod-inventory-update (the back end module).
Inventory Import provides a way to bring bibliographic information into FOLIO. Any XML-based format can be used, including MARCXML, and transformation pipelines can be defined for different import channels. Imported records, once transformed, are added to FOLIO inventory. Inventory import is format agnostique and scalable. It can be used for import of very large jobs; e.g. during data migration, or used for upsert jobs with near real time updates of FOLIO inventory.
The main elements of the Inventory import are
a channel
a processing pipeline
the "transformation" has an ordered set of "transformation steps"
the outcome of this process is sent to CRUD component that will batch and persist the results to inventory storage.
Inventory import allows for configuring, running, and monitoring import jobs and troubleshoot e.g. failed record via access to log information and more.
The work is developed by GBV and Index Data as the Sif development team. The solution is designed to support the German library’s networks CBS2FOLIO workflow for metadata, but is build generic so it can support also MARC21 records with a complying XSLT mapping sheet.
The design considerations are described more in depth on the FOLIO wiki: Design of XML importing APIs in mod-inventory-update
Use case
Inventory Import provides a way to bring bibliographic information into FOLIO. Any XML-based format can be used, including MARCXML, and transformation pipelines can be defined for different import channels. Imported records, once transformed, are added to FOLIO inventory.
Key difference when comparing Data import and Inventory import
Inventory import can load very large files, e.g. when doing an initial data migration.
When loading data using Inventory import is populating Inventory records (instance, holdings, item), but do not store underlying MARC records in Source Record Storage (SRS).
Related JIRA
https://folio-org.atlassian.net/browse/UXPROD-5571
https://folio-org.atlassian.net/browse/UXPROD-5130
🎥 Video:
[insert link here]
How to import XML files
The importing component Inventory import (and `mod-inventory-update´) consists of so-called import "channels". Each channel has a file queue of jobs, that source files can be uploaded to, and a processing pipeline that can perform the importing of source files from the queue to inventory storage.
The channel design is meant to allow for multiple input formats, for example, potentially, binary MARC or JSON files. However, the current pipeline implementation supports XML source files.
Another important piece of this new app, is in detail to inspect eventually failed records.
Inventory import has two segments:
Jobs
Failed records
Jobs
A job is the output of the defined XML transformation. The columns in the result list can be configured to show, only the selected columns.
When clicking on the Job Id in the result list, then the given job is displayed in detailed view. The job can have different different statuses; e.g. done, and interrupted.
If a fatal error occurs, the import job will be marked paused so that processing can potentially be resumed once the problem is resolved. When the job is resumed, any processing statistics and failed records will then count towards that same import job until the last file in the queue is processed.
Interrupted:
Failed records
In the Search and filter pane the user can define the time period for the failed records to investigate. The display of columns can be configured for the given session.
It is possible to export the failed record as a .csv file.
When clicking on the instance title of a given failed record in the result list, then the detailed information for that record is displayed in the third pane.
Note the arrows pointing to the left can be clicked and will then more information is unfolded.
How to manage Inventory import
Settings > Inventory import
Inventory import process a set of one or more custom written XSLT style sheets that transform arbitrary metadata records into a structure that is compatible with the schema of inventory storage.
In Settings the transformation pipeline can be pieced together by multiple transformation steps (XSLT style sheets) that can be added, viewed, reordered or deleted through Settings > Inventory import. Example on a transformation step is e.g. step-pica2instance
Once the XML transformation is completed the module will convert the XML to JSON, gather the records in batches, and push them to inventory storage through the upsert engine.
Setting up the Transformation pipelines
A processing pipeline, which is called a "transformation pipeline"
Defining the Transformation steps
The "transformation" has an ordered set of "transformation steps" with XSLT style-sheets, where the last step will be a generic (system supplied) crosswalk of the XML to JSON.
You can duplicate, edit and delete a transformation step.
Note when you edit your XSLT stylesheet you’d probably prefer to use a more robust XSLT editor. Inventory import do have a XSLT validation build in to secure your transformation step is valid code.
The channel design is meant to allow for multiple input formats, for example, potentially, binary MARC or JSON files. However, the current pipeline implementation supports XML source files. Type: XmlTransformationStep.
Setting up the Channel
To use the XML importing API, then each of the channels used for inventory import must be configured with a transformation pipeline (See further in this document). The transformation pipeline is a chain of zero or more XSLT transformation steps that will restructure XML records of arbitrary format into "inventory record sets" suitable for running insert or updates on instances, holdings and item records.
When clicking on a given channel, e.g. Sample PICA channel then you see this channel in detailed view:
A given import channel has an associated queue of files.
Each channel has a file queue that source files can be uploaded to, and a processing Transformation pipeline that can perform the importing of source files from the queue to inventory storage.
When a channel gets enabled/commissioned it will also have a file queue that source files can be uploaded to. If the channel is also listening it will pick files from the queue. The channel API has some extra properties displaying the status of the channel's file queue.
Terminology
Channel: An import configuration, and an active process ready to import if the channel is enabled.
Channel listening?: The channel is set to process files from the queue (if also enabled).
Channel enabled?: The channel is set to allow uploading of files to the queue and should be commissioned (have an active import process).
Channel commissioned? The channel currently has an active import process running. If the channel is disabled no process will be running, if the channel is enabled it will be running unless the module was just restarted.
The channel displayed in third pane in edit view:
When clicking the Actions menu, then the third action is Upload. And you upload new files by doing drag and drop:
Configuring the Log deletion
The given library can configure for how long the log files are to be kept in the system. The Number and the time interval can be set; e.g. 5 weeks, or 2 months.
Links and more background documentation
GitHub documentation: mod-inventory-update (detailed README file and API documentation)
GitHub documentation: ui-inventory-import
Slide deck Inventory import - the admin tool for mod-inventory-update, presented at May 11, 2026
Video (insert link when available)