Import Orders in MARC format
Author | Olamide Kolawole/Serhii Nosko |
---|---|
JIRA task | |
Business Requirements | |
Architects Review | IN PROGRESS |
PO review | IN PROGRESS |
Revision History
Version | Date | Owner | Description | Reason |
---|---|---|---|---|
v1.0 | 06.26.2022 | Initial version | ||
v1.1 | 06.28.2022 | Add model schemas | ||
v1.2 | 06.30.2022 | Add diagrams for Inventory creation | ||
v1.3 | 06.30.2022 | Add diagrams for creating and opening orders | ||
v1.4 | 07.07.2022 | Formatting changes | ||
v1.5 | 08.07.2022 | Serhii_Nosko | Add receiving API | |
v1.6 | 26.10.2022 | Kateryna Senchenko | Add flow description with explicit DI actions for creating Order, Instance, Holdings and Item (requires post-processing step in mod-orders) |
- Overview
- Solution Design
- Architectural Requirements
- Process Flows
- Basic flow for creating Open Order
- Create Pending Order with subsequent creation of Inventory entities (with source=FOLIO, no link to SRS MARC Bib)
- Create Order, Instance, Holdings, Item following the JobProfile (requires Post-processing step for Orders) - Applicable for Orders that should be created in Open status
- APIs
- Work Breakdown Structure
- Appendix
Overview
This document will illustrate the design that will integrate the Orders module area with Data Import(DI) module area. Order lines will be defined in MARC Bibliographic(MARC Bib) files by a user. The MARC Bib files will be parsed by DI and have records sent to mod-orders via Kafka eventing.
Solution Design
Architectural Requirements
- Dependency Inversion: Unlike traditional integrations that have been implemented with Inventory and less so with Invoices, Orders will be a consumer of Data Import. Data Import will be responsible for parsing input files from a user into a form that FOLIO modules will understand to Create/Update their respective objects.
- Single Source of Truth: This design should ensure that business logic only lives in one location within the FOLIO system. Orders and Data Import should not have copies/variants of the same functionality. This will reduce maintenance issues, clearly designate responsibilities and reduce cognitive load when developing the Orders and Data Import sub-systems.
- Clearly Defined Input & Outputs of Components: Each component in the process flow should have clearly defined inputs and outputs. The cardinality of inputs and outputs of each component should be generally less than three. If the cardinality is more than three, then division of the component should be considered.
Process Flows
Basic flow for creating Pending Order
User triggers creating of the order via API
Basic flow for creating Open Order
User triggers creation of the Order with Inventory Instance, Holdiings and Item
Create Pending Order with subsequent creation of Inventory entities (with source=FOLIO, no link to SRS MARC Bib)
If importing Order should be created in Pending status, no subsequent actions in the JobProfile will be processed. Therefore, Inventory entities that should be created when the Order is opened will not be mapped from the MARC source, but instead created out of existing Order information via Inventory API and assigned source=FOLIO. In this case the value Create Inventory, selected in the MARC-to-Order Mapping Profile, would determine which entities should be created. DI_COMPLETED event should be issued and JobExecution completed regardless of subsequent actions in the JobProfile.
Create Order, Instance, Holdings, Item following the JobProfile (requires Post-processing step for Orders) - Applicable for Orders that should be created in Open status
JobProfile for this flow should contain explicit actions with linked mappings for each entity that is expected to be created as a result of data import job. In this case value selected in Create Inventory field of the MARC-to-Order Mapping profile will be ignored.
Sequence diagram. Please note that DI_ERROR can be issued on each step of the processing, but was omitted for simplicity.
APIs
DI will communicate with mod-orders using kafka messaging, the same approach used for Inventory and Invoice module areas
Business Requirements
- Should orders be created as pending or open?
- Use cases for both, so include required field at top of field mapping profile that allows the user to set the status
Create orders
Applies when user choses to create pending order on the field mapping profile screen.
There is field in compositePurchaseOrder request payload named workflowStatus, it will be PENDING by default.
For creating orders CreateOrderEventHandler will use the same implementation as corresponding endpoint uses.
Method | Url | Request parameters | Request payload | Description |
---|---|---|---|---|
POST | /orders/composite-orders | lang | compositePurchaseOrder | Post a purchase order (PO) and a number of PO lines; record fund transactions corresponding to the order. Only in case an acquisition unit has to be assigned to |
Open orders
Applies when user choses to create open order on the field mapping profile screen.
There is field in compositePurchaseOrder request payload named workflowStatus, it should be set as OPEN.
For creating orders CreateOrderEventHandler will use the same implementation as corresponding endpoint uses.
Method | Url | Request parameters | Request payload | Description |
---|---|---|---|---|
POST | /orders/composite-orders | lang | compositePurchaseOrder | Post a purchase order (PO) and a number of PO lines; record fund transactions corresponding to the order. Only in case an acquisition unit has to be assigned to |
Receive orders
Basically from mod-orders side receiving and check-in API it the same thing.
Method | Url | Request parameters | Request payload | Description |
---|---|---|---|---|
POST | /orders/check-in | lang | checkingCollection | Check-in items spanning one or more PO lines |
Work Breakdown Structure
Open Items
Question | Status | Answer |
---|---|---|
Oder payload has field 'approved', that is false by default. What value for this field DI should populate? | IN PROCESS | Ann-Marie Breaux (Deactivated) will discuss with SMEs. |
When we have situation that max order lines limit is 10, but have 15 order lines and 2 orders in marc file, how they should be mapped? | IN PROCESS | Ann-Marie Breaux (Deactivated) will discuss with SMEs. |
PO number can be auto generated by mod-orders system. Does it necessary to place this field on Order mapping profile? | IN PROCESS | Ann-Marie Breaux (Deactivated) will discuss with SMEs. |
When creating order lines - its necessary to specify order source from these allowed values: [User, API, EDI, MARC, EBSCONET]. For our case it should be MARC? | CLOSED | MARC value should be used. |
Do we need to support orders check-in after receiving? | CLOSED | Updated after conversation with Thunderjet Actually, checking and receive is the same thing, DI should use checking endpoint implementation |
Is it possible scenario when DI going to open order, but not create any Instance, Holdings. Item? | CLOSED | Yes, it's possible the library is ordering something that they do not want represented in Inventory (like a database that they maybe want to handle through the ERM app), or other random things that they do not plan to keep in their permanent collection. |
Open items to brainstorm with SA, Devs
Question | Status | Answer |
---|---|---|
We use database tables for DI handlers deduplication. Mod-orders does not have schema in DB and any DB related code. Do we need to follow the same approach with introducing schema as we did in mod-inventory | CLOSED | Yes, we need to follow the same approach for event deduplication as in mod-inventory. |
If we are going to create Inventory Instance, Holdings, Item separately - need to discuss with Thunderjet possibility to indroduce new order parameter inventoryFlow with allowed values: [Synchronized, Independent], the same as did in Order Receiving flow | CLOSED | We will not introduce new parameter inventoryFlow, if need possibility to distinguish logic - orderSource field should be used. DI will always pass MARC as order source. |
Created stories
Description | Module | Story |
---|---|---|
Create default order mapping profile | data-import-converter-storage | TBD |
Extend mapping engine with ability to create order and multiple order lines from the same MARC record. | data-import-processing-core | TBD |
Adjust mod-orders to validate purchase order lines limit from Orders mapping profile | ||
Add kafka handlers in mod-orders | mod-orders | TBD |
Appendix
Data Model: Purchase Order
Required fields
Field name | Allowed values | Description |
---|---|---|
orderType | [One-Time, Ongoing] | DI will setup only One-Time order type |
vendor | - | Vendor that user chooses on Order field mapping profile |
Not applicable fields for DI orders flow
Business requirements to answer why provided fields are not necessary to populated
Business Requirements
- Use for one-time orders (physical, electronic, or both). Do not use for ongoing or package orders
- Should field mapping profiles use order templates?
- No. The field mapping profile and its mapped/default data already acts as a template. Once a field mapping profile is set up, it does not have to be re-created each time a user imports
- Can it create/assign order notes?
- No. Not the order notes created in the separate notes app
- Can it create/assign tags?
- No, not yet, for any record type. Aiming to support tags in a future version of DI
Field name | Allowed values | Description |
---|---|---|
reEncumber | [True, False] | Indicates this purchase order should be re-encumbered each fiscal year |
needReEncumber | [True, False] | Indicates that order needs to be re-encumbered |
ongoing | - | Ongoing object includes these fields: interval, isSubscription, manualRenewal, notes, reviewPeriod, renewalDate, reviewDate All these fields should not be populated from DI, because DI does not support ongoing orders. |
template | - | Predefined template to create order from |
notes | - | Free-form notes associated with this purchase order |
tags | - | List of simple tags that can be added to the purchase order |
Data Model: Composite Purchase Order Lines
Required Fields
Field name | Allowed values | Description |
---|---|---|
acquisitionMethod | - | The UUID format string |
cost | - | The purchase order line cost. Includes fields: "listUnitPrice", "listUnitPriceElectronic", "currency", "additionalCost", "discount", "discountType", "exchangeRate", "quantityPhysical", "quantityElectronic", "poLineEstimatedPrice", "fyroAdjustmentAmount" Currency field is required. |
orderFormat | [Electronic Resource, P/E Mix, Physical Resource, Other] | The purchase order line format |
source | [User, API, EDI, MARC, EBSCONET] | The source of the order |
titleOrPackage | - | The title of the materials |
Not applicable fields for DI orders flow
Business requirements to answer why provided fields do not necessary to populate
Requirements from business
Use for one-time orders (physical, electronic, or both). Do not use for ongoing or package orders
Field name | Allowed values | Description |
---|---|---|
isPackage | [True, False] | Indicates that this POL is for a package. Default - false |
packagePoLineId | - | Specify package po line id, applicable only package orders |
renewalNote | - | Specify renewal note, applicable only for ongoing orders |
Data Model: Check-in Collection
Receiving and check-in are the same things from mod-orders API side
Required fields
Field name | Allowed values | Description |
---|---|---|
toBeCheckedIn | - | Contains purchase order line id and collection of pieces to be checked in |
totalRecords | - | Total records size to be checked in |