High-level architecture

A remote storage location is a physical location where books and other physical items are stored.

Image Added

Note: Accession queue, accession table and remote location settings are 3 tables in Postgres database (in the corresponding schema for tenant).

Remote-storage related business logic

Inventory-storage module

• The module should send messages (hereinafter update notifications) on every update/create/delete to the corresponding Kafka topic (e.g. instance-updates, holding-updates, item-updates)

Remote-Storage module

• The module should be used for storing and retrieving all remote-storages settings
• The module should be used to set remote-storage for the location from UI. This info should be stored in Locations table in remote-storage DB schema
• There module has access to the Accession queue table in the same DB schema
• The module subscribes for holding/items update notifications. If permanent or temporary location is changed from the one which is not marked as Remote (e.g. doesn't exist in Locations table) to the one which is marked as remote and corresponding remote-storage accession setting is set to "Folio-initiated", this item/holding is added to the Accession queue
  
  
• The module should provide API for adapter modules, namely:
  • The REST method to retrieve items for accession list by: flag isAccessioned, remote-storage name, date, limit.
  • The REST method to set flag isAccessioned to true
  • The REST method to retrieve configuration of the remote storage (e.g. name, URL, etc.)

Note: In order to increase performance remote-storage settings and remote-storage locations could be cached. If there is one instance of module the cache eviction can be simply managed (as all updates are done through this module). If there are several instances of module cache eviction can be managed by some some eviction policy or the cache could be made distributed.

Requesting a Remote Storage Item for Circulation

Remote-storage module should listen to events in PubSub, which are provided for audit functionality.

1. If there is a page request for this item and item’s effective location (retrieved from inventory-storage and cached) is remote - circulation Request is sent to retrieval requests queue. Retrieval requests queue is database table like the accession queue database table. The same pooling from dematic-edge module mechanism should be used for it.
2. Remote Storage response indicates whether an item is successfully requested or rejected (with rejection details).
   1. If item was successfully requested, item is deleted from the retrieval request queue. 
   2. If an item is failed to be requested because of the server error, FOLIO keeps the item in the queue.
   3. If an item is rejected because of a logical reason (item not found, item is locked, item is out of the container etc) then the item is deleted from the queue and error is reported. 
3. Edge-dematic module should provide API for check-in items in its service point (e.g. remote service point)

Dematic edge module 

• Module will have connection to remote-storage module API
• The module should send items to Dematic (for Dematic Staging director) based on schedule or time interval, which is configured in remote-storage settings
• The module should response with the list of items (for Dematic EMS)

Dematic edge module deployment details

Dematic StagingDirector connection should be established from the dematic edge Folio module. Therefore Dematic edge module needs to know the name of all tenants, which has StagingDirector connection. In order to provide it before the deployment the list of tenant names (e.g. ids) should be put to AWS parameters store. The tenant names list separated by comma (e.g. diku, someothertenantname) should be stored in AWS param store (like it is put for API_KEYs) in the variable with key: stagingDirector_tenants.

API endpoints of remote-storage module

API provides the following URLs for working with mappings between Folio locations and remote storage configurations:

API to provide access to accession queue

Dematic edge endpoints *

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>

<asrItems>

<asrItems>

<asrItem>

<itemNumber>82703197</itemNumber> <-- barcode

<title>Philologia antiqua.</title>

<author> </author> <-- primary contributer

<callNumber> PA1.A1P45 </callNumber> <-- effective call number

</asrItem>

</asrItems>

</asrItems>

Empty response (if there is no items for accession):
<?xml version="1.0"encoding="UTF-8" standalone="yes"?>

<asrResponse>

<code>011</code>

<message>Currently no request found</message>

</asrResponse>

Response with items:
<?xml version="1.0"encoding="UTF-8" standalone="yes"?>

<asrRequests>

<asrRequests>

<asrRequest>

<holdId>354167</holdId>

<itemBarcode>82703197</itemBarcode>

<title>Philologia antiqua</title>

<author></author>

<callNumber>PA1.A1P45</callNumber>

<patronBarcode>7137806</patronBarcode>

<patronName>Bottorff,David</patronName>

<requestDate>2020-07-29 00:00:00.0</requestDate>

<pickupLocation>JRLMAIN</pickupLocation>

<requestStatus>1</requestStatus>

<requestNote></requestNote>

</asrRequest>

</asrRequests>

</asrRequests>

Note: * Dematic StagingDirector does not use an API and connect to the Folio system via TCP socket.

• CaiaSoft API guide

High-level architecture

A remote storage location is a physical location where books and other physical items are stored.

Image Added

Note: Accession queue, accession table and remote location settings are 3 tables in Postgres database (in the corresponding schema for tenant).

Remote-storage related business logic

Inventory-storage module

• The module should send messages (hereinafter update notifications) on every update/create/delete to the corresponding Kafka topic (e.g. instance-updates, holding-updates, item-updates)

Remote-Storage module

• The module should be used for storing and retrieving all remote-storages settings
• The module should be used to set remote-storage for the location from UI. This info should be stored in Locations table in remote-storage DB schema
• There module has access to the Accession queue table in the same DB schema
• The module subscribes for holding/items update notifications. If permanent or temporary location is changed from the one which is not marked as Remote (e.g. doesn't exist in Locations table) to the one which is marked as remote and corresponding remote-storage accession setting is set to "Folio-initiated", this item/holding is added to the Accession queue
  
  
• The module should provide API for adapter modules, namely:
  • The REST method to retrieve items for accession list by: flag isAccessioned, remote-storage name, date, limit.
  • The REST method to set flag isAccessioned to true
  • The REST method to retrieve configuration of the remote storage (e.g. name, URL, etc.)

Note: In order to increase performance remote-storage settings and remote-storage locations could be cached. If there is one instance of module the cache eviction can be simply managed (as all updates are done through this module). If there are several instances of module cache eviction can be managed by some some eviction policy or the cache could be made distributed.

Requesting a Remote Storage Item for Circulation

Remote-storage module should listen to events in PubSub, which are provided for audit functionality.

1. If there is a page request for this item and item’s effective location (retrieved from inventory-storage and cached) is remote - circulation Request is sent to retrieval requests queue. Retrieval requests queue is database table like the accession queue database table. The same pooling from dematic-edge module mechanism should be used for it.
2. Remote Storage response indicates whether an item is successfully requested or rejected (with rejection details).
   1. If item was successfully requested, item is deleted from the retrieval request queue. 
   2. If an item is failed to be requested because of the server error, FOLIO keeps the item in the queue.
   3. If an item is rejected because of a logical reason (item not found, item is locked, item is out of the container etc) then the item is deleted from the queue and error is reported. 
3. Edge-dematic module should provide API for check-in items in its service point (e.g. remote service point)

Dematic edge module 

• Module will have connection to remote-storage module API
• The module should send items to Dematic (for Dematic Staging director) based on schedule or time interval, which is configured in remote-storage settings
• The module should response with the list of items (for Dematic EMS)

Dematic edge module deployment details

Dematic StagingDirector connection should be established from the dematic edge Folio module. Therefore Dematic edge module needs to know the name of all tenants, which has StagingDirector connection. In order to provide it before the deployment the list of tenant names (e.g. ids) should be put to AWS parameters store. The tenant names list separated by comma (e.g. diku, someothertenantname) should be stored in AWS param store (like it is put for API_KEYs) in the variable with key: stagingDirector_tenants.

API endpoints of remote-storage module

API provides the following URLs for working with mappings between Folio locations and remote storage configurations:

API to provide access to accession queue

Dematic edge endpoints *

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>

<asrItems>

<asrItems>

<asrItem>

<itemNumber>82703197</itemNumber> <-- barcode

<title>Philologia antiqua.</title>

<author> </author> <-- primary contributer

<callNumber> PA1.A1P45 </callNumber> <-- effective call number

</asrItem>

</asrItems>

</asrItems>

Empty response (if there is no items for accession):
<?xml version="1.0"encoding="UTF-8" standalone="yes"?>

<asrResponse>

<code>011</code>

<message>Currently no request found</message>

</asrResponse>

Response with items:
<?xml version="1.0"encoding="UTF-8" standalone="yes"?>

<asrRequests>

<asrRequests>

<asrRequest>

<holdId>354167</holdId>

<itemBarcode>82703197</itemBarcode>

<title>Philologia antiqua</title>

<author></author>

<callNumber>PA1.A1P45</callNumber>

<patronBarcode>7137806</patronBarcode>

<patronName>Bottorff,David</patronName>

<requestDate>2020-07-29 00:00:00.0</requestDate>

<pickupLocation>JRLMAIN</pickupLocation>

<requestStatus>1</requestStatus>

<requestNote></requestNote>

</asrRequest>

</asrRequests>

</asrRequests>

Note: * Dematic StagingDirector does not use an API and connect to the Folio system via TCP socket.