What is Codex?
...
Folio is a platform that is based on a (specific) microservices architecture. Codex plays a key role in that architecture. A fully featured platform of microservices can easily consist of dozens of individual microservices.
<IMAGE: MONOCHROME MICROSERVICES>
However, some microservices are closely related to others, as shown below using common colors.
<IMAGE: COLORED MICROSERVICES>
Those closely related microservices can be grouped together into what we call a domain. The modules which belong to the same domain will reflect the following characteristics:
a shared context
a shared vocabulary
awareness of each other
a shared data model
<IMAGE: COLORED DOMAINS>
Codex is a Domain
Codex is a domain which consists of multiple modules (and apps). The most familiar Codex app is the Codex Search app. It provides the ability to search for resources across a number of other apps that may be used to manage resources: e.g. eholdings, inventory, institutional repositories, etc… A given app, such as the Codex Search app, typically consists of multiple modules.
...
But in practice, domains, and the microservices they contain, form a system and there are necessary interactions between the microservices, including across domain boundaries. Those interactions create dependencies, and in a flat model, this leads to entanglement - thus breaking the promise of a loosely coupled system.
The Entanglement Problem
The interaction between microservices (i.e. modules) is an integration problem. It requires that individual modules be aware of other modules. For modules in the same domain, this is expected and not an issue. However, the same is not true across domain boundaries.
...
Imagine that a Folio system has 3 resource managing modules (e.g. one from each of the inventory, KB, and acquisitions domains). Each will require 2 integration points, one for each of the other 2 domains. In total there are 3 integration links between the 3 domains. .
<IMAGE: DEPENDENCIES 3>
This may seem like a manageable approach that can scale linearly. But it is not. Now imagine that a Folio system has 5 resource managing modules. In all likelihood, in a flat modular system, each module will likely need to be aware of each of the other 4. As shown below, the number of interdependencies grows to 10.
<IMAGE: DEPENDENCIES 3>
In the general case, if a Folio system has N resource managing modules, the number of potential integration points isN(N-1)/2, which is not linear, but scales on the order of N2.
...
Essentially, the brokering module approach introduces a hierarchy. Through the use of a hierarchy, a module only needs to maintain integration with the level above it. It no longer needs to manage dependencies with its peers for the same purposes. With the use of a hierarchical layer such as Codex, the set of dependencies remains linear upon scaling. Individual resource managing modules only need to concern themselves with integration to Codex.
<IMAGE: DEPENDENCIES CODEX>
Codex is a hierarchical domain that acts as a coordinator for resource management. As a coordination domain it is Codex’s business to know about the other domains.
A Hierarchical Domain Model
A significant responsibility of Folio is to manage resources, and more specifically the records that describe those resources. However, not all parts of Folio require the same level of detail for those resources. Nor do they have the same scope requirements: Circulation doesn’t much care about the purchase price of an item, but Acquisitions certainly does. The following diagram focuses on resource management specifically. As shown, the resource managing domains can be organized into layers based on the depth of detail that they require and the purpose they serve.
<IMAGE: RECORD LAYERS>
Starting from the bottom:
Formal Records. These are the most detailed records available in the Folio system. This is where MARC records are found in the Folio system. The MARCcat app is used to edit, validate and even create MARC records which live in Source Records. Eventually, Folio will provide similar support for Dublin Core, BibFrame and other non-MARC formats. Note that In addition to bibliographic records, formal records can include such things as Order artifacts (EDI) and License artifacts (e.g. scanned paper licenses). In support of all source records is a sophisticated Import system that manages the ingestion and lifecycle of these documents.
Working Records. These are the app-specific versions of records that are necessary for the apps to deliver their functionality. These records may be bound to related source records at the lower layer. For example, an Instance record in Inventory may be bound to a MARC record in Source Records. When bound to a formal record the working record does not need to duplicate all the fields contained in the formal records - it only needs those that are required to complete the functionality delivered by the apps in that domain. Note that working records may also contain additional fields, not found in the formal records, but which are transient or operational in nature.
Unifying Records. This top layer is the one in which Codex and its modules operate. The purpose of this layer is to provide a uniform and high-level view across all of Folio. In particular Codex presents a complete and consistent view of all resource related objects from anywhere in the Folio system. Codex is afforded visibility to all the resource managing domains that might exist within a Folio system. It is not to say that Codex contains all the metadata related to any resource within Folio. Instead, resource records surfaced through Codex are linked to working (functional) records in the various resource managing apps, In turn those may also be linked to formal records, thus forming a path from Codex to the finest degree of metadata granularity.
...
How to tell if a particular resource is available to an institution, either available for circulation or for access or even just on order?
<IMAGE: USE CASE 1>
In this example there are 3 domains shown, each of which might represent any number of resource managing apps within.
None of the individual resource managing apps has a complete picture of all resources available in the system.
Codex is the starting point for locating a resource. Each of the resource managing apps “registers” itself to Codex so that resources may be located.
The resource managing apps inform Codex of the matching resource for which it is responsible..
Once the resource is located, the user can drill down into the responsible app to retrieve full details about that resource.
The Order system is a resource managing app within the acquisitions domain.
...
How do I synchronize my catalog with my OPAC or discovery system?
<IMAGE: USE CASE 2>
In this example there are 3 apps shown, but there could be more, such as additional KB apps, or Institutional Repository app(s) or even another Inventory app.
The SourceRecord Manager holds the MARC (or DC) records describing the resources in the Catalog.
None of the individual resource managing apps have a complete picture of all resources available in the system.
Codex is the starting point for gathering resources to be exported to the OPAC. Each of the resource managing apps “registers” itself to Codex so that its managed resources may be aggregated into the OAI-PMH response.
OAI-PMH requires resource descriptions in MARC (or Dublin Core) format. These will be found in SourceRecord storage. The resource managing apps will contain links to the appropriate bibliographic records in Source Record Storage (SRS).
An Edge API is used to deliver OAI-PMH responses to the external system. It allows the remote system to integrate to Folio using existing conventions (OAI-PHM).
The OPAC can be made aware of pending orders since the Order system can surface resources (pseudo-resources) that are “on order” - just as in Use Case 1 above.
There are no direct dependencies between the various functional modules (inventory, eHoldings, Orders) since all connections go through Codex.
...
Where do I store the relationship between two different records?
<IMAGE: USE CASE 3>
It would be simple enough to create a structure to represent and store such a Work in Inventory or likewise in eHoldings. But the problem arises when the two instances to be related, exist in two different parts of Folio: in this example, in Inventory and in eHoldings. Do we create the work structure in Inventory? Or do we do it in eHoldings? Or do we do it both? The answer is neither. We create the Work object in the Codex domain and link it to the resources that are managed respectively in Inventory and in eHoldings. This avoids a direct entanglement between Inventory and eHoldings. In this case, both can remain blissfully unaware of each other. Furthermore, since we are linking and not duplicating resource records between domains, we also avoid the problem of data synchronization.
...
Each Folio installation has its own Codex domain consisting of relevant Codex apps. From this perspective, Codex is not only the starting point from which to start resource management within each Folio. It is also the point where each Folio can pull in Codex compatible components from other Folio installations.
<IMAGE: USE CASE 4>
Multiple scenarios are Illustrated above:
...