Architectural PoC Preview Feedback
Introduction
In early March, 2024 an architectural proof of concept was shared with the community along with a spreadsheet providing context and serving as a means for the Folio community to provide feedback/questions. Such feedback was collected for roughly a month, reviewed and captured here along with responses. We chose to put the responses on the wiki with the goal of making this information easier to find later on (e.g. via wiki searches)
What if I have feedback/questions which don’t appear below?
Additional feedback/questions about the architectural PoC can be added to the table below. Relevant new entries will be reviewed and answered as time allows while the PoC is ongoing. Notes and recording from two Q&A session are also available here: https://folio-org.atlassian.net/wiki/spaces/TC/pages/158236674 and https://folio-org.atlassian.net/wiki/spaces/TC/pages/178913282. This page also provides a technical overview of the system: https://folio-org.atlassian.net/wiki/spaces/PLATFORM/pages/193134643 .
| First/Last Name | Summary (Short) | Description (Long) | Additional Notes | Response |
1 | @Ingolf Kuss (SysOps) | KeyCloak receives positive feedback | The use of KeyCloak receives a generally positive feedback of the SysOps group. This will allow us to better fulfil legal security requirements. Changes in administrative software / scripts are expected to be not great. | KeyCloak helps to bind identity management to FOLIO. | Agreed. |
2 | @Ingolf Kuss (SysOps) | Role Bases Access Controls generally positive | The work on role based access controls is seen generally positive by SysOps SIG. It will strengthen the security of the system. |
| Agreed. |
3 | @Ingolf Kuss (SysOps) | Log out in test environment doesn't work | The Logout of the test environment (arch-poc) doesn't work. One stays logged-in. | Maybe this points to some issue with the new keycloak-mechanism ? | This issues is specific to Firefox. It should behave properly on Chrome. See |
4 | @Ingolf Kuss (SysOps) | Reason for changing to Kong | SysOps SIG don't see a reason to change from Okapi to Kong. What can Kong do that Okapi can't do ? |
| Replacing Okapi was not something we set out to do, but rather something we arrived at while making other enhancements. This eventually led to a choice:
Here are some of the benefits of Kong:
|
5 | @Ingolf Kuss (SysOps) | License issue with Kong | Kong would need to be run in the commercial premium version to fulfil requirements of LSP hosting for libraries. This contradicts to the o in folio. |
| The OSS version of Kong is sufficient for Folio’s needs and uses the Apache 2.0 license. See |
6 | @Ingolf Kuss (SysOps) | Missing documentation for Kong | How does Kong interact with FOLIO ? There is no documentation for us to review. |
| We acknowledge that documentation for Kong in the context of Folio is still forthcoming. |
7 | @Ingolf Kuss (SysOps) | Retooling for using Kong in existing FOLIO SysOps architecture | Doing things with Kong that we are now familiar doing with Okapi will take a lot of retooling for integrations, deployment scripts and jobs for upgrading the system. |
| While some retooling may be required, it isn’t expected to be extensive. Direct interaction with Kong on the control plane is limited. The following manager components handle these Kong interactions for you:
|
8 | @Ingolf Kuss (SysOps) | Questions about Sidecars | SysOps: Where does the code live ? Inside or outside the module ? Missing Documentation for Sidecars. Found some documentation for Kubernetes deployment, but what about plain docker deployment scenarios ? | Need more information about the use of Sidecars in FOLIO. | Sidecar code is distinct from the module code. Modules are unaware of sidecars and for the most part remain unchanged. We acknowledge that documentation for the module sidecars is lacking but is forthcoming. Sidecars can be deployed either directly via docker, or via any container management system, e.g. AWS ECS, Kubernetes, etc. |
9 | @Ingolf Kuss (SysOps) | Reasoning of Sidecars | SysOps SIG questions if this is the right approach, as it couples the modules closer together. Will Sidecars replace Kafka ? | What exactly is the problem that Sidecars are solving in FOLIO ? | Sidecars provide individual modules with authorization, which they otherwise lack. They also provide new capabilities such as direct module-to-module routing, structured transaction logging conforming to the Common Log Format, etc. Modules are unaware of the sidecars. Pairing sidecars with modules does not increase coupling between modules. Sidecars will not replace Kafka. |
10 | @Jakub Skoczen (TC) | Architectural diagrams | Can you please provide detailed architectural description of the PoC that includes systems and network diagrams and outlines responsibilities and interactions (including protocols used) between components included in the POC? |
| We acknowledge that documentation for this is still forthcoming. |
11 | @Jakub Skoczen (TC) | Alternatives/evaluation | Has there been a process for reviewing alternative solution and components as part of the PoC that resulted in the selection of specific components? If so what evaluation criteria was used to decide on this selection (e.g. why was Kong selected) |
| Yes, alternative solutions/technologies were evaluated. Documentation describing the selection process will be forthcoming (e.g. as part of the RFCs).
|
12 | @Jakub Skoczen (TC) | Hosting impact | Can you provide information on the impact on hosting (deployment and maintenance): e.g. is scalability impacted (has the effect been measured, if so, how), is observability (e.g. request tracking) |
| Deployment and maintenance - From a hosting perspective, Kong and Keycloak do not add complexity, they are fully containerized like the rest of Folio, and provide graphical administrative user interfaces. Observability - Kong and Keycloak both support observability. For Kong this is via OSS plugins, e.g. prometheus, open telemetry, datadog, etc. See , and for Keycloak it’s supported out of the box. |
13 | @Uwe Reh | What the intention of this change? | The tools mentioned in the 'Scope' section seem to be able to replace OKAPI. Is this intended? |
| See above (4) |
14 | @Uwe Reh | Why Kong Gateway? | Yes, Kong does seem to be leading the market. But the free version of the gateway is limited. |
| See above (4) |
15 | @Uwe Reh | Performance | Suppose OKAPI is a bottleneck. Wouldn't mgr-applications and mgr-tenant-entitlements become the new bottleneck? |
| The manager components operate in the control plane (Used for system administration, e.g. creation/management of tenants, registration of applications, enabling applications for tenants, etc). The manager components are not involved in the data plane (Routing/servicing API requests at runtime) |
16 | @Uwe Reh | Performance | Direct communication between modules may take the pressure off a centralized broker. But it comes with new costs like latency (more internal http calls) and logging issues. |
| The number of hops/internal calls is actually the same or less with sidecars than it is with Okapi. We are working on visualizations of these flows. Regarding logging, See above (9) |
17 | @Jenn Colt | UX changes to user management | Is there a PO assigned to work with the user management sig to discuss changes to permissions/roles/etc. that will manifest in the UX of managing users? | This came up in the PC. meeting, PC may help with bringing the community to the discussion | AFAIK Folio does not have a dedicated Users PO. However, we did work with Kimie K. to develop UX designs. We’d be happy to present these changes to the Users management SIG. |
18 | @Jenn Colt | Gateway options/introduction | Does Kong fully replace Okapi right away or will there be a case for 'legacy' FOLIO applications to be on Okapi and new applications to have Kong? |
| A Folio Platform will use a single API Gateway at its core, Okapi or Kong. There is no support for a hybrid system, nor is there any reason to operate in that manner. Folio Applications are unaware of - and unconcerned with - the API Gateway in use. |
19 | @Mike Gorrell | Both Kong and Okapi for a while? | Given the response to (18), is EBSCO’s plan to have everyone stop using Okapi and use Kong as of a certain date (e.g. the Ramsons release), OR, is it expected there will be a mechanism to allow a hosting provider to choose which API gateway to use for some period of time? |
|
|
20 | @Julian Ladisch | Kong Gateway OSS Support | Can you confirm that only the latest minor version of Kong Gateway OSS is supported and gets security fixes? https://github.com/Kong/kong/discussions/10866 | Kong Gateway release dates: |
|
21 | @Julian Ladisch | SAML authentication | Does the free OSS version of Kong support SAML authentication, or is the paid enterprise version needed? | Many universities use SAML authentication. |
|
22 | @Maccabee Levine | Single-server installation | Will the single-server installation documents (fresh install and upgrade) be updated to reflect the various architectural changes? |
|
|
23 | @Maccabee Levine | Vagrant deployment | Will the Vagrant boxes be updated to support the various architectural changes? |
|
|
24 | @Julian Ladisch | Keycloak | Will FOLIO be a Keycloak client only, or will the FOLIO release come with a Keycloak server similar to Kafka and Minio? If server what does it store (patrons, staff, permissions, loans, …)? |
|
|
25 | @Jeremy Huff | Kong and Keycloak | Are these changes necessary for application formalization? Whether or not they are a good idea, should the question of their use be handles separately from the conversation surrounding application formalization? |
|
|
26 | @Jeremy Huff | Integration Issues | What changes do we anticipate to the API face of FOLIO, specifically in regards to integrations between FOLIO and other Library systems that are clients of OKAPI? |
|
|
27 | @Thomas Trutt | Keycloak | From the UI previews that have been shared briefly it appears that user permissions will be more granular. Is there a plan to provide a script to map current permissions over to new Keycloak rules or will institutions have to set that up themselves. |
|
|
28 | @Thomas Trutt | Keycloak | Related to #27. Is there a plan to have an overlap of the current auth system and Keycloak or will it be a hard change over? |
|
|
29 | @Marc Johnson | Permissions | How do existing permissions defined by modules work with Eureka architecture (Kong, keycloak, sidecars etc) e.g. what checks that a client can make a request to a module? |
|
|
30 | @Julian Ladisch | Kong | Many universities and libraries use SAML/Shibboleth, advanced (!) LDAP, and/or OpenID Connect (OIDC) for authentication. The respective Kong plugins are not OSS but require a monthly payment: What effort is needed to support these authentication methods without these plugins but with OSS software? | Discussed during Authentication/authorization is handled by Keycloak not Kong so these plugins are not needed. Kong has fewer responsibilities than Okapi did. |
|
31 | @Julian Ladisch | Kong | Mutual TLS (mTLS) is a state-of-the-art authentication method available for more than 20 years that at least some universities/libraries need to use to protect their systems against attacks in the next years, for example for machine-to-machine communication. However, Kong's mTLS plugin is not OSS: How can a FOLIO system that uses Kong can support mTLS without paying for that plugin, what effort is needed? | Discussed during Authentication/authorization is handled by Keycloak not Kong so these plugins are not needed. Kong has fewer responsibilities than Okapi did. |
|
32 | @Julian Ladisch | Kong | Zero trust environments require that machine-to-machine communication uses mutual authentication where each machine must authenticate (note: this is about machine authentication, not about human authentication). Kong's mTLS plugin is not OSS: . How can mTLS between Kong and the modules/sidecars been enabled without that Kong plugin? |
|
|
|
|
|
|
|
|
The link to “architechtural proof of concept” is not working. Not here and not in the spreadsheet.