folio-module-sidecar
- 1 Overview
- 2 Sidecar initializing and runtime
- 3 Sidecar Runtime
- 3.1 Sidecar filters
- 3.1.1 Ingress request filters in details
- 3.1.1.1 1. SelfRequestFilter
- 3.1.1.2 2. KeycloakSystemJwtFilter
- 3.1.1.3 3. KeycloakJwtFilter
- 3.1.1.4 4. KeycloakTenantFilter
- 3.1.1.5 5. TenantFilter
- 3.1.1.6 6. KeycloakImpersonationFilter
- 3.1.1.7 7. KeycloakAuthorizationFilter
- 3.1.1.8 8. SidecarSignatureFilter
- 3.1.1.9 9. DesiredPermissionsFilter
- 3.1.2 Egress Request Filters in details
- 3.1.2.1 1. SidecarSignatureRemover
- 3.1.1 Ingress request filters in details
- 3.1 Sidecar filters
- 4 Module Prefix strategies
- 5 Signing Key Rotation
- 6 Sidecar communication example
- 7 Deployment
- 7.1 Java-based Docker image
- 7.1.1 Minimal memory requirements
- 7.1.2 Algorithm for calculation
- 7.1.2.1 Example
- 7.1.3 Minimal CPU Requirements
- 7.2 Native Docker Image
- 7.1 Java-based Docker image
Overview
The Sidecar architectural pattern is a design pattern used in software engineering to enhance or extend the functionality of a main application by attaching a secondary application or service to it. This secondary application, known as the "sidecar," runs alongside the main application and provides supporting features such as monitoring, logging, configuration, or networking services.
In this pattern, the sidecar is tightly coupled with the main application, sharing the same lifecycle and always deployed together. However, the sidecar's runtime environment remains independent, allowing it to be developed and maintained separately from the main application.
The Sidecar pattern is commonly used in microservices architectures, where each microservice can have its sidecar to handle cross-cutting concerns. This allows developers to implement these concerns consistently and avoid duplicating code across services. It also simplifies the main application code by offloading auxiliary tasks to the sidecar.
GitHub repository: https://github.com/folio-org/folio-module-sidecar
folio-module-sidecar
uses Quarkus as a framework:
It allows building native image
(only non-FIPS-compliant docker images because some of the BouncyCastle FIPS libraries: Quarkus FIPS Security)FIPS-compliant
Lightweight and quick (compared to other solutions like Spring Boot Native Image, Envoy)
Java-based
Vertx under the hood, providing general router for ingress and egress requests and WebClient for external calls
folio-module-sidecar
provides the following functionality:
Ingress requests handling
incoming requests from API Gateway or another module to the current Folio moduleEgress requests handling
outcoming requests from the current Folio module to another Folio moduleAuthentication
based onx-okapi-token
orAuthorization
headerAuthorization
based on Keycloak policies and permissionsSelf-requests
based onx-okapi-sidecar-signature
header valueUser impersonation
Consortia support and cross-tenant requestsHTTP Transactional logging
Sidecar initializing and runtime
Bootstrap process
During startup, folio-module-sidecar
calls the following modules according to the sequence diagram:
Sidecar, as shown on the diagram, calls mgr-components to be part of the Eureka system:
It calls
mgr-applications
to construct egress and egress request caches, that store routing entries from module descriptor by id and for other related module descriptors, defined by optional and required interface dependencies.For egress requests there is a support to call API Gateway if the egress routing entry is not matched, it is controlled via environment variables:
SIDECAR_FORWARD_UNKNOWN_REQUESTS
andSIDECAR_FORWARD_UNKNOWN_REQUESTS_DESTINATION
which by default pointing to the Kong API Gateway.
It calls
mgr-tenant-entitlements
andmgr-tenants
to find out for which tenants sidecar is enabled, requests, containing tenant id, that is not cached in sidecar will be rejected with400 Bad Request Application is not enabled for tenant: {{tenantName}}
Sidecar Runtime
During runtime, folio-module-sidecar
receives events from the message bus (Apache Kafka) containing:
Tenant entitlement, revoke, and upgrade events. This information is used to update the active tenant's cache to support the following cases:
During the application uninstallation (revoke) - all related sidecars must be disabled for the affected tenant
During the application upgrade process - all upgraded (new) modules must be enabled for the tenant and all deprecated modules must be disabled for the affected tenant
Discovery information change information:
Sidecar can update an egress request location cache if discovery information is changed in
mgr-applications
Sidecar also caches authorized requests for the JWT using
session_state
claim if present until the token is expired:If a user performs logout or logout all operations -
folio-module-sidecar
can clean affected caches and forbid the next request using cached JWT
Sidecar filters
The filter approach is a way to intercept and process HTTP requests and responses before they reach a point when the request must be forwarded to the underlying Folio module. In request, they are responsible for all functionality and executed in chain.
Ingress request filters in details
1. SelfRequestFilter
Description
This filter is responsible for validating if a request is a module self-request. These types of requests are allowed without authentication and authorization.
Sidecar signature is unique for each sidecar and it’s based on the current time milli-seconds value defined during sidecar initialization, hashed using SHA-256 algorithm
Skip Conditions
Never
2. KeycloakSystemJwtFilter
Description
Retrieves and parses system token, issued by keycloak in another sidecar. This functionality was added to support module-to-module requests when the system user is not defined by the module or not all permissions are present in the existing user JWT, including tenant and module permissions.
System JWT is validated using publicly available JWKs public certificates, provided by Keycloak.
io.smallrye:smallrye-jwt
provides a JWTParser that is able to retrieve public keys automatically and supports the ability to cache and refresh JWKs. It’s controlled by environment variables KC_JWKS_REFRESH_INTERVAL
and KC_FORCED_JWKS_REFRESH_INTERVAL
This token is issued using client credential flow, Keycloak client is created during tenant and realm initialization in mgr-tenants
and controlled by the following environment variables:
KC_SERVICE_CLIENT_ID
Tenant-specific client ID for authenticating module-to-module requests.service client password is stored in SecureStore: AWS Parameter Store, HashiCorp Vault or provided via an ephemeral (dev mode) store configured by system properties.
Skip conditions
matched routing entry with
interfaceType
is equal tosystem
matched routing entry does not require any permission
a request does not contain a value in
X-System-Token
header
3. KeycloakJwtFilter
Description
Parses the JWT from x-okapi-token
or Authorization
headers.
User or system user JWT is validated using publicly available JWKs public certificates, provided by Keycloak.
io.smallrye:smallrye-jwt
provides a JWTParser that is able to retrieve public keys automatically and supports the ability to cache and refresh JWKs. It’s controlled by environment variables KC_JWKS_REFRESH_INTERVAL
and KC_FORCED_JWKS_REFRESH_INTERVAL
Also, support “dummy” token for mod-pubsub
:
When a module (for example mod-circulation) registers events in mod-pubsub the request is sent without x-okapi-token (as we don't pass it during tenant install), mod-pubsub-client lib adds a default "dummy" token to
such requests and therefore sidecar should be able to process such request.
Skip conditions
A routing entry matched with
interfaceType
is equal tosystem
, but interfaceid
is not equal to_timer
4. KeycloakTenantFilter
Description
Resolves a tenant name from system JWT and user JWT.
Allows cross-tenant requests, if environment variable
ALLOW_CROSS_TENANT_REQUESTS
for sidecar is set totrue
.Validates that
x-okapi-token
andx-system-token
are issued for the same tenant (skipped if #2 is defined)Validates that
x-okapi-tenant
is the same as resolved tenant id (skipped if #2 is defined)
Skip conditions
an interface
id
is not equal to_timer
interfaceType
is equal tosystem
Self request
Routing entry without required permissions
5. TenantFilter
Description
Check if a tenant in x-okapi-tenant
header is in the cache with active tenants.
Skip conditions
Tenant install request (interface
_tenant
v1.0, v1.1, or v2.0)
It is done to allow installation requests frommgr-tenant-entitlements
to enable modules, an event with the enabled application for a tenant is received only when a response is received from_tenant
API call
6. KeycloakImpersonationFilter
Description
Retrieves user from
mod-users-keycloak
If found the filter obtains an access token from Keycloak using impersonation: Server Administration Guide: Impersonating a user
Impersonation call is done using Keycloak token endpoint and grant_type equal tourn:ietf:params:oauth:grant-type:token-exchange
Sets the obtained impersonated JWT to
x-okapi-token
header
Skip conditions
The tenant name or JWT is not present in the request
A token issuer is equal to
x-okapi-tenant
header
7. KeycloakAuthorizationFilter
Description
Authorizes JWT in Keycloak using grant_type
equal to urn:ietf:params:oauth:grant-type:uma-ticket
and scope
equal to {routingEntryStaticPath}#{httpMethod}
Skip conditions
matched routing entry with
interfaceType
is equal tosystem
matched routing entry does not require any permission
8. SidecarSignatureFilter
Description
Adds a unique sidecar signature to x-okapi-sidecar-signature
header. This signature is used to identify self-request from module
Skip conditions
Never
9. DesiredPermissionsFilter
Description
Removes the
X-Okapi-Permissions
header from the request.Retrieves the
userId
from the request headers.If the
userId
is not found, it logs a debug message and returns the context as is.If the request has desired permissions, it fetches the user permissions using
mod-users-keycloak
and populates theX-Okapi-Permissions
header with these permissions.
Skip conditions
Never
Egress Request Filters in details
1. SidecarSignatureRemover
Description
Removes self-request signature from x-okapi-sidecar-signature
header
Module Prefix strategies
The module prefix strategies allow to adjust sidecar for different discovery URLs in the deployment
It can be defined using the environment variable: SIDECAR_MODULE_PATH_PREFIX_STRATEGY
with the following values: PROXY
, STRIP
, and NONE
PROXY
is used when routing between Kong, Sidecar, and Module requires a path prefix in the location URL:## Module sidecar location https://sidecar-foo.module-subnet.example.com/sc-foo ## Folio module location https://mod-foo.module-subnet.example.com/mod-foo
STRIP
is used when routing between Kong, Sidecar requires a path prefix in the location URL, but Sidecar and Module can be routed to each other using a private subnet:NONE
is used when routing between Kong, Sidecar, and Module does not require a path prefix in the location URL
Module prefix is based on the environment variable: MODULE_NAME
, which is necessary for sidecar
Signing Key Rotation
Sidecars must be aware of the new signing keys. This will be done automatically by calling “/protocol/openid-connect/certs
“ once again if it receives a token with a new signing key id. But it will be done only once per hour, as currently there is a line: jwtAuthContextInfo.setForcedJwksRefreshInterval($KC_FORCED_JWKS_REFRESH_INTERVAL);
which means that forced refresh (if the key is not found) will be performed only once per specified value in minutes (by default it is one hour).
This logic is provided by the following libraries:
User: The user initiates the process by logging into Keycloak.
Keycloak (keycloak): Keycloak authenticates the user and returns a JWT token with a new signing key ID.
User to Sidecar: The user requests "Module A" via the sidecar, providing the JWT token.
Sidecar Processing:
The sidecar checks the signing key ID in the JWT token.
If the signing key ID is unknown and the last JWKS refresh was over an hour ago, the sidecar fetches the new signing keys from Keycloak and updates its local cache. It then forwards the user's request to "Module A".
If the signing key ID is known, or if a JWKS refresh occurred within the last hour, the sidecar forwards the user's request to "Module A" without fetching new keys.
If the signing key ID is unknown and a JWKS refresh occurred less than one hour ago, the sidecar rejects the user's request to avoid too frequent key updates.
Module A (moduleA): "Module A" receives and processes the request forwarded by the sidecar.
Sidecar communication example
The example is based on the 2 simple modules in app-platform-minimal
- mod-notes
and mod-users
, when the user can create a note and mod-notes
will retrieve metadata from mod-users
Deployment
folio-module-sidecar
allows the building of a Docker container using OpenJDK base container or native image.
Java-based Docker image
Minimal memory requirements
RAM requirements are calculated from two things: java process RAM and containers OS ram usage. This values should be configured separately and container limit should include both values.
JAVA_OPTIONS
-XX:+UseZGC -Xms64m -Xmx64m -XX:MetaspaceSize=88m -XX:MaxMetaspaceSize=128m
Docker Container’s OS
128m
Algorithm for calculation
Module’s memory / 4 and it will be a Xms and Xmx numbers
Xms and Xmx numbers should not be less then 64m and not higher then 256m
Metaspace size should be 96m or higher
Container limit should be settled as Xms value + Container’s overhead value ((Module’s memory / 4) >64m <256m + 128m + 128m)
Example
mod-inventory 2048m RAM
Xmx and Xms value = 2048/4 = 512m
512m is higher than 256m, so the final value for Xmx and Xms will be 256m
JAVA_OPTIONS should be like
-XX:+UseZGC -Xms256m -Xmx256m -XX:MetaspaceSize=88m -XX:MaxMetaspaceSize=128m
Container limit should be 256m + 128m + 128m = 512m
Minimal CPU Requirements
CPU allocation depends on container’s load but not less then 0.2 CPU
Native Docker Image
To be implemented