Page Comparison

...

Spike Overview

User Story: https://folio-org.atlassian.net/browse/EUREKA-211 Spike - Revisit routing of mod-scheduler requests

...

• Multiple ways for sidecars to get discovery information when “Dynamic Routing“ is enabled.

• Multiple modules for dynamic routing resolution increase the overall complexity of the system.

• Dynamic routing can potentially introduce additional latency due to the extra steps required to resolve discovery info. The usage of the cache should mitigate the issue.

• mgr-discovery becomes a critical component. if it fails, it could disrupt the entire routing and bootstrapping mechanism. HA and redundancy should be used to mitigate this.

Option 3.2

...

• mod-scheduler becomes aware of mgr-* components, which breaks the separation of concerns and potentially introduces tighter coupling between these components.

• Increases complexity in mod-scheduler as it now needs to handle route resolution logic.

• Could potentially increase the load on mod-scheduler for frequent route resolutions.

• Deviates from the standard pattern where modules interact with the system through their Sidecars.

Option 3.3

...

1. Bootstrap Timer Route Loading: During the bootstrap phase, the Sidecar loads all timer routes from mgr-discovery, which compiles this information from mgr-tenant-entitlements and mgr-applications.

2. Static Routing: The Sidecar uses the pre-loaded routing information for timer requests, eliminating the need for dynamic routing resolution during runtime.

3. Lightweight mgr-discovery: The mgr-discovery module primarily serves bootstrap information and doesn't need to handle runtime route resolution requests.

Pros:

• Simplifies runtime routing by eliminating the need for dynamic resolution.

• Potentially improves performance by reducing the number of interactions required for each timer request.

• Reduces load on mgr-discovery during normal operation.

Cons:

• Lacks a clear mechanism for updating routing information when changes occur (e.g., during the entitlements process for different tenants and applications).

• May lead to outdated routing information if changes occur after the bootstrap phase.

• Increases memory usage in Sidecars as they need to store all timer routes.

• Less flexible than dynamic routing options when dealing with frequent changes or a large number of routes.

...

Resolving this gap is crucial for ensuring that the Sidecars always have up-to-date routing information, especially in a dynamic multi-tenant environment where entitlements may change frequently.

Option 3.4

Description

In this option, the mgr-discovery component is removed, and its functionality is fully integrated into the Sidecar. This allows the Sidecar to handle all dynamic route discovery, caching, and forwarding processes internally. The architecture simplifies the overall system by reducing external dependencies while still enabling dynamic routing through an explicit configuration mechanism.

A key aspect of this option is the introduction of a Dynamic Routing Enabled/Disabled switch within the Sidecar. This switch must be explicitly enabled via a configuration parameter for the Sidecar to support dynamic routing. When dynamic routing is disabled, the Sidecar relies on preloaded configured routes. When enabled, the Sidecar dynamically discovers and routes requests using the internal cache and communication with external components, such as mgr-Tenant-Entitlements and mgr-Applications if a route is not present in the preconfigured set.

Components Involved:

1. mod-Scheduler: Handles timer-based jobs that initiate requests for various timers interfaces.

2. Sidecar (mod-Scheduler): Now equipped with additional functionality for dynamic routing. It manages route discovery, caching with expiration policies, and directly interacts with other system modules.

3. Sidecar (Module A): Receives the routed request from the Sidecar (mod-Scheduler) and forwards it to the appropriate internal modules after performing authorization.

4. Keycloak: Provides authorization services, ensuring that requests are authenticated with a system token. The system user has access to all resources.

5. mgr-Tenant-Entitlements: Provides module version information required for routing.

6. mgr-Applications: Supplies the discovery information necessary for dynamic routing and module interactions.

7. mod-Users-Keycloak: Manages the creation and enablement of system users for authentication purposes.

Flow:

1. The mod-Scheduler prepares a request (POST to a timer URL) and includes an impersonated token for a system user in the request header. It sends this request to the Sidecar (mod-Scheduler).

2. The Sidecar (mod-Scheduler) checks its internal Routes Cache for the required route. If the route information is available and still valid, it proceeds to step 5. If the route is expired or missing, it dynamically resolves the route by retrieving data from external modules.

3. The Sidecar (mod-Scheduler) calls mgr-Tenant-Entitlements to resolve the version of the module enabled for a current tenant (Module A) that has timers associated with the requested URL.

4. The Sidecar (mod-Scheduler) then queries mgr-Applications to obtain the actual route for the resolved module version.

5. Once the route is determined, the Sidecar (mod-Scheduler) forwards the request to the appropriate Sidecar (Module A).

6. The Sidecar (Module A) verifies the request by retrieving and validating the system token. It uses Keycloak to evaluate the permissions, confirming that the system user has access to all resources.

7. After successful authorization, the Sidecar (Module A) forwards the request to Module A for further processing.

Key Features:

1. Internal Routes Cache with Expiration Policy: The Sidecar (mod-Scheduler) maintains a cache of routes with an expiration policy to ensure up-to-date and valid routing information. When a route expires, the Sidecar automatically refreshes the route information by querying mgr-Tenant-Entitlements and mgr-Applications.

2. Dynamic Route Discovery: The Sidecar is responsible for dynamically resolving routes for module versions and interfaces by querying the required information from mgr-Tenant-Entitlements and mgr-Applications. This eliminates the need for external mgr-discovery.

3. Authorization via Keycloak: All interactions with Module A are secured using Keycloak, ensuring that only authenticated system users with appropriate permissions can access the resources.

4. Self-Sufficient Sidecar: The Sidecar (mod-Scheduler) is now a self-sufficient component that handles all aspects of route discovery, caching, and dynamic routing without relying on external components. This reduces system complexity and increases the Sidecar's autonomy in managing requests.

Pros:

1. Reduced System Complexity: By eliminating the mgr-discovery component and integrating its functionality into the Sidecar, the overall system architecture is simplified. This reduces the need for external dependencies and minimizes potential points of failure.

2. Centralized Routing Logic: The Sidecar now handles all aspects of routing, including dynamic route resolution and caching, reducing the need for multiple components and simplifying management.

3. Improved Performance: The internal route cache with an expiration policy allows the Sidecar to quickly resolve frequently used routes, reducing the latency associated with querying external components repeatedly.

4. Greater Autonomy for Sidecars: Each Sidecar becomes more self-sufficient, handling route discovery and caching independently, which can make the system more scalable and less reliant on centralized routing services.

5. Easier Maintenance: With fewer moving parts (no need for a separate mgr-discovery component), maintaining and troubleshooting the system becomes easier, with a more focused logic inside the Sidecar.

Cons:

1. Increased Sidecar Complexity: By integrating route discovery and caching functionalities into the Sidecar, the component becomes more complex to develop, maintain, and debug. This increased complexity could lead to longer development cycles and more potential bugs.

2. Higher Memory and Resource Usage: The Sidecar now stores route information in a cache, potentially increasing its memory footprint. For systems with many routes, this could require more memory and CPU resources to manage effectively.

3. Potential Cache Staleness: While the cache has an expiration policy, there is still a risk that the route information could become outdated before the cache refreshes. This could lead to routing errors until the cache is updated.

4. Lack of Centralized Route Management: Since each Sidecar now independently handles route discovery, ensuring consistency across different instances of Sidecars could be more challenging, especially in multi-tenant environments.

5. Greater Dependency on Sidecar Robustness: The Sidecar becomes a critical point of failure. If a Sidecar encounters an issue, routing could be disrupted for that particular service, affecting its ability to process requests effectively.

6. Potential Latency from Cache Expiration: When the cache expires, the Sidecar needs to query external modules to refresh routing information. This could introduce additional latency during cache refreshes, particularly in high-traffic deployments.

Conclusion

During several rounds of discussion, option 3.4 was selected because it best suits the requirements. We’d like to consolidate the mechanisms used by the sidecars to obtain discovery information.

Spike Status: