KONG-39 folio-kong issue with LB on k8s
https://folio-org.atlassian.net/browse/KONG-39
- 1 Executive Summary
- 1.1 Dedicated Load Balancing (AWS ALB / Kubernetes NGINX) Existing aproach
- 1.2 Sidecar Load Balancing (Egress Requests)
- 1.3 Sidecar Load Balancing (Egress and Ingress Requests)
- 1.4 Kong Load-Balancing Approach
- 1.4.1 Kong Gateway supports managing multiple upstreams, health checks, and service discovery.
- 1.4.2 APISIX supports managing multiple upstreams, health checks, and service discovery.
- 1.4.3 Gloo (Gloo Gateway / Gloo Edge) provides the same core capabilities as Kong and APISIX in terms of upstream management, health checks, and service discovery.
- 1.4.4 Pros:
- 1.4.5 Cons:
Executive Summary
Purpose: Evaluate and recommend the optimal load balancing strategy for the FOLIO platform.
Decision Required: Select between dedicated, egress, or ingress load balancing architectures.
Key Trade-offs: Cost, Complexity, Performance, Cloud-Agnosticism.
Dedicated Load Balancing (AWS ALB / Kubernetes NGINX) Existing aproach
AWS
Kubernetis
Pros:
Out-of-the-box solution requiring no code changes.
Ensures reliable load distribution.
Cons:
Higher infrastructure cost and added operational complexity.
AWS ALB is not a cloud-agnostic solution.
Introduces a single point of failure and potential additional latency due to proxying requests.
Sidecar Load Balancing (Egress Requests)
AWS
Kubernetis
Pros:
Reduces cost by eliminating external load-balancing services (e.g., AWS ALB, Kubernetes NGINX).
Manageable implementation effort — all changes are contained within Eureka components.
Platform-agnostic solution.
Cons:
Imperfect traffic distribution — multiple sidecar instances may route to the same target instance.
Requires an additional discovery mechanism or service.
Sidecar Load Balancing (Egress and Ingress Requests)
AWS
Kubernetis
Pros:
Provides efficient and even load distribution (comparable to dedicated load balancers).
Significantly reduces costs by removing AWS ALB and decreasing the number of sidecar instances (one per service instead of per module).
Enables fully distributed traffic flow — no centralized point of failure.
Platform-agnostic solution.
Cons:
Requires significant code refactoring, including modifications to module logic (can be mitigated through a shared/common library).
Requires an additional discovery mechanism or service.
Kong Load-Balancing Approach
To avoid additional development and reduce the effort required on the sidecar, we can use an API gateway–based load-balancing approach where all traffic flows through a central gateway. This allows us to reuse the existing component without introducing further complexity.
Kubernetis
Kong Gateway supports managing multiple upstreams, health checks, and service discovery.
1. Multiple Upstreams Support
Kong’s core concept of an Upstream is a DNS-style pool representing backend targets (nodes/instances).
Each Upstream can have multiple targets (hosts/ports). Traffic is load-balanced among healthy targets.
You can define as many Upstreams as you need, and each route/service can refer to an Upstream for balancing.
2. Upstream Health Checks
Kong supports both active and passive health checks for upstream targets:
Active: Kong periodically sends HTTP or TCP probes to check target health. Unhealthy targets are automatically removed from the load-balancing ring.
Passive: Kong observes responses to real API traffic and marks targets unhealthy on failures/timeouts.
Health check results can be viewed via the admin API and dashboards.
3. Service Discovery Integration
Kong can integrate with service discovery systems like Consul, AWS (with Kong Konnect), Kubernetes, and DNS:
With plugins and configuration, Kong will dynamically discover, register, and deregister upstream targets as they are added/removed by the service registry.
For Consul and DNS-based discovery, Kong natively refreshes the backends as records/services in Consul and DNS updates.
APISIX supports managing multiple upstreams, health checks, and service discovery.
As an alternative, we are evaluating APISIX as our primary API gateway. I reviewed its capabilities to ensure it provides the same features as Kong, and I can confirm that APISIX fully supports all required functionality:
1. Multi-protocol Upstream Support
APISIX uses upstreams as backend service pools, supporting both HTTP(S) and TCP/UDP (stream) services.
This allows load balancing across multiple protocols—including Web, gRPC, and raw TCP—through a unified upstream abstraction.
2. Upstream Health Checks
APISIX natively supports both active and passive health checks.
You can configure health-check rules per upstream (intervals, types, thresholds).
Health checks work for HTTP(S), gRPC, and TCP, and any failed nodes are automatically removed from the load-balancing rotation.