https://folio-org.atlassian.net/browse/KONG-38

API Gateway Comparison: APISIX vs Gloo vs Kong - Decision Matrix for FOLIO Platform

Executive Summary

Purpose: This document provides a comprehensive architectural comparison, financial analysis, and strategic recommendation for the selection of the primary API gateway for the FOLIO Library Services Platform (LSP).

Scope: This report compares three market-leading solutions:

1. Kong Gateway (Baseline): Current solution for EUREKA, built on NGINX and OpenResty.

2. Apache APISIX: A high-performance, dynamic, cloud-native gateway also based on NGINX/OpenResty but utilizing etcd for configuration.

3. Gloo Edge : A Kubernetes-native gateway built on Envoy Proxy, representing the "next generation" of data plane technologies.

Key Findings Summary

Strategic Recommendation Preview:

While Kong is the "safe" default due to current integration efforts, Apache APISIX is recommended as the strategic alternative for the FOLIO platform. It offers the optimal balance of performance, cost-efficiency, and migration feasibility. It shares Lua DNA with Kong, allows for the portability of critical FOLIO logic (such as the auth-headers-manager plugin), while its etcd-backed architecture solves the configuration propagation latency issues inherent in the current Postgres-backed Kong architecture.

Introduction and Architectural Context

Evaluation Criteria and Methodology

The Architecture has defined a specific set of weighted criteria for this decision, reflecting the unique constraints of the FOLIO community (which includes commercial hosting providers like EBSCO and Index Data, as well as self-hosting universities like Stanford).

1. Performance & Scalability (30%): The gateway must minimize latency overhead. Crucially, it must support rapid "route convergence." In a multi-tenant environment where tenants (libraries) are onboarded dynamically, the time between a configuration change and its propagation to all gateway nodes must be near-instantaneous to prevent routing errors.

2. Cost Efficiency (25%): As an open-source project, FOLIO must remain cost-effective for self-hosting institutions. However, commercial hosting providers need enterprise features (high availability, support). The TCO analysis must account for both licensing fees and the "hidden" costs of infrastructure (e.g., database maintenance).

3. FOLIO Compatibility (20%): The solution must support existing FOLIO paradigms, specifically the propagation of the X-Okapi-Tenant header and the ability to port the custom logic currently residing in folio-kong.

4. Operational Complexity (15%): The ease of management, upgrades, and observability is critical. This includes the complexity of the backing store (e.g., managing a Postgres cluster vs. using Kubernetes-native storage).

5. Community & Ecosystem (10%): The long-term viability of the project, the health of its open-source community, and the availability of plugins are essential to prevent vendor lock-in.

Solution Deep Dive: Architectural & Functional Analysis

Baseline: Kong Gateway

Kong is the current solution for the Eureka platform. The folio-org/folio-kong repository contains a customized Docker image that layers FOLIO-specific configurations on top of the official Kong image.

Architecture and Data Plane:

Kong is built on OpenResty, which combines the NGINX proxy with the LuaJIT (Just-In-Time) compiler. This architecture allows developers to write plugins in Lua that hook into the NGINX request lifecycle. In the standard deployment model, Kong relies on a relational database— PostgreSQL to store configuration data (routes, services, plugins).

Configuration Propagation (The Polling Model):

A critical architectural characteristic of Kong is its polling mechanism. Kong nodes periodically poll the PostgreSQL database to check for configuration changes. By default, this polling interval creates a window (typically 5-10 seconds) during which different nodes in a cluster may be serving different configurations. In static environments, this is acceptable. However, for a multi-tenant platform like FOLIO where tenant provisioning and module enabling happen dynamically via API calls, this lag can result in transient 404 errors or misrouted traffic immediately after a configuration update.

FOLIO Integration Status:

The folio-kong repository is not a configuration wrapper; it contains active logic.

• auth-headers-manager: This custom component (likely implemented as a script or plugin) manages the translation of Keycloak tokens or legacy Okapi headers into the format backend modules expect. It bridges the gap between the new OIDC-based world of Keycloak and the legacy expectations of FOLIO modules.

Pros for FOLIO:

• Existing Investment: the folio-kong container exists and is functional within the Eureka bootstrap process.

• Developer Familiarity: NGINX is a ubiquitous technology. Many DevOps engineers in the library technology space are familiar with NGINX syntax and Kong's declarative configuration tool.

• Plugin Ecosystem: Kong boasts a massive library of over 50+ official plugins and hundreds of community contributions, covering everything from rate limiting to OIDC authentication.

Cons for FOLIO:

• Reconfiguration Lag: The polling-based architecture introduces a convergence delay that contradicts the "real-time" requirements of modern SaaS platforms.

• Licensing Barriers: The main contributor to Kong OSS, Kong inc. has decided to largely stop participating in the open source community. No new features, or binary distributions, less frequent patching of security vulnerabilities, etc.

Apache APISIX

APISIX is a cloud-native API gateway built on OpenResty (NGINX + LuaJIT), but it follows a different architectural philosophy from Kong, particularly in how it manages configuration and performance.

Architecture and Data Plane

Like Kong, APISIX relies on NGINX and LuaJIT. However, it differs fundamentally in its choice of control-plane backing store. APISIX uses etcd—the same distributed key-value store used by Kubernetes—for storing and synchronizing configuration.
However, Kubernetes does not permit using its internal etcd outside the control plane, meaning APISIX cannot rely on the cluster’s native etcd. As a result, APISIX requires running its own dedicated etcd deployment inside the cluster, typically as a separate set of pods.

Configuration Propagation (Event-Driven Model)

Instead of polling a database, APISIX establishes a continuous watch on its etcd cluster. When configuration updates occur in etcd, they are pushed immediately—with millisecond-level latency—to all gateway nodes.
This event-driven model enables near-instant route convergence, making APISIX highly suitable for dynamic environments where tenants and routes are created frequently.

FOLIO Fit and Lua Compatibility:

• Lua Portability: Since APISIX is also Lua-based, the logic inside folio-kong (specifically the auth-headers-manager) can be ported with minimal refactoring. The underlying NGINX primitives (ngx.ctx, ngx.header) are identical, reducing the migration risk significantly compared to non-Lua gateways.

• Multi-Tenancy: APISIX utilizes a radix tree for route matching, which is algorithmically superior to linear regex matching for large routing tables. This makes it exceptionally well-suited for FOLIO's multi-tenant model, where the number of routes scales linearly with the number of tenants and modules.

Pros for FOLIO:

• Performance: Synthetic benchmarks and architectural analysis consistently show APISIX delivering lower latency and higher throughput than Kong, particularly as the complexity of the routing table grows.

Cons for FOLIO:

• Migration Effort: While tools like kong-to-apisix exist to automate the translation of declarative configurations, manual verification and testing are still required.

• Community Maturity: While growing rapidly (approaching 15k GitHub stars), the APISIX community is younger than Kong's (39k+ stars), meaning fewer StackOverflow answers and third-party tutorials are available.

Gloo Edge

Gloo Edge represents the "modern" wave of gateways, built on Envoy Proxy rather than NGINX. It markets itself as a "Kubernetes-native" gateway, tightly integrating with the cluster's own configuration mechanisms.

Architecture and Data Plane:

Gloo Edge uses Envoy Proxy as its data plane. The control plane (Gloo) runs as a Kubernetes deployment, watching for Custom Resource Definitions (CRDs) like VirtualServices and RouteTables, and translating them into Envoy's xDS configuration protocol.

Configuration and Kubernetes Integration:

Gloo Edge is configured entirely through Kubernetes CRDs. This eliminates the need for a separate database or even a separate etcd instance (it leverages the Kubernetes cluster's existing etcd). This provides a "GitOps-ready" experience out of the box, where gateway configuration lives alongside application manifests.

FOLIO Fit:

• Protocol Support: Gloo Edge (via Envoy) offers best-in-class support for gRPC, HTTP/2

• WASM Extensibility: Custom logic like auth-headers-manager would essentially need to be rewritten. While Envoy supports Lua filters, the ecosystem pushes toward WebAssembly (WASM) using languages like Rust, Go, or C++. This represents a significant paradigm shift and a high-friction migration point for the FOLIO team, which is currently invested in Lua.

Pros for FOLIO:

• Kubernetes Native: For a platform running on K8s (like Eureka), Gloo feels like a natural extension of the cluster. There is no separate database to manage, back up, or secure.

Cons for FOLIO:

• High Complexity: Envoy’s configuration is significantly more complex than NGINX, and troubleshooting xDS synchronization or filter chain behavior requires specialized expertise that the FOLIO community may not currently have.

• Plugin Rewrites: The existing folio-kong Lua logic cannot simply be copy-pasted. It must be re-engineered, likely requiring a new development lifecycle for WASM modules.

Comprehensive Comparison Matrix

Core Functionality & Architecture

Performance Benchmarks (Synthetic)

Based on architectural analysis and aggregated industry benchmarks from multiple sources :

Community & Ecosystem

Infrastructure Cost Analysis

To fully understand the financial implications, we must examine the “hidden” costs associated with the supporting infrastructure for each gateway.

Kong:
Requires a highly available PostgreSQL database. On AWS, this typically means an RDS Multi-AZ deployment.

• Cost: An RDS db.t3.large Multi-AZ instance is approximately $130/month, plus storage and provisioned IOPS.

• Administration: Adds the need for database backups, point-in-time recovery configuration, and complex schema migrations during gateway upgrades.

APISIX:
Requires an etcd cluster.

• Cost: Etcd is extremely lightweight. It can often run in small instances or even as co-located containers. It consumes significantly fewer IOPS than a relational database like PostgreSQL.

• Administration: Snapshotting etcd is available, and since etcd is already used in Kubernetes and AWS infrastructure, operations teams may already be familiar with its use/maintenance.

Gloo Edge:
Uses Consul or etcd plus additional components, such as the Discovery Service and Proxy Plane, all of which must scale independently.

• Cost: Significantly more components must be hosted and maintained to make the API gateway fully operational.

• Risk: If any of these components fail, the entire gateway can become unstable or non-functional, increasing operational risk and reducing reliability.

Migration Complexity Assessment

The folio-kong Hurdle

The existence of the folio-kong repository creates a specific migration challenge. It is not just a configuration repository; it contains custom customization logic.

• The Challenge: The auth-headers-manager plugin likely intercepts requests to inspect tokens and inject headers required by the sidecars or backend modules. This logic is crucial for the platform's security model.

• Kong -> APISIX Migration:

  • Complexity: Low/Medium.

  • Reasoning: Both gateways use Lua. The existing code in folio-kong can likely be copied and adapted with minimal changes to fit APISIX's Lua object model (e.g., changing kong.request.get_header to ngx.req.get_headers). Furthermore, tools like kong-to-apisix are available to automate the translation of the static route configurations.

• Kong -> Gloo Migration:

  • Complexity: High.

  • Reasoning: This migration path would require rewriting the Lua logic entirely. While Envoy supports Lua, the ecosystem strongly favors WebAssembly (WASM) for performance and compatibility. This would necessitate rewriting the auth logic in Rust or Go, introducing a completely new development lifecycle, build pipeline, and testing framework.

FOLIO-Specific Considerations

FIPS Compliance

For deployments in government or regulated libraries (e.g., Library of Congress or university libraries with federal contracts), FIPS compliance may be a requirement.

• Kong Enterprise: Offers FIPS-compliant builds for the enterprise version

• Gloo Edge Enterprise: Offers FIPS-compliant builds for the enterprise version

• APISIX open source can be configured for FIPS compliance (FIPS 140-2 Level 1), but it is not FIPS certified “out-of-the-box”. You must specifically build and run it with a FIPS-validated version of OpenSSL operating in FIPS mode

Decision Framework & Recommendations

Weighted Scoring Matrix

Strategic Recommendation

Primary Recommendation: Migrate to Apache APISIX.

Rationale:

1. Performance Alignment with Eureka Goals: The Eureka initiative is fundamentally about modernizing FOLIO for scale. APISIX's sub-millisecond latency and instant configuration propagation align perfectly with the requirements of a modern, dynamic, multi-tenant platform.

2. Low-Risk Migration: Since both Kong and APISIX share the OpenResty/Lua DNA, the custom folio-kong logic is portable with minimal effort. The team does not need to be retrained on Envoy/WASM or rewrite critical auth logic in a new language.

3. Vendor Neutrality: As a top-level Apache Software Foundation project, the core gateway code remains truly open.

Not Recommended: Gloo Edge.

While Gloo Edge is technologically advanced and highly "Kubernetes-native," the complexity tax of managing Envoy configurations and the high effort required to rewrite FOLIO's custom auth logic into WASM or Go make it the least attractive option for this specific migration scenario.

Migration: The Kong-to-apisix Tool

The existence of the Kong-to-APIsix tool is a significant de-risking factor for this recommendation. This open-source tool automates the translation of:

• Services -> Upstreams

• Routes -> Routes

• Consumers -> Consumers

• Plugins (Rate limiting, proxy-cache, key-auth)

This tool drastically reduces the "Route Migration" phase from weeks to days. It effectively "reads" the Kong declarative configuration and "writes" the equivalent APISIX configuration. No comparable tool exists for migrating Kong configurations to Gloo Edge CRDs, making the APISIX path significantly smoother.