/
KONG-35 PoC/Deep Dive: Apache APISIX

KONG-35 PoC/Deep Dive: Apache APISIX

https://folio-org.atlassian.net/browse/KONG-35

APISIX with PostgreSQL Backend: Production Implementation Guide

Approaches Evaluated

This guide evaluates two primary approaches for implementing PostgreSQL as APISIX's configuration backend:

  1. Kine-based Implementation: Uses the proven K3s etcd translation layer

  2. etcd-adapter Implementation: API7's APISIX-specific adapter solution

Production Readiness Assessment

Approach

Status

Recommendation

Approach

Status

Recommendation

Kine

Production-ready

✅ Recommended - Proven in K3s production environments

etcd-adapter

Experimental

⚠️ Not recommended - Incomplete feature set, known stability issues

Recommended Approach

Kine is the recommended approach for production deployments due to its proven stability in K3s environments and comprehensive etcd API compatibility.

High-Level Architecture

The implementation introduces a translation layer between APISIX and PostgreSQL:

Approach 1: Kine Implementation

Overview

Kine (Kine Is Not etcd) is an etcd API compatibility layer that translates etcd v3 API calls into SQL operations for various databases, including PostgreSQL. Originally developed for K3s, it has proven production-ready through widespread deployment in resource-constrained environments.

Production Readiness Assessment

✅ Production-Ready

  • Proven Track Record: Core component of K3s, used in millions of production deployments

  • Active Development: Maintained by the CNCF and Rancher/SUSE with regular updates

  • Comprehensive Testing: Extensively tested through K3s production usage

  • Performance Validated: Handles Kubernetes workloads efficiently with minimal overhead

Architecture Diagram

Docker Compose Configuration

version: '3.8' services: postgres: image: postgres:14-alpine environment: POSTGRES_DB: kine POSTGRES_USER: kine POSTGRES_PASSWORD: ${PG_PASSWORD:-secure_password} POSTGRES_INITDB_ARGS: "--encoding=UTF-8 --lc-collate=C --lc-ctype=C" volumes: - postgres_data:/var/lib/postgresql/data - ./init-db.sql:/docker-entrypoint-initdb.d/init-db.sql ports: - "5432:5432" healthcheck: test: ["CMD-SHELL", "pg_isready -U kine -d kine"] interval: 10s timeout: 5s retries: 5 restart: unless-stopped kine: image: rancher/kine:v0.11.9 command: - kine - --endpoint=postgres://kine:${PG_PASSWORD:-secure_password}@postgres:5432/kine?sslmode=disable - --listen-address=0.0.0.0:2379 ports: - "2379:2379" depends_on: postgres: condition: service_healthy healthcheck: test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:2379/health"] interval: 10s timeout: 5s retries: 5 restart: unless-stopped apisix: image: apache/apisix:3.4.0-debian volumes: - ./apisix-config.yaml:/usr/local/apisix/conf/config.yaml:ro ports: - "9080:9080" - "9180:9180" - "9443:9443" depends_on: kine: condition: service_healthy environment: - APISIX_STAND_ALONE=false restart: unless-stopped volumes: postgres_data:

APISIX Configuration (apisix-config.yaml)

deployment: role: traditional role_traditional: config_provider: etcd admin: allow_admin: - 0.0.0.0/0 admin_key: - name: "admin" key: edd1c9f034335f136f87ad84b625c8f1 role: admin etcd: host: - "http://kine:2379" prefix: "/apisix" timeout: 30 apisix: node_listen: 9080 enable_admin: true admin_listen: ip: 0.0.0.0 port: 9180

Database Initialization (init-db.sql)

-- Create optimized indexes for Kine operations CREATE INDEX IF NOT EXISTS kv_name_index ON kv(name); CREATE INDEX IF NOT EXISTS kv_name_id_index ON kv(name, id); CREATE INDEX IF NOT EXISTS kv_id_deleted_index ON kv(id, deleted); -- Optimize PostgreSQL settings for Kine workload ALTER SYSTEM SET shared_preload_libraries = 'pg_stat_statements'; ALTER SYSTEM SET track_io_timing = on; ALTER SYSTEM SET log_min_duration_statement = 1000;

Production Deployment Considerations

High Availability Setup

# docker-compose-ha.yaml version: '3.8' services: postgres-primary: image: postgres:14-alpine environment: POSTGRES_DB: kine POSTGRES_USER: kine POSTGRES_PASSWORD: ${PG_PASSWORD} POSTGRES_REPLICATION_USER: replicator POSTGRES_REPLICATION_PASSWORD: ${REPLICA_PASSWORD} volumes: - ./postgresql-primary.conf:/etc/postgresql/postgresql.conf - postgres_primary_data:/var/lib/postgresql/data command: postgres -c config_file=/etc/postgresql/postgresql.conf postgres-replica: image: postgres:14-alpine environment: PGUSER: ${POSTGRES_USER:-kine} POSTGRES_PASSWORD: ${PG_PASSWORD} POSTGRES_MASTER_USER: replicator POSTGRES_MASTER_PASSWORD: ${REPLICA_PASSWORD} POSTGRES_MASTER_HOST: postgres-primary POSTGRES_MASTER_PORT: 5432 volumes: - postgres_replica_data:/var/lib/postgresql/data depends_on: - postgres-primary kine-1: image: rancher/kine:v0.11.9 command: - kine - --endpoint=postgres://kine:${PG_PASSWORD}@postgres-primary:5432/kine?sslmode=require - --listen-address=0.0.0.0:2379 ports: - "2379:2379" kine-2: image: rancher/kine:v0.11.9 command: - kine - --endpoint=postgres://kine:${PG_PASSWORD}@postgres-primary:5432/kine?sslmode=require - --listen-address=0.0.0.0:2379 ports: - "2380:2379"

Conclusion

Unfortunately, APISIX was able to start successfully and create schemas in the PostgreSQL database, but when I began migrating routes from Kong, it started failing. The logs showed many errors where APISIX attempted to call the /version endpoint on etcd, but Kine does not support this endpoint at all. This indicates that not all etcd APIs are implemented, meaning there is no guarantee that APISIX will work properly in this setup.

Approach 2: etcd-adapter Implementation

Overview

The etcd-adapter is an APISIX-specific project developed by API7 that wraps the Kine library with additional APISIX-focused optimizations. However, it remains in experimental status with significant limitations.

Production Readiness Assessment

⚠️ Not Production-Ready

  • Experimental Status: Explicitly marked as experimental by API7

  • Incomplete Feature Set: Does not support all etcd v3 API features required by APISIX

  • Known Issues: Reports of basic operations failing in testing environments

  • Limited Support: Not donated to the Apache Foundation, indicating maturity concerns

Architecture Diagram

Known Limitations

  1. Feature Incompleteness: Missing critical etcd v3 API features

  2. Stability Issues: Reports of "Key not found" and "404 Route Not Found" errors

  3. Limited Database Support: No official PostgreSQL support (MySQL only)

  4. Maintenance Concerns: Limited development activity and community support

Evaluate the Ability to Handle Routes with Expressions

Since we use route expressions in Kong (including complex rules like regular expressions), we need to ensure they can be imported into APISIX and continue to work as expected.

During my investigation, I noticed that APISIX provides similar functionality, but it refers to them as vars. After a deeper dive, I found that both expressions (in Kong) and vars (in APISIX) are based on the NGINX core functionality, which underpins both Kong and APISIX.

This means that if we decide to migrate to another gateway in the future, having routes defined in a way that aligns with NGINX fundamentals could give us better portability.

Example of a Route Using vars Migrated from Kong to APISIX

{ "name": "2960c2388f72a5a5e6e4898563844a143e172eea", "uri": "/*", "vars": [ ["request_uri", "~~", "^/change-manager/jobExecutions/([^/]+)/records$"], ["request_method", "==", "DELETE"], ["http_x_okapi_tenant", "~~", ".*"] ], "upstream": { "scheme": "http", "type": "roundrobin", "nodes": { "mod-source-record-manager:8082": 1 } } }

Migration Script and Results

I wrote a Python script that migrated all routes from the Spring testing environment into my local APISIX instance.
Below is an excerpt of the console output from the migration process. The script is also attached to this page and can be used as a starting point for building a production-ready migration tool if we decide to proceed with APISIX.

Starting Kong to APISIX migration... Processing Kong Route ID: 000bb679-6730-4199-9d75-587b88909f37 (Name: e463f7672058f2c72e73c4cb12c801196c40e5a4) -> Found expression: (http.path ~ "^/transfer-criterias/([^/]+)$" && http.method == "PUT" && http.headers.x_okapi_tenant ~ r#"."#) -> Translated to APISIX vars: [["request_uri", "", "^/transfer-criterias/([^/]+)$"], ["request_method", "==", "PUT"], ["http_x_okapi_tenant", "", "."]] -> APISIX Payload: { "name": "e463f7672058f2c72e73c4cb12c801196c40e5a4", "id": "000bb679-6730-4199-9d75-587b88909f37", "vars": [ ["request_uri", "", "^/transfer-criterias/([^/]+)$"], ["request_method", "==", "PUT"], ["http_x_okapi_tenant", "", "."] ], "uris": ["/"], "upstream": { "scheme": "http", "type": "roundrobin", "nodes": { "mod-feesfines:8082": 1 } } } -> Successfully migrated route 000bb679-6730-4199-9d75-587b88909f37. Status: 201


Community Health Analysis – Apache APISIX

Current Governance Model

  • Apache APISIX is a top-level Apache Software Foundation (ASF) project, donated by API7.ai 2019.

  • ASF governance provides safeguards against unilateral changes to licensing.

  • However, http://API7.ai remains the dominant contributor and the original creator of the project.

API7.ai ’s Role and Influence

  • API7 was the original developer and remains a major contributor.

  • The company invests heavily in talent and resources to support Apache APISIX.

  • API7 maintains a clear commercial strategy with enterprise and cloud offerings that extend the open-source version.

Community Contribution Patterns

  • Contributions reportedly come from hundreds of companies worldwide.

  • APISIX is described as one of the most active API Gateway open-source projects.

  • However, detailed contribution metrics comparing API7 to the broader community are not publicly available.

Enterprise / OSS Split – Risk Assessment

Positive Indicators

  • Apache governance protects against sudden license changes.

  • Strong enterprise adoption across well-known organizations.

  • API7 emphasizes the importance of an open and diverse community.

Warning Signs

  • API7 maintains significant control as the founder and primary investor.

  • Commercial offerings introduce exclusive enterprise-only features.

  • The business model depends on monetizing APISIX through enterprise products.

Recommendation – Moderate Risk

While Apache governance provides stability, API7’s dominant position introduces moderate long-term risk.

Actions to consider:

  • Monitor contribution diversity and track growth of non-API7 participants.

  • Watch for feature divergence between open-source and enterprise versions.

There is no immediate risk due to ASF governance, but the concentration of control and commercial strategy requires ongoing monitoring.

Migration Summary: Kong to APISIX Auth Headers Manager

This chapter summarizes the steps, main deliverables, and testing approach for migrating the auth-headers-manager Kong plugin to Apache APISIX.

Accomplishments

  • Migrated the original Kong auth-headers-manager plugin, preserving the logic for converting folioAccessToken cookies into either X-Okapi-Token or Authorization headers.

  • Implemented a new APISIX-compatible plugin module (auth-headers-manager.lua) using the APISIX core plugin API, optimized for use in the rewrite phase.

  • Dockerized the solution for easy installation/testing and reliable operation within modern API gateway stacks.

  • Developed automation scripts for the setup and validation of plugin functionality.

Key Files

  • Plugin Implementation:
    apisix-auth-headers-manager/plugins/auth-headers-manager.lua

  • Testing Script:
    test-plugin.sh

  • Docker Deployment Example:
    docker-compose.apisix.yml
    with APISIX configuration in apisix-config.yaml

Usage and Testing

  1. Install and Start:

    • Build and run all services with Docker Compose.

  2. Verify:

    • Use setup.sh to configure the APISIX test environment.

    • Run the provided test-plugin.sh script to send requests and validate plugin behavior, covering:

      • Cookie to header transformation

      • Header validation

      • Consistency checks and error conditions

Main Test Script (test-plugin.sh excerpt):

bash #!/bin/bash set -e GATEWAY_URL=http://localhost:9080 TEST_TOKEN=eyJhbGciOiJIUzI... echo "Testing APISIX Auth Headers Manager Plugin" # Test scenarios for the auth-headers-manager plugin test_request() { local description="$1" local curl_args="$2" echo "Test: $description" echo "Command: curl $curl_args" response=$(eval curl -s -w '\n--- Response Info ---\nhttp_code: %{http_code}\ntime: %{time_total}\n' "$curl_args") echo "$response" echo "----------------------------------------" } # Example call test_request "Cookie with folioAccessToken" \ "$GATEWAY_URL/api/test -H 'Cookie: folioAccessToken=$TEST_TOKEN; otherCookie=value123' -v" # ... see the full script for other test cases (including header and error scenarios)

For complete coverage, see test-plugin.sh in the attachment.​

Migrated Plugin (APISIX Format):

The full auth-headers-manager.lua plugin is implemented using APISIX's Lua interface, with schema validation, header/cookie manipulation, token consistency checks, and logging for auditability. See the attachment file for the exact implementation.

 

 

Conclusion

In the end, we can migrate all functionality from Kong to APISIX, but APISIX will use etcd instead of PostgreSQL for storing configuration data.