Self-hosted GitHub runners on AWS EKS

This SPIKE evaluates the feasibility, cost-efficiency, and performance of using an existing AWS EKS cluster to host self-hosted GitHub Actions runners instead of using GitHub-hosted runners.

Current Context

AWS EKS Cluster: Already provisioned and used for Jenkins Kubernetes dynamic agents.
Required Capacity:
- 30–60 simultaneous workflows every 15–30 minutes (potentially staggered every 5 minutes).
- 1–3 additional workflows per hour for platform validation and release tasks.

Benefits

Potential cost savings for large-scale, frequent workflows.
Full control over runner configuration and scaling.
Possibility to reuse existing EKS infrastructure.
Avoid runner queue limitations on GitHub-hosted runners.

Drawbacks

Additional operational complexity and maintenance overhead.
Requires secure management of runner registration tokens or GitHub App credentials.
Must handle scaling and lifecycle of runner pods.

Solution Overview

We will deploy self-hosted GitHub Actions runners using the actions-runner-controller Helm chart, authenticated via GitHub App.

Solution Architecture

Tools and Components

Component	Purpose

Component	Purpose
`actions-runner-controller`	Manages self-hosted runners in Kubernetes
Helm	Simplifies deployment and lifecycle management
Horizontal Pod Autoscaler (HPA)	Dynamically scales runner pods
AWS IAM Roles for Service Accounts (IRSA)	Secure pod-level permissions
GitHub App	Secure authentication and integration

Step-by-Step Deployment Guide

Prerequisites

Provisioned AWS EKS cluster.
kubectl configured to access the EKS cluster.
Helm installed locally.
Configured GitHub App with App ID, Installation ID, and Private Key.

1. Configure GitHub App

Steps to Create GitHub App

Navigate to your GitHub organization settings.
Go to Developer settings > GitHub Apps.
Click New GitHub App.
Fill out the following fields:
- GitHub App name: Example: folio-eks-self-hosted-runner
- Homepage URL: https://github.com/actions/actions-runner-controller`.
- Webhook URL: Leave blank (possibly will be implemented later).
- Webhook secret: Optional.

Permissions

Repository permissions:
- Actions: Read & Write
Organization permissions:
- Self-hosted runners: Read & Write

Finalize

Click Create GitHub App.
Generate and download the private key.
Save:
- App ID
- Installation ID (found after installing the app to your organization/repositories)
- Private key (used in Helm deployment)

Documentation:

2. Create Kubernetes Namespace and GitHub App Secret

kubectl create namespace folio-gha-runner-system

kubectl create secret generic github-app-gha-runner-secret \
   --namespace=folio-gha-runner-system \
   --from-literal=github_app_id=<appId> \
   --from-literal=github_app_installation_id=<installationId> \
   --from-literal=github_app_private_key='-----BEGIN RSA PRIVATE KEY-----*****'

3. Install gha-runner-scale-set-controller

helm install gha-runner-controller \
    --namespace folio-gha-runner-system \
    --create-namespace \
    oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set-controller

4. Install gha-runner-scale-set

helm install gha-runner-scale-set \
    --namespace folio-gha-runner-system \
    --create-namespace \
    --set githubConfigUrl=https://github.com/<your_enterprise/org/repo>  \
    --set githubConfigSecret=github-app-gha-runner-secret \
    oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set

Cost and Performance Comparison

Feature	GitHub-Hosted Runners	Self-Hosted Runners in EKS

Feature	GitHub-Hosted Runners	Self-Hosted Runners in EKS
Cost	$0.008 per minute per Linux job	EKS node costs + operational overhead
Scalability	Limited by GitHub quotas	Scalable with Kubernetes HPA and Cluster Autoscaler
Performance	Standardized, may queue under load	Can scale aggressively, less queueing
Security Control	Limited	Full VPC isolation, custom security
Management Overhead	None	Requires active monitoring and scaling management

Decision Recommendation

Proceed with phased adoption of self-hosted runners on AWS EKS for high-volume workflows while maintaining GitHub-hosted runners for low-frequency or ad-hoc tasks to balance risk and operational overhead.