Pipeline engine alternatives || RANCHER-2326
This document evaluates Jenkins and GitHub Actions as CI/CD pipeline engines for Eureka, based on available technical information. The objective is to determine which engine best fits Eureka's requirements in terms of performance, scalability, maintenance, and operational efficiency.
Comparesion is aimed to identify the most suitable tool to be used for Eureka CI processes on GitHub platform for entire folio.org organisation.
Questions:
Pipeline Performance (Execution Speed)
Scalability (Concurrency, Parallel Builds)
Maintenance Overhead (Infrastructure, Upgrades)
Security & Access Control
Cost Considerations
Developer Experience
Integration with Existing Tools
Findings Summary(comparesion table)
Answers:
Factor | Jenkins | GitHub Actions |
|---|---|---|
Execution Speed | Fast, depends on internal infrastructure. | Generally good, but queued at peak times. |
Scalability | Scalable, but requires maintaining agents(additional || existing cluster(s). | GitHub-hosted runners limited, but scalable with self-hosted runners. |
Maintenance Overhead | High — requires maintaining servers, plugins, upgrades, agents etc. | Low — GitHub maintains infrastructure. |
Security | Full control, but responsibility on DevOps team. | GitHub-managed, integrates with GitHub permissions. |
Cost | Fixed infra cost; depends on runner setup. | Included in GitHub subscription; extra cost for self-hosted runners. |
Developer Experience | Requires knowledge of Jenkins; UI less intuitive. | Integrated in GitHub UI; easier for most devs. |
Integration with Tools | Broad plugin ecosystem; highly flexible. | Native GitHub integration; external tools supported via Actions. |
Performance Summary | Stable builds; observed infra bottlenecks under heavy load. | Faster setup for small/medium pipelines; occasional queuing during peak times. |
Simple apps generation pipelines can be handled via GitHub Actions(including apps registration in mgr-applications(FAR mode). More complex pipeline(s) should be handled on Jenkins side.
Plan | Storage | Minutes (per month) | Price |
|---|---|---|---|
GitHub Free | 500 MB | 2,000 | 0 |
GitHub Pro | 1 GB | 3,000 | 4 USD/month (invidual plan) |
GitHub Free for organizations(our case) | 500 MB | 2,000 | 0 |
GitHub Team | 2 GB | 3,000 | 4 USD per user/month |
GitHub Enterprise Cloud | 50 GB | 50,000 | 21 USD per user/month |
Pricing source: https://docs.github.com/en/billing/managing-billing-for-your-products/about-billing-for-github-actions
Before going further, PoC for full GitHub Actions approache must be conducted.
In overall comperasion GitHub Actions look more suitable for maintaining more than 20 Eureka applications' repositories. All executions will be independent and can run in parallel(repo level based runs) without any intersections.
If GitHub Actions attempt fail, hybrid variant should be considered (self-hosted runners).
Since folio.org is OSS project and all repositories are public, the following runners' configuration is applicable: https://docs.github.com/en/actions/how-tos/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories
vCPU: 4 RAM: 16 GB
Prelimanary decision GitHub Actions is more suitable for Eureka CI process(less management overhead, cost - Self-Hosted Runners will be charged from AWS side, GitHub is not charging for self-hosted runners: https://github.com/orgs/community/discussions/55015 ).
P\S According to clarification related to cost and usage of GHA in folio-org (GitHub organisation), limits do not apply.