RANCHER-2358: Analysis of Release Branch Preparation via GitHub Actions

Research Objective

Evaluate the feasibility of automating the preparation of release branches for FOLIO applications using GitHub Actions, and identify potential challenges, limitations, and key issues.

Key Technical Findings

• Manual Workflow Dispatch Requires Default Branch: To trigger a workflow manually in the GitHub Actions UI, the workflow file must already reside in the default branch (master). GitHub does not display the “Run workflow” button for workflows defined only in feature branches or not yet merged into master. Therefore, adding new workflows requires creating and merging a pull request into master, which can introduce delays and complicate testing. However, such a workflow can still be invoked via the GitHub CLI or API by specifying the branch with --ref; this option is not exposed in the UI.

• Tooling Requirements (Terraform, kubectl, Helm): GitHub-hosted runners do not include specialized tools out of the box. You can either:

  • Use built-in setup actions (e.g., actions/setup-terraform, actions/setup-kubectl) to install tools at runtime, suitable for simple or one-off tasks.

  • Provide a custom Docker container image pre-installed with your toolchain, ensuring a consistent environment both locally and in CI. This is especially useful with the Actions Runner Controller (ARC) in Kubernetes, which can spin up runner pods from your custom image.

  Note: Docker containers only run on Linux runners. Avoid Docker-in-Docker due to security and debugging complexity. Use custom containers only when full environment control is required.

• Complex Logic and Error Handling: High-level languages like Python simplify retry logic and error handling. For simpler or cross-platform scripts, prefer the official actions/setup-python action.

• Secrets Management: Store secrets in GitHub Secrets at the organization level to ensure centralized control and avoid context or token-passing issues between repositories and workflows.

Potential Technical Challenges & Solutions

• GitHub Actions Concurrency Limits: Runners have a parallel job limit (20–40 concurrent runs, depending on plan). Plan matrix sizes and parallelism accordingly.

• Python & Other Tooling:

  • Complex tasks (e.g., creating namespaces from branch names) may require running Python within containers.

  • For simpler, cross-platform needs, use actions/setup-python.

  • Python’s state does not persist between steps, limiting object-oriented approaches and requiring reinitialization in each step.

  • Environment initialization in Python can be time-consuming, increasing overall run time.

  • Breaking logic into small Bash scripts often improves simplicity and debuggability.

  • Use GITHUB_OUTPUT to pass data between steps, but structure your code around this mechanism.

• Default Branch Workflow Location: Keep the default branch (master) up to date with stable module versions. Any new application or module must first land in master before inclusion in the release pipeline.

Best Practices & Recommendations

Bash vs. Python

• When to Use Bash:

  • Text manipulation, condition checks, file operations, and simple utility chaining.

  • Faster startup, native on all runners, minimal dependencies.

• When to Use Python:

  • Complex REST API integrations, stateful operations, or heavy JSON processing.

  • Better structure, reusable modules, clearer error handling.

• GitHub Actions Constraints:

  • Steps are isolated; Python code cannot maintain state between steps, limiting OOP patterns (e.g., singletons only work within a single step).

  • Recommended approach: extract Python logic into standalone .py scripts, invoke them from Bash steps, and keep workflows modular.

• REST API Example: FOLIO’s pipelines-shared-library: eureka REST v2 can be implemented in Bash, but with significant verbosity and reduced readability.

• Workflow's Structure:

  • Keep steps focused and concise; delegate complex logic to external scripts or reusable actions.

  • Avoid monolithic workflows; split them into logical units where possible.

• Secrets Management:

  • Store secrets at the organization level in GitHub Secrets to centralize control.

  • Use a GitHub App for cross-repository workflows, granting minimal required scopes and using short-lived tokens.

  • Rotate secrets regularly and audit usage to minimize risk.

• Container Usage:

  • Only use custom Docker containers when you need a reproducible environment with specific tool versions.

  • Default to runtime installation via setup-* actions for standard tools.

  • Do not use Docker-in-Docker; prefer single-container execution.

GitHub Actions workflow_dispatch vs. Jenkins Manual Triggers

Key takeaway: If you need ad‑hoc manual runs with simple parameters and prefer a clean, integrated UI, GitHub Actions workflow_dispatch is not sufficient, keeping in mind the default‑branch limitation. Jenkins offers richer parameter forms and branch flexibility.

Next Steps

1. Implement & Test Python Scripts: Develop standalone Python scripts for complex tasks and integrate them into workflows.

2. Run End-to-End Tests: Execute the release branch preparation pipeline with real modules; document any issues.

3. Identify Python Use Cases: Catalog scenarios where Python offers clear benefits and provide examples and guidelines.

4. Finalize Container Strategy: Decide if custom container images are necessary for your environment; build and publish images if so.