Spike - Revisit platform descriptors

1 Spike Overview
2 Business Drivers:
3 Workshop with DevOps to determine what information is required, including:
4 Platform descriptor example:
5 Platform descriptors inheritance concept.
- 5.1 Decision:Instead of introducing a third tier to the extended platform descriptor, we will include a list of applications in the functional platform descriptor, specifying whether each application is optional or required. This approach will reduce the number of platform descriptors and simplify the process for the DevOps team to parse them.
6 Questions:
7 Conclusion

Spike Overview

JIRA ID: EUREKA-285 Revisit platform descriptors

Objective: In Eureka, applications are composed of modules, defined by application descriptors. These applications, in turn, compose platforms represented by platform descriptors. While the concept of platform descriptors is not new, we have yet to design or implement a concrete solution.

For the legacy platform, we use an install.json file, which essentially lists modules and their versions. In Eureka, we aim to work with applications, so we've been providing DevOps with informal lists of applications, their versions, and other components (mgr-*, Keycloak, Kong, etc.). Our goal is to formalize this into a versioned platform descriptor that includes all relevant component and infrastructure versions.

Several additional concepts are also relevant here:

Virtual applications: These are applications not composed of modules but rather infrastructure components. While some aspects may have been included in platform schema examples, we've never formalized this concept.
Platform inheritance: This introduces the idea of "base" and "extended" platforms. Extended platforms can inherit from a base platform, allowing us to create a foundational platform with the essential components for a minimal, functional system. Multiple extended platforms (LSP, ERM, Locate, etc.) can then build upon this shared base.

Business Drivers:

To formalize the release process for the Eureka platform by clearly defining which applications and environment components will be deployed to the cluster, we need to establish the structure of the platform descriptor, detailing its format and the necessary components for the DevOps team.

Workshop with DevOps to determine what information is required, including:

A list of applications and their versions
For infrastructure components, we need to provide a range of versions that can be used. The implementation should include a list of supported versions. For example, in the Sunflower release, we can support multiple versions of PostgreSQL
Key/value properties allow us to extend the platform descriptor with additional information in the future without requiring code changes.
Platform name
Version number
Description (if needed)
Manager service versions
The sidecar version does not need to be managed at the platform level, as it will be included in the application descriptor. However, as discussed, the sidecar should be part of the platform descriptor within the base platform.
Infrastructure versions
The Keycloak version should be managed as the folio-keycloak application, as we use a custom build and do not use the standard version.
The Kong version should be managed as the folio-kong application, as we use a custom build
Kafka version

What else should be considered?

Platform descriptor example:

{
  "$schema": "../platformDescriptor.json",
  "id": "7a5a48b1-359a-4ec9-b980-7533b38be518",
  "name": "base-platform",
  "version": "1.1.0",
  "type": "base",
  "description": "Base platform",
  "applications": [
    {
      "name": "postgres",
      "version": "12.11.0",
      "type": "system"
    },
    {
      "name": "keycloak",
      "version": "19.0.0",
      "type": "system"
    },
    {
      "name": "kafka",
      "version": "3.2.0",
      "type": "system"
    },
    {
      "name": "kong",
      "version": "2.4.2",
      "type": "system"
    },
    {
      "name": "application-manager",
      "version": "1.0.0",
      "type": "system"
    },
    {
      "name": "users",
      "version": "16.2.0",
      "type": "functional"
    },
    {
      "name": "tags",
      "version": "7.2.5",
      "type": "functional"
    }
  ]
}

{
  "$schema": "../platformDescriptor.json",
  "id": "6628487d-23d1-4e98-8203-4e59deb907eb",
  "name": "lsp-platform",
  "version": "1.1.0",
  "type": "extended",
  "description": "LSP platform",
  "basePlatform": {
    "name": "base-platform",
    "version": "1.1.0"
  },
  "applications": [
    {
      "name": "circulation",
      "version": "11.2.5",
      "type": "functional"
    },
    {
      "name": "acquisitions",
      "version": "8.2.3",
      "type": "functional"
    },
    {
      "name": "inventory",
      "version": "12.9.0",
      "type": "functional"
    },
    {
      "name": "finance",
      "version": "5.1.2",
      "type": "functional"
    }
  ]
}

add key value structure at application section to be able extend in the future

Platform descriptors inheritance concept.

The platform consists of three tiers:

Base platform descriptor: This includes essential applications and infrastructure components required for the minimal functionality of the FOLIO platform.
Functional platform descriptor: A list of applications must be installed on the cluster.
Extended platform descriptor: This can be added later to expand the cluster by incorporating new applications for interim updates.

Decision:
Instead of introducing a third tier to the extended platform descriptor, we will include a list of applications in the functional platform descriptor, specifying whether each application is optional or required. This approach will reduce the number of platform descriptors and simplify the process for the DevOps team to parse them.

Questions:

Who will maintain the platform descriptor? The same team members who currently generate the install.json files for the Okapi platform will maintain the platform descriptor.
Should the platform descriptor be included within the platform to ensure it's accessible by REST when the cluster is established and the DevOps team needs to retrieve the current platform version? not for now
How will the platform descriptor update functionality be managed? The platform descriptor does not support updates; instead, a new version of the platform descriptor will be created each time, including updated versions of applications and components that require updating.

Conclusion

During the solution architecture meeting, we reviewed the document and agreed on the next steps to finalize the implementation:

Design a new JSON schema for the platform descriptor, incorporating all the changes outlined in the document. Also, create documentation for the DevOps team to ensure they can work with it effectively.
Build a simple POC (Proof of Concept) of the platform descriptor based on the new structure, using the Ramsons release and ECs patches. This will help us see how it looks in practice.
Hold a session with the FSE/Kitfox team to share the outcome of the POC, enabling them to start using and implementing it within their pipelines.
Support the FSE/Kitfox team in building their pipelines to work with the new platform descriptors, adjusting based on their specific requirements.

Spike Status: Complited