FMDR: Fleet Deployment Templates

This page is intended to describe the details on configuring and building Fleet deployment templates for use by build_deployments.sh.

Example templates can be found in folio-module-descriptor-registry/tree/main/template/deploy/input.

1 Main Input Template Files
- 1.1 The base.json File
- 1.2 The deployment.json File
- 1.3 The maps.json File
- 1.4 The names.json File
- 1.5 The service.json File
- 1.6 The vars.json File
- 1.7 The deployment_sidecar.json File
- 1.8 The service_sidecar.json File
- 1.9 The stateful_set.json File
2 The Individual Variable Files
3 The Individual Specific Files
4 The Launch Files
5 Template Variables
6 Expansion and Substitution Order of Operations
7 Using Config Maps and Secrets
8 Recommended Metadata Labels

Main Input Template Files

The template directory template/deploy/input/main/ contains several JSON files used to provide configuration for helping facilitate the generation of Fleet deployment templates, known as a Fleet Manifest.

All configuration files are in JSON format.
Only the final, generated, Fleet Manifest is stored as a YAML file.
A YAML file is used because that is the expected format for a Fleet Manifest.

The main input template files consists of the following files:

base.json
deployment.json
maps.json
names.json
service.json
vars.json

Additional main input files might also exist as defined via the map.json, such as these:

deployment_sidecar.json
service_sidecar.json
stateful_set.json

The `base.json` File

The base.json file defines the primary JSON object used to contain the base image for building the final Fleet Manifest.
This file almost always never changes and should not need to be modified.

The `deployment.json` File

The deployment.json file is the base template used as the starting point for constructing and expanding a deployment into the Fleet Manifest.

It is expected to contain a complete deployment JSON file ("kind": "Deployment").

This file can be replaced by another file via the deploy property in the maps.json file, such as "deploy": "deployment_sidecar" to designate using the file deployment_sidecar.json.

The `maps.json` File

The maps.json file provide a way to alter the default generation behavior for deployments and services.

There are two ways to match and change the behavior.

Exact Match
PCRE Match (Perl Compatible Regular Expression)

The Exact Match matches on the exact module name to allow for altering the behavior of a specific module.
A common change would be to replace a deployment with a stateful set.

The PCRE Match utilizes the grep tool in PCRE mode (via the -P argument) to match anything based on the string provided in the key.
The example below shows that anything that matches mod-* shall be replaced with the deployment_sidecar and service_sidecar JSON files.

The following is an example maps.json file used in a Eureka deployment:

{
  "exact": {
    "mod-agreements":            { "deploy": "stateful_set", "service": "service" },
    "mod-circulation-storage":   { "deploy": "stateful_set", "service": "service" },
    "mod-di-converter-storage":  { "deploy": "stateful_set", "service": "service" },
    "mod-finance-storage":       { "deploy": "stateful_set", "service": "service" },
    "mod-inventory-storage":     { "deploy": "stateful_set", "service": "service" },
    "mod-invoice-storage":       { "deploy": "stateful_set", "service": "service" },
    "mod-licenses":              { "deploy": "stateful_set", "service": "service" },
    "mod-lists":                 { "deploy": "stateful_set", "service": "service" },
    "mod-orders-storage":        { "deploy": "stateful_set", "service": "service" },
    "mod-organizations-storage": { "deploy": "stateful_set", "service": "service" },
    "mod-patron-blocks":         { "deploy": "stateful_set", "service": "service" },
    "mod-source-record-storage": { "deploy": "stateful_set", "service": "service" }
  },
  "pcre": {
    "mod-*": { "deploy": "deployment_sidecar", "service": "service_sidecar" }
  } 
}

The `names.json` File

The names.json file describes all DVV names.

These names are exact matches and must have any appropriate wrapping characters such as brackets [ and ].

Example names.json content:

[
  "[CONTAINER_ARGS]",
  "[CONTAINER_IMAGE]",
  "[CONTAINER_IMAGE_PULL_POLICY]",
  "[CONTAINER_IMAGE_PULL_SECRETS]",
  "[CONTAINER_PORT]"
]

The `service.json` File

The service.json file is the base template used as the starting point for constructing and expanding a service into the Fleet Manifest.

It is expected to contain a complete service JSON file ("kind": "Service").

This file can be replaced by another file via the service property in the maps.json file, such as "deploy": "stateful_set" to designate using the file stateful_set.json.

The `vars.json` File

The vars.json file contains all of the DVV variables and their values at a global scope.

This file is processed first before all other variables in the substitution process.
Therefore, every value provided within this file is subject to being overwritten by another means.
Consider these default global values.

Specifying as much in this vars.json file is recommended to avoid needing to repeat any variable values anywhere else.

The `deployment_sidecar.json` File

The deployment_sidecar.json file is an additional file needed by Eureka to provide a deployment that has an attached sidecar.

This file is not provided by default, but the scratch environment likely has this file defined.

The `service_sidecar.json` File

The service_sidecar.json file is an additional file needed by Eureka to provide a service that has an attached sidecar.

This file is not provided by default, but the scratch environment likely has this file defined.

The `stateful_set.json` File

The service.json file is the base template used as the starting point for constructing and expanding a stateful set into the Fleet Manifest.

It is expected to contain a complete stateful set JSON file ("kind": "StatefulSet").

This file can be replaced by another file via the deploy property in the maps.json file, such as "deploy": "stateful_set" to designate using the file stateful_set.json.

The Individual Variable Files

Each module may match a variables, or “vars” for short, file that provides variable overrides specific to each individual module.
The files are stored in template/deploy/input/vars/[module].json where [module] would be the name of the module (like template/deploy/input/vars/mod-workflow.json for mod-workflow).

Any variable define in these is used in preference to any variable defined in the global main vars.json file.

The Individual Specific Files

Each module may have a specific file used as a substitute to anything define within the deployment.json file (or its appropriate mapping deploy replacement file).
The files are stored in template/deploy/input/specific/[module].json where [module] would be the name of the module (like template/deploy/input/specific/mod-workflow.json for mod-workflow).

The JSON from each specific file is merge-joined with the deployment.json file (or appropriate mapping file).

The “specific” files were originally used for module details but fell out of use in favor to using the individual variables files.

The behavior has been left for now but has not been well tested after numerous changes and might be removed in the near future if deemed too problematic.

The Launch Files

Each module may have a launch files to specificy project-specific defaults recommended based on the launchDescriptor section defined in Module Descriptor JSON file for that module.

Unless overriden by an individual variable file for some named module, this data, if generated and available, is automatically added during the build process.

The launchDescriptor data is not in a form easily processed as a template and must be converted using another script like the build_launches.sh script.

The template/launch/input/instructions.json file defines the settings that are extracted from the Module Descriptor JSON file launchDescriptor field.

The launch files are not required but recommended in order to preserve the settings defined by the project owners.
However, it has been discovered that in several situations certain environments require more or less resources than suggested by the project owners.
There is a good chance that these values may need to be ignored and overwritten.
In such cases, it might be better to not build and use the launch files at all.

For these reasons that it is recommended to not save the generated launch files in a repository and to always generate before the build process if they are going to be used.

Template Variables

There are two types of variables supported in the template files.

Direct Value Variable
Inline Value Variable

The Direct Value Variable (DVV) are variables defined in the names.json file that are used to completely replace a given JSON value (but not the key).
The DVV is used as a complete JSON value string type.

There is no specific syntax required for the name of the DVV.
Whatever is defined in the names.json is used exactly as is and much exactly match to be expanded.

The standard practice is to use wrap the DVV in brackets ([ and ]) as part of the name and to follow standard global environment variable name syntax rules.
The standard global environment variable name syntax rules is to be all upper case, ASCII alphabet word characters, and digits in any character position other than the first (A-Z, _, and 0-9).

Examples:

"[JAVA_OPTIONS]"
"[JAVA_OPTIONS_VALUE]"
"[MODULE_ID]"
"[RESOURCES]"

These DVV can provide substitute in whole JSON objects.
One such use case for this is to define environment variables using "[ENVIRONMENT]" and adding additional properties like "[JAVA_OPTIONS]" where the additional properties themselves provide an object with value of "[JAVA_OPTIONS_VALUE]".

If a DVV expands into nothing (such as NULL), then that key and value pair is entirely removed from the JSON.

Any syntax for a DVV name can be used, but braces ({ and }) should be not be used to avoid conflicting with IVV.

The Inline Value Variable (IVV) are reserved variables used for regex replacements on the entire template file.
The IVV are used to replace exact raw string matches for the entire files and does not offer any JSON interpretation or handling.
The replacements are typically done using the sed command line tool on the entire JSON file.

The syntax for IVV is of the following form: {key:value} where keyis a placeholder for a select list of words and value is a conditionally optional value.
In cases where value is optional, this can be an empty such that the form is instead {key:}.
A common form of this uses a module name for value.

Double quotes are explicitly prohibited in the key or the value names in order to prevent confusing this with valid JSON syntax.
Any unsupported {key:value} pair should result in invalid JSON in order to help expose and help avoid hiding problems.

The IVV is useful for situations such as where the module named mod-consortia-keycloak needs to define MOD_USERS_URL with a value of http://mod-users-19.6.0-SNAPSHOT.350.folio-modules.svc.

This can be done using a value like: http://mod-users-{version:mod-users}.{global:namespace}.svc.

The following table describes the reserved words, the requirements for the value, and how the replacement works for that reservation.

Reserved Word	Value Word	Example	Description

Reserved Word	Value Word	Example	Description
`id`	The `value` is a module name or is empty.	`{id:mod-users}`	Expand into the `id` property of the Module Discovery for the named module. An empty value, such as `{version:}` will expand to the currently operating module during expansion.
`global`	The `value` is always `app`.	`{global:app}`	Expand into the Application name. The Application name value is expanded into is defined by the build_deployments.sh script. The Application name can be set via `BUILD_DEPLOY_APP` environment variable.
`global`	The `value` is always `namespace`.	`{global:namespace}`	Expand into the namespace being used. The namespace value is expanded into is defined by the build_deployments.sh script. The namespace can be set via `BUILD_DEPLOY_NAMESPACE` environment variable.
`location`	The `value` is a module name or is empty.	`{location:mod-users}`	Expand into the `id` property of the Module Discovery for the named module. An empty value, such as `{version:}` will expand to the currently operating module during expansion.
`name`	The `value` is a module name or is empty.	`{name:mod-users}`	Expand into the `id` property of the Module Discovery for the named module. An empty value, such as `{version:}` will expand to the currently operating module during expansion.
`repository`	The `value` is a module name or is empty.	`{repository:mod-users}`	Expand into the repository name specifically as named in the repositories.json settings file. If there is no Repository Location JSON file for any given named module, then the default repository (usually `folioci`) is used. The default repository is also used when `value` is empty, such as with `{repository:}`. The default repository can be be set via `BUILD_DEPLOY_DEFAULT_REPOSITORY` environment variable.
`version`	The `value` is a module name or is empty.	`{version:mod-users}`	Expand into the `id` property of the Module Discovery for the named module. An empty value, such as `{version:}` will expand to the currently operating module during expansion.

As example, consider this vars.json file:

{
  "[ENVIRONMENT]": [
    "[JAVA_OPTIONS]",
    "[MODULE_ID]",
    "[MODULE_NAME]",
    "[MODULE_VERSION]"
  ],
  "[JAVA_OPTIONS_VALUE]": "-XX:MaxRAMPercentage=80.0 -Dlog4j2.formatMsgNoLookups=true",
  "[JAVA_OPTIONS]": {
    "name": "JAVA_OPTIONS",
    "value": "[JAVA_OPTIONS_VALUE]"
  },
  "[MODULE_ID]": {
    "name": "MODULE_ID",
    "value": "{name:}-{version:}"
  },
  "[MODULE_NAME]": {
    "name": "MODULE_NAME",
    "value": "{name:}"
  },
  "[MODULE_VERSION]": {
    "name": "MODULE_VERSION",
    "value": "{version:}"
  }
}

Along with this (trimmed) deployment.json file:

{
  "metadata": {
    "name": "{name:}",
    "namespace": "{global:namespace}"
  },
  "spec": {
    "template": {
      "spec": {
        "containers": [
          {
            "env": "[ENVIRONMENT]"
          }
        ]
      }
    }
  }
}

The expansion for mod-workflow with version 1.2.0 should look something like this:

{
  "metadata": {
    "name": "mod-workflow",
    "namespace": "folio-modules"
  },
  "spec": {
    "template": {
      "spec": {
        "containers": [
          {
            "env": [
              {
                "name": "JAVA_OPTIONS",
                "value": "-XX:MaxRAMPercentage=80.0 -Dlog4j2.formatMsgNoLookups=true"
              },
              {
                "name": "MODULE_ID",
                "value": "mod-workflow-1.2.0"
              },
              {
                "name": "MODULE_NAME",
                "value": "mod-workflow"
              },
              {
                "name": "MODULE_VERSION",
                "value": "1.2.0"
              }
            ]
          }
        ]
      }
    }
  }
}

Expansion and Substitution Order of Operations

The variable substitution and module processing order are as follows:

deployment.json, or other file such as stateful_set.json as defined by maps.json
vars.json
launches/ JSON file for each module
vars/ JSON file for each module
specific/ JSON file for each module

The substitution and expansion process does not operate recursively.

Instead, a limited set of passes are performed in a loop based on the BUILD_DEPLOY_PASSES setting.
The default might be something small, like 4.

Using Config Maps and Secrets

It is highly recommended to use Config Maps and Secrets rather than specifying environment variables.

Both the Config Maps and the Secrets should be set as optional to prevent the Fleet Manifest file from causing problems that can be solved by simply creating a Config Map or Secret independent of this generation process.

The DVV "[ENV_FROM]" is generally used to expand into any number of Config Maps and Secrets.
Common examples of these are "[ENV_FROM_GLOBAL]" and "[ENV_FROM_MODULE_SECRET]".

The "[ENV_FROM_GLOBAL]" might look something like this:

  "[ENV_FROM_GLOBAL]": {
    "configMapRef": {
      "name": "folio-modules-global",
      "optional": true
    }
  }

The "[ENV_FROM_MODULE_SECRET]" might look something like this:

  "[ENV_FROM_MODULE_SECRET]": {
    "secretRef": {
      "name": "{name:}",
      "optional": true
    }
  }

It is strongly suggest to have a "[ENV_FROM_MODULE_CONFIG]" and a "[ENV_FROM_MODULE_SECRET]" created and expanding into the module name as shown in the above example for "[ENV_FROM_MODULE_SECRET]".
Thus, in conjunction with the "optional": true property, any module could have a Config Map or Secret created (or removed) at any point in runtime without having to change or alter the Fleet Manifest.

Recommended Metadata Labels

Labels assist in management and organization of deployments and their associated resources.
When combined with selectors, labels become very powerful and helpful.

The Kubernetes documentation provides good documentation regarding this:

Based on the Kubernetes documentation and recommendations, the following labels are recommended for FOLIO deployments:

Recommended Label	Example Value	Purpose

Recommended Label	Example Value	Purpose
`app`	`app-platform-complete`	The name of the application bundle. For Eureka, this will be just the application name some set of modules are part of. For legacy OKAPI, this will be a flower release with `app-` prepended, like `app-poppy`.
`app.kubernetes.io/instance`	`mod-workflow-1.2.0`	The unique identifier of the module.
`app.kubernetes.io/name`	`mod-workflow`	The name of the module.
`app.kubernetes.io/part-of`	`FOLIO`	Always `FOLIO`. This is not directly used, but may be helpful to have for organizational reasons.
`app.kubernetes.io/version`	`1.2.0`	The module version.
`environment`	`DEV`	The environment build in which this is run within. This could be an operational name like `PROD` or even a team name like `Aggies`.

Example metadata labels showing mod-workflow version 1.2.0:

      "metadata": {
        "labels": {
          "app": "app-poppy",
          "app.kubernetes.io/instance": "mod-workflow-1.2.0",
          "app.kubernetes.io/name": "mod-workflow",
          "app.kubernetes.io/part-of": "FOLIO",
          "app.kubernetes.io/version": "1.2.0",
          "environment": "DEV"
        }
      },

FMDR: Fleet Deployment Templates

Main Input Template Files

The base.json File

The deployment.json File

The maps.json File

The names.json File

The service.json File

The vars.json File

The deployment_sidecar.json File

The service_sidecar.json File

The stateful_set.json File