Marc's guide to local schema upgrade testing

Owned by Marc Johnson
Sep 16, 2021

Introduction

This document intends to describe how I do local schema upgrade testing. I hope it might be useful to some folks.

Included is a worked example based upon testing upgrading mod-circulation-storage from the latest 2021 R2 version to the intended 2021 R3 version

Prerequisites

  • Docker Desktop
  • Local copy of FOLIO tools repository
  • Local copy of the module to be tested
  • Postman (optional, I use it for enabling the module versions and checking endpoints)

Overall Process

  • Deploy the infrastructure needed (e.g. PostgreSQL, Kafka)
  • Deploy the starting version of the module
  • Enable the starting version of the module for a tenant (with sample and reference data)
  • Check that the module version works as expected
  • Undeploy the starting version of the module
  • Deploy the destination version of the module
  • Enable the destination version of the module for a tenant (without sample and reference data)
  • Check that the module version works as expected
  • Undeploy the destination version of the module
  • Undeploy the infrastructure

Detailed Steps

Deploy the infrastructure needed (e.g. PostgreSQL, Kafka)

I use https://github.com/folio-org/folio-tools/tree/master/infrastructure/local to deploy containers for PostgreSQL and Kafka

  • Clone the folio-tools repository
  • From the infrastructure/local directory, run: docker compose up -d

Deploy the starting version of the module

  • Check out the version of the module intended as the starting point
  • Build the module, run: mvn clean package
  • Deploy the module, in the local copy of the module, run: docker compose up --build -d
    • If the module does not have a docker compose file already, an example is provided below
    • It is important to always rebuild the image to make sure the right version of the module is being deployed

Enable the starting version of the module for a tenant (with sample and reference data)

  • Begin enabling this version of the module by making the following HTTP request
    • The version needs to be the complete module name, for mod-circulation-storage version 13.0.0 it would be mod-circulation-storage-13.0.0
    • Make a note of the id property in the response
Begin tenant enablement
POST http://localhost:8081/_/tenant
{
  "module_to": "{{INITIAL-VERSION}}",
  "parameters": [{
    "key": "loadSample",
    "value": "true"
  },{
    "key": "loadReference",
    "value": "true"
  }]
}
  • Check that the enablement has finished
    • Newer versions of the tenant API respond immediately with reference to use to check on it's status later
    • Make the following HTTP request using the id captured from the response to the first request
    • GET http://localhost:8081/_/tenant/{{VERSION-ENABLEMENT-ID}}?wait=30000
    • Check that the complete property is true, otherwise repeat the HTTP request until it either is or an error is reported

Check that the module version works as expected

This will vary from module to module.

Some folks do database schema diffs and check that against expected changes. I rarely have time to do this.

I believe the intent is to identify critical failures that would stop use in the BugFest environments and do rudimentary checks on a selection of endpoints to get a sense that the module is running and can be used by client.

Undeploy the starting version of the module

  • Undeploy the module, in the local copy of the module, run: docker compose down

Deploy the destination version of the module

  • Check out the destination version of the module
    • As this is intended to be done prior to the initial release for a distribution, this is usually the head of the mainline
  • Follow the same steps as above for building and deploying the module
    • Remember to always rebuild both the module and the docker image

Enable the destination version of the module for a tenant (without sample and reference data)

  • Repeat the process described above with a different starting HTTP request

    • Use the same version as above for the initial version
    • The upgrade version needs to be the complete module name, for the head of the mainline for mod-circulation-storage version it would be mod-circulation-storage-13.1.0-SNAPSHOT
Begin tenant enablement
POST http://localhost:8081/_/tenant
{
  "module_from": "{{INITIAL-VERSION}}",
  "module_to": "{{UPGRADE-VERSION}}",
  "parameters": [{
     "key": "loadSample",
     "value": "false"
    },{
     "key": "loadReference",
     "value": "false"
  }]
}

Check that the module version works as expected

I do the same checks as with the starting version (described above)

Undeploy the destination version of the module

  • This is the same as for the starting version, as above

Undeploy the infrastructure

  • From the infrastructure/local directory, run: docker compose down

Additional Resources

Craig's Guide

Craig McNally wrote a similar guide a while ago that goes into more detail about what can be checked during this process.

Official Guidance

Official guidance on what to do is provided on Slack

Postman Collection

I use Postman for enabling / disabling the module and checking that endpoints work. An export of what I used is provided here: schema-upgrade-testing.postman_collection.json

Example Docker Compose File

Docker compose makes it easy to define configuration for a module. An example of this which may work for most modules is docker-compose.yml