Marc's guide to local schema upgrade testing
Introduction
This document intends to describe how I do local schema upgrade testing. I hope it might be useful to some folks.
Included is a worked example based upon testing upgrading mod-circulation-storage from the latest 2021 R2 version to the intended 2021 R3 version
Prerequisites
- Docker Desktop
- Local copy of FOLIO tools repository
- Local copy of the module to be tested
- Postman (optional, I use it for enabling the module versions and checking endpoints)
Overall Process
- Deploy the infrastructure needed (e.g. PostgreSQL, Kafka)
- Deploy the starting version of the module
- Enable the starting version of the module for a tenant (with sample and reference data)
- Check that the module version works as expected
- Undeploy the starting version of the module
- Deploy the destination version of the module
- Enable the destination version of the module for a tenant (without sample and reference data)
- Check that the module version works as expected
- Undeploy the destination version of the module
- Undeploy the infrastructure
Detailed Steps
Deploy the infrastructure needed (e.g. PostgreSQL, Kafka)
I use https://github.com/folio-org/folio-tools/tree/master/infrastructure/local to deploy containers for PostgreSQL and Kafka
- Clone the folio-tools repository
- From the infrastructure/local directory, run:
docker compose up -d
Deploy the starting version of the module
- Check out the version of the module intended as the starting point
- The version can be find by reviewing the branch of platform complete for the last release
- For 2021 R2, the version of mod-circulation-storage according to https://github.com/folio-org/platform-complete/blob/R2-2021/okapi-install.json is
13.0.0
- this version is tagged as
v13.0.0
- Build the module, run:
mvn clean package
- Deploy the module, in the local copy of the module, run:
docker compose up --build -d
- If the module does not have a docker compose file already, an example is provided below
- It is important to always rebuild the image to make sure the right version of the module is being deployed
Enable the starting version of the module for a tenant (with sample and reference data)
- Begin enabling this version of the module by making the following HTTP request
- The version needs to be the complete module name, for mod-circulation-storage version 13.0.0 it would be
mod-circulation-storage-13.0.0
- Make a note of the
id
property in the response
- The version needs to be the complete module name, for mod-circulation-storage version 13.0.0 it would be
POST http://localhost:8081/_/tenant { "module_to": "{{INITIAL-VERSION}}", "parameters": [{ "key": "loadSample", "value": "true" },{ "key": "loadReference", "value": "true" }] }
- Check that the enablement has finished
- Newer versions of the tenant API respond immediately with reference to use to check on it's status later
- Make the following HTTP request using the
id
captured from the response to the first request - GET http://localhost:8081/_/tenant/{{VERSION-ENABLEMENT-ID}}?wait=30000
- Check that the
complete
property is true, otherwise repeat the HTTP request until it either is or an error is reported
Check that the module version works as expected
This will vary from module to module.
Some folks do database schema diffs and check that against expected changes. I rarely have time to do this.
I believe the intent is to identify critical failures that would stop use in the BugFest environments and do rudimentary checks on a selection of endpoints to get a sense that the module is running and can be used by client.
Undeploy the starting version of the module
- Undeploy the module, in the local copy of the module, run:
docker compose down
Deploy the destination version of the module
- Check out the destination version of the module
- As this is intended to be done prior to the initial release for a distribution, this is usually the head of the mainline
- Follow the same steps as above for building and deploying the module
- Remember to always rebuild both the module and the docker image
Enable the destination version of the module for a tenant (without sample and reference data)
Repeat the process described above with a different starting HTTP request
- Use the same version as above for the initial version
- The upgrade version needs to be the complete module name, for the head of the mainline for mod-circulation-storage version it would be
mod-circulation-storage-13.1.0-SNAPSHOT
POST http://localhost:8081/_/tenant { "module_from": "{{INITIAL-VERSION}}", "module_to": "{{UPGRADE-VERSION}}", "parameters": [{ "key": "loadSample", "value": "false" },{ "key": "loadReference", "value": "false" }] }
Check that the module version works as expected
I do the same checks as with the starting version (described above)
Undeploy the destination version of the module
- This is the same as for the starting version, as above
Undeploy the infrastructure
- From the infrastructure/local directory, run:
docker compose down
Additional Resources
Craig's Guide
Craig McNally wrote a similar guide a while ago that goes into more detail about what can be checked during this process.
Official Guidance
Official guidance on what to do is provided on Slack
Postman Collection
I use Postman for enabling / disabling the module and checking that endpoints work. An export of what I used is provided here: schema-upgrade-testing.postman_collection.json
Example Docker Compose File
Docker compose makes it easy to define configuration for a module. An example of this which may work for most modules is docker-compose.yml