[FOLIO-2198] Run API tests on a periodic/regular basis Created: 29/Jul/19  Updated: 03/Jun/20

Status: Open
Project: FOLIO
Components: Continuous Integration
Affects versions: None
Fix versions: None

Type: Story Priority: P3
Reporter: Craig McNally Assignee: John Malconian
Resolution: Unresolved Votes: 0
Labels: platform-backlog
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Sprint:
Development Team: Core: Platform

 Description   

Overview

API tests exist and are maintained on an ongoing basis for a decent cross-section of FOLIO modules. However, these tests are not run as part of the regular community CI process. I believe doing so would be an easy way to provide early warning that there are integration level problems.

For instance, mod A depends on mod B's interface v1.0. Mod B makes some breaking changes but forgets to increase the major interface version - or doesn't realize its a breaking change. As a result mod A is broken or misbehaving, but since it's not actively being developed or tested anymore this turns into an escape defect... oops!

Approach

  • May be run the API tests (if they exist) against folio-testing or folio-snapshot on a daily basis, outside of the daily build/deployment process.
  • Ideally notifications could be setup so that a team or the responsible maintainer is notified tests fail.
  • * Alternatively, a dashboard that reports API test pass/fail status w/ links to run logs would probably suffice.

Acceptance Criteria

  • Existing API tests are run on a periodic (daily?) basis


 Comments   
Comment by Craig McNally [ 29/Jul/19 ]

These tests are also helpful for catching issues with permissions - Some API tests are implemented such that a temporary test user is created with only relevant permissions and used for all API requests. This has identified cases where required permissions are mistakenly omitted from endpoints in module descriptors - These problems are often tricky to diagnose w/o direct access to the logs.

Comment by John Malconian [ 24/Sep/19 ]

Touched on this issue again with Craig today. Some additional information:

  • The tests are maintained in the public repo, https://github.com/folio-org/folio-api-tests
  • Tests can be invoked via cli with 'newman'
  • Some tests are maintained more than others. Need to identify those which are maintained and those that are less maintained and include the appropriate tests.
Generated at Thu Feb 08 23:18:56 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.