...
Functional test with large dataset?
- Test a simple workflow. You test that the feature/function works.
- No other concurrent workflows or transactions
- We do this now BUT we are not always doing this with a large dataset
- Provide the dataset. Ask SIGs, etc.
- Use Rancher environment for test
- Test done by Development team
- Can be a standalone user story OR acceptance criteria for a story
- This test should be deemed as a criteria for definition of done for the feature (sprint level)
- Can be a test case for Bugfest
...
Performance test (API)?
- Test a simple workflow via API. Evaluate performance.
- May include concurrent users IF applicable.
- Single tenant environment
- Use Rancher environment for test.
- Test done by Development team. Anton Emelianov (Deactivated) and Sergiy Vysotskiy will organize a developers' training on tools.
- May follow these steps to create user story - Steps to create Performance Testing Scenario (for PO) for PTF Team if team is unclear about APIs to test. Anton Emelianov (Deactivated) will schedule a meeting with each development team to answer any questions on creating stories.
- And review document for proposed SLAs > /wiki/spaces/DQA/pages/2658550
- This test should be deemed as a criteria for definition of done for the feature (release level)
- This test should be run anytime new functionality is added AND for each release to measure any degradations or improvements
...
- . If there are degradations then development team must note this issue in Release notes and make all attempts to correct.
Performance test (robust/complex)?
- Test a complex workflow. Complex = A complex workflow has several actions, concurrent workflows, or concurrent transactions transactions, etc.
- Single tenant or Multi-tenant environments.
- Use PTF environments for test
- Test done by PTF team
- May Must follow these steps to create user story - Steps to create Performance Testing Scenario (for PO) for PTF Team.
- Anton Emelianov (Deactivated) will schedule a meeting with each development team to answer any questions on creating stories.
- AND review document for proposed SLAs > /wiki/spaces/DQA/pages/2658550
- This test should be deemed as a criteria for definition of done for the feature (release level)
- This test should be run anytime new functionality is added AND for each release to measure any degradations or improvements. If there are degradations then development team must note this issue in Release notes and make all attempts to correct.
Test Examples
| Test example | What type of test? | Who should test? | Environment | Jmeter required? | JIRA examples | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Test: Upload a large number of records. Test objective: It works when uploading a large number of records. | Functional testing with a large dataset | Development Team (FE/BE) | Rancher environment | No |
| ||||||||||||||||||||||||
Test: Search results Test objective: Test that up to <<TBD number of results>> is returned in <<TBD seconds>> with <<TBD concurrent users>> | Performance (API) | Development Team (FE/BE) | Rancher environment | Yes | |||||||||||||||||||||||||
Test: Search results Test objective: Test that up to <<TBD number of results>> is returned in <<TBD seconds>> with <<TBD concurrent users>> with <<TBD concurrent actions/workflow/transactions>> | Performance | PTF | PTF | Yes | Run test each release to check for degradation -
Measure against PO defined statistics:
Gather baseline statistics:
| ||||||||||||||||||||||||
Test: Inventory instance has 500+ holdings records attached Test objective: Ease of use in navigating through instance record AND holdings records. | Functional (if one record) | Development team | Rancher environment | No |
|