What is a functional test with large dataset?
- Test a simple workflow. You test that the feature/function works.
- No other concurrent workflows or transactions
- We do this now BUT we are not always doing this with a large dataset
- Provide the dataset. Ask SIGs, etc.
- Test done by Development team
- Can be a standalone user story OR acceptance criteria for a story
- This test should be deemed as a criteria for definition of done for the feature (sprint level)
- Can be a test case for Bugfest
What is a performance test (API)?
- Test a simple workflow via API. Evaluate performance.
- May include concurrent users IF applicable.
- Test done by Development team
- May follow these steps to create user story - Steps to create Performance Testing Scenario (for PO) for PTF Team
- And review document for proposed SLAs > /wiki/spaces/DQA/pages/2658550
- This test should be deemed as a criteria for definition of done for the feature (release level)
- This test should be run anytime new functionality is added AND for each release to measure any degradations or improvements
What is a performance test (robust/complex)?
- Test a complex workflow.
- Complex = several actions, concurrent workflows, or concurrent transactions
- Single tenant or Multi-tenant environments
- Test done by PTF team
- May follow these steps to create user story - Steps to create Performance Testing Scenario (for PO) for PTF Team
- AND review document for proposed SLAs > /wiki/spaces/DQA/pages/2658550
- This test should be deemed as a criteria for definition of done for the feature (release level)
- This test should be run anytime new functionality is added AND for each release to measure any degradations or improvements
Test Examples
| Test example | What type of test? | Who should test? | Environment | Jmeter required? |
|---|---|---|---|---|
Test: Upload a large number of records. Test objective: It works when uploading a large number of records. | Functional testing with a large dataset | Development Team (FE/BE) | Rancher environment | No |
Test: Search results Test objective: Test that up to <<TBD number of results>> is returned in <<TBD seconds>> with <<TBD concurrent users>> | Performance (API) | Development Team (FE/BE) | Rancher environment | Yes |
Test: Search results Test objective: Test that up to <<TBD number of results>> is returned in <<TBD seconds>> with <<TBD concurrent users>> with <<TBD concurrent actions/workflow/transactions>> | Performance | PTF | PTF | Yes |
Test: Inventory instance has 500+ holdings records attached Test objective: Ease of use in navigating through instance record AND holdings records. | Functional (if one record) | Development team | Rancher environment |