pytest-scenario: parameterized test case instances and test scenarios.¶
pytest-scenario is a pytest plugin that aims to extend current test parameterization capabilities. After installing pytest-scenario you will be able run a test suite constructed from a JSON formatted test plan (AKA Test Scenario).
Note: pytest-scenario is currently classified as alpha, feel free to contact me with any issue at: https://github.com/OriMenashe
Features¶
- Test parameterization (including fixtures and test arguments).
- Fixture parameterization on a test level.
- Test instantiation - run multiple test instances with different parameters.
- Test ordering - running tests in a thoughtful, user-defined order.
- Test exclusion - excluding unwanted tests during collection stage.
Quickstart¶
Test parameterization is done by using a new test_case marker as follows:
@pytest.fixture def not_parametrized_fixture(): return "I am not parametrized" @pytest.fixture def int_parametrized_fixture(request): return request.param.num @pytest.fixture def string_parametrized_fixture(request): return request.param.string class TestParametrize: @pytest.mark.test_case(fixture_binding=[('first_fixture_place_holder', {'func': 'int_parametrized_fixture', 'scope': 'function', 'params': [('num', 100)] }), ('second_fixture_place_holder', {'func': 'string_parametrized_fixture', 'scope': 'class', 'params': [('string', 'Hello World')] }), ('third_fixture_place_holder', {'func': 'not_parametrized_fixture', 'scope': 'module', }) ], test_params=[('test_param', 1.04)]) def test_fixture_param(self, first_fixture_place_holder, second_fixture_place_holder, third_fixture_place_holder, test_param): print('\n') assert isinstance(first_fixture_place_holder, int) print('first_fixture_place_holder: %d' % first_fixture_place_holder) assert isinstance(second_fixture_place_holder, str) print('second_fixture_place_holder: %s' % second_fixture_place_holder) assert isinstance(third_fixture_place_holder, str) print('third_fixture_place_holder: %s' % third_fixture_place_holder) assert isinstance(test_param, float) print('test_param: %s' % test_param)
- Output:
first_fixture_place_holder: 100 second_fixture_place_holder: Hello World third_fixture_place_holder: I am not parametrized test_param: 1.04 PASSED ====================================== test_fixture_param_persistency finished ======================================
Another test can benefit from privious parameterization if defined in the same scope, i.e.:
@pytest.mark.test_case(fixture_binding=[('fixture_place_holder', {'func': 'string_parametrized_fixture'})]) def test_fixture_param_persistency(self, fixture_place_holder): print('\n') assert isinstance(fixture_place_holder, str) print('fixture_place_holder param: %s' % fixture_place_holder)
- Output:
fixture_place_holder param: Hello World PASSED ====================================== test_fixture_param_persistency finished ======================================
Test scenario is represented by a JSON file located at:
<projects_root>/sut/scenarios/<scenario_name>.json
Below is an example for a scenario named “main scenario”:
[ { "id": 1, "module_name": "tests.test_parametrize", "class_name": "TestParametrize", "test_name": "test_scenario_instantiation", "fixture_binding": { "fixture_place_holder": { "func": "string_parametrized_fixture", "scope": "session", "params": { "string": "Hello" } } }, "test_params": { "test_param": "World" }, "skip": false, "xfail": false }, { "id": 2, "@ref": "sub scenario" } ]
“main scenario” is referencing a second scenario named “sub scenario” (Nesting is supported):
[ { "id": 1, "module_name": "tests.test_parametrize", "class_name": "TestParametrize", "test_name": "test_scenario_instantiation", "fixture_binding": { "fixture_place_holder": { "func": "string_parametrized_fixture", "scope": "session" } }, "test_params": { "test_param": "Bob" }, "skip": false, "xfail": false }, { "id": 2, "module_name": "tests.test_parametrize", "class_name": "TestParametrize", "test_name": "test_scenario_instantiation", "fixture_binding": { "fixture_place_holder": { "func": "string_parametrized_fixture", "scope": "function", "params": { "string": "Bye" } } }, "test_params": { "test_param": "Bob" }, "skip": false, "xfail": false } ]
- Invocation of a test scenario would be done as follows:
~/workspace/projects_root$ py.test tests/ --scenario="main scenario"
- Output:
collected 3 items selected scenario: _ _ _ __ ___ __ _(_)_ __ ___ ___ ___ _ __ __ _ _ __(_) ___ | `_ ` _ \ / _` | | `_ \ / __|/ __/ _ \ `_ \ / _` | `__| |/ _ \ | | | | | | (_| | | | | | \__ \ (_| __/ | | | (_| | | | | (_) | |_| |_| |_|\__,_|_|_| |_| |___/\___\___|_| |_|\__,_|_| |_|\___/ tests/test_parametrize.py::TestParametrize::test_scenario_instantiation[main scenario-1] Hello World PASSED =============================== test_scenario_instantiation[main scenario-1] finished =============================== tests/test_parametrize.py::TestParametrize::test_scenario_instantiation[main scenario-2.sub scenario-1] Hello Bob PASSED ======================= test_scenario_instantiation[main scenario-2.sub scenario-1] finished ======================== tests/test_parametrize.py::TestParametrize::test_scenario_instantiation[main scenario-2.sub scenario-2] Bye Bob PASSED ======================= test_scenario_instantiation[main scenario-2.sub scenario-2] finished ========================