Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test parametrization does not affect the fixture dependency tree #2220

Closed
sermoz opened this issue Jan 24, 2017 · 8 comments
Closed

Test parametrization does not affect the fixture dependency tree #2220

sermoz opened this issue Jan 24, 2017 · 8 comments
Labels
topic: parametrize related to @pytest.mark.parametrize type: bug problem that needs to be addressed

Comments

@sermoz
Copy link
Contributor

sermoz commented Jan 24, 2017

pytest v3.0.5

@pytest.fixture
def fix1(fix2):
   return fix2 + '1'

@pytest.fixture
def fix2(fix3):
   return fix3 + '2'

@pytest.fixture
def fix3():
   print("fix3 instantiated!!")
   return '3'

@pytest.mark.parametrize('fix2', ['2'])
def test_it(fix1):
   assert fix1 == '21'

This example demonstrates that fix3 is still instantiated, although fix2 - fix3 dependency should have been broken because of parametrization.

@nicoddemus
Copy link
Member

Thanks for the report! 👍

@nicoddemus nicoddemus added type: bug problem that needs to be addressed topic: parametrize related to @pytest.mark.parametrize labels Jan 25, 2017
@sermoz
Copy link
Contributor Author

sermoz commented May 15, 2017

Can I expect this to be fixed by some date? Is this issue included in any future release?

@The-Compiler
Copy link
Member

The-Compiler commented May 15, 2017

Nobody is currently working on this - so unless you submit a fix yourself (or get a support contract with Merlinux), there isn't really a guarantee things will get fixed at some point. Keep in mind we're all volunteers 😉

@timofurrer
Copy link

What is the expected behavior for the above example?
I might be able to work on it.

@nicoddemus
Copy link
Member

@timofurrer

Here's a more complete example:

import pytest

@pytest.fixture
def fix1(fix2):
   return fix2 + '1'

@pytest.fixture
def fix2(fix3):
    print('fix2 instantiated!!')
    return fix3 + '2'

@pytest.fixture
def fix3():
    print("fix3 instantiated!!")
    return '3'

@pytest.mark.parametrize('fix2', ['2'])
def test_it(fix1):
   assert fix1 == '21'

Running pytest -s:

fix3 instantiated!!
.
1 passed in 0.01 seconds

As can be seen, even though fix2 was not instantiated because it was shadowed by the parametrize call in test_it, fix3 was still instantiated because of the fix2 dependency to fix3.

At least that's how I understood it, @egnartsms feel free to correct me.

@timofurrer
Copy link

fix1 is instantiated, too. Which of the fixtures should be instantiated? Non of them?

@nicoddemus
Copy link
Member

@timofurrer yeah it is a little confusing at first, here's a summary of what's happening:

  • fixture fix1 should be instantiated because test_it explicitly asks for it in its parameter list;
  • parameter fix2 is injected into test_it because of the parametrize call;
  • fixture fix2 is never instantiated because it is not part of the parameter list of test_it;
  • fixture fix3 should never be instantiated because, while it is a dependency of fixture fix2, fix2 itself was not instantiated and so shouldn't fix3.

It seems fixture fix2 is at some point being considered a dependency of test_it (which implicitly also makes fix3 a dependency), and then later fix2 is replaced by the parameter fix2 but fix3 is kept. It warrants more investigation for sure. 😅

@nicoddemus
Copy link
Member

Fixed by #3629, thanks @egnartsms!

cmccandless referenced this issue in cmccandless/tools Jul 31, 2018
This PR updates [pytest](https://pypi.org/project/pytest) from **3.6.4** to **3.7.0**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.7.0
   ```
   =========================

Deprecations and Removals
-------------------------

- `2639 &lt;https://github.com/pytest-dev/pytest/issues/2639&gt;`_: ``pytest_namespace`` has been deprecated.

  See the documentation for ``pytest_namespace`` hook for suggestions on how to deal
  with this in plugins which use this functionality.


- `3661 &lt;https://github.com/pytest-dev/pytest/issues/3661&gt;`_: Calling a fixture function directly, as opposed to request them in a test function, now issues a ``RemovedInPytest4Warning``. It will be changed into an error in pytest ``4.0``.

  This is a great source of confusion to new users, which will often call the fixture functions and request them from test functions interchangeably, which breaks the fixture resolution model.



Features
--------

- `2283 &lt;https://github.com/pytest-dev/pytest/issues/2283&gt;`_: New ``package`` fixture scope: fixtures are finalized when the last test of a *package* finishes. This feature is considered **experimental**, so use it sparingly.


- `3576 &lt;https://github.com/pytest-dev/pytest/issues/3576&gt;`_: ``Node.add_marker`` now supports an ``append=True/False`` parameter to determine whether the mark comes last (default) or first.


- `3579 &lt;https://github.com/pytest-dev/pytest/issues/3579&gt;`_: Fixture ``caplog`` now has a ``messages`` property, providing convenient access to the format-interpolated log messages without the extra data provided by the formatter/handler.


- `3610 &lt;https://github.com/pytest-dev/pytest/issues/3610&gt;`_: New ``--trace`` option to enter the debugger at the start of a test.


- `3623 &lt;https://github.com/pytest-dev/pytest/issues/3623&gt;`_: Introduce ``pytester.copy_example`` as helper to do acceptance tests against examples from the project.



Bug Fixes
---------

- `2220 &lt;https://github.com/pytest-dev/pytest/issues/2220&gt;`_: Fix a bug where fixtures overridden by direct parameters (for example parametrization) were being instantiated even if they were not being used by a test.


- `3695 &lt;https://github.com/pytest-dev/pytest/issues/3695&gt;`_: Fix ``ApproxNumpy`` initialisation argument mixup, ``abs`` and ``rel`` tolerances were flipped causing strange comparsion results.
  Add tests to check ``abs`` and ``rel`` tolerances for ``np.array`` and test for expecting ``nan`` with ``np.array()``


- `980 &lt;https://github.com/pytest-dev/pytest/issues/980&gt;`_: Fix truncated locals output in verbose mode.



Improved Documentation
----------------------

- `3295 &lt;https://github.com/pytest-dev/pytest/issues/3295&gt;`_: Correct the usage documentation of ``--last-failed-no-failures`` by adding the missing ``--last-failed`` argument in the presented examples, because they are misleading and lead to think that the missing argument is not needed.



Trivial/Internal Changes
------------------------

- `3519 &lt;https://github.com/pytest-dev/pytest/issues/3519&gt;`_: Now a ``README.md`` file is created in ``.pytest_cache`` to make it clear why the directory exists.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Repo: https://github.com/pytest-dev/pytest/issues
  - Homepage: http://pytest.org
</details>
jezdez referenced this issue in mozilla/telemetry-analysis-service Aug 14, 2018
This PR updates [pytest](https://pypi.org/project/pytest) from **3.6.2** to **3.7.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.7.1
   ```
   =========================

Bug Fixes
---------

- `3473 &lt;https://github.com/pytest-dev/pytest/issues/3473&gt;`_: Raise immediately if ``approx()`` is given an expected value of a type it doesn&#39;t understand (e.g. strings, nested dicts, etc.).


- `3712 &lt;https://github.com/pytest-dev/pytest/issues/3712&gt;`_: Correctly represent the dimensions of an numpy array when calling ``repr()`` on ``approx()``.

- `3742 &lt;https://github.com/pytest-dev/pytest/issues/3742&gt;`_: Fix incompatibility with third party plugins during collection, which produced the error ``object has no attribute &#39;_collectfile&#39;``.

- `3745 &lt;https://github.com/pytest-dev/pytest/issues/3745&gt;`_: Display the absolute path if ``cache_dir`` is not relative to the ``rootdir`` instead of failing.


- `3747 &lt;https://github.com/pytest-dev/pytest/issues/3747&gt;`_: Fix compatibility problem with plugins and the warning code issued by fixture functions when they are called directly.


- `3748 &lt;https://github.com/pytest-dev/pytest/issues/3748&gt;`_: Fix infinite recursion in ``pytest.approx`` with arrays in ``numpy&lt;1.13``.


- `3757 &lt;https://github.com/pytest-dev/pytest/issues/3757&gt;`_: Pin pathlib2 to ``&gt;=2.2.0`` as we require ``__fspath__`` support.


- `3763 &lt;https://github.com/pytest-dev/pytest/issues/3763&gt;`_: Fix ``TypeError`` when the assertion message is ``bytes`` in python 3.
   ```
   
  
  
   ### 3.7.0
   ```
   =========================

Deprecations and Removals
-------------------------

- `2639 &lt;https://github.com/pytest-dev/pytest/issues/2639&gt;`_: ``pytest_namespace`` has been deprecated.

  See the documentation for ``pytest_namespace`` hook for suggestions on how to deal
  with this in plugins which use this functionality.


- `3661 &lt;https://github.com/pytest-dev/pytest/issues/3661&gt;`_: Calling a fixture function directly, as opposed to request them in a test function, now issues a ``RemovedInPytest4Warning``. It will be changed into an error in pytest ``4.0``.

  This is a great source of confusion to new users, which will often call the fixture functions and request them from test functions interchangeably, which breaks the fixture resolution model.



Features
--------

- `2283 &lt;https://github.com/pytest-dev/pytest/issues/2283&gt;`_: New ``package`` fixture scope: fixtures are finalized when the last test of a *package* finishes. This feature is considered **experimental**, so use it sparingly.


- `3576 &lt;https://github.com/pytest-dev/pytest/issues/3576&gt;`_: ``Node.add_marker`` now supports an ``append=True/False`` parameter to determine whether the mark comes last (default) or first.


- `3579 &lt;https://github.com/pytest-dev/pytest/issues/3579&gt;`_: Fixture ``caplog`` now has a ``messages`` property, providing convenient access to the format-interpolated log messages without the extra data provided by the formatter/handler.


- `3610 &lt;https://github.com/pytest-dev/pytest/issues/3610&gt;`_: New ``--trace`` option to enter the debugger at the start of a test.


- `3623 &lt;https://github.com/pytest-dev/pytest/issues/3623&gt;`_: Introduce ``pytester.copy_example`` as helper to do acceptance tests against examples from the project.



Bug Fixes
---------

- `2220 &lt;https://github.com/pytest-dev/pytest/issues/2220&gt;`_: Fix a bug where fixtures overridden by direct parameters (for example parametrization) were being instantiated even if they were not being used by a test.


- `3695 &lt;https://github.com/pytest-dev/pytest/issues/3695&gt;`_: Fix ``ApproxNumpy`` initialisation argument mixup, ``abs`` and ``rel`` tolerances were flipped causing strange comparsion results.
  Add tests to check ``abs`` and ``rel`` tolerances for ``np.array`` and test for expecting ``nan`` with ``np.array()``


- `980 &lt;https://github.com/pytest-dev/pytest/issues/980&gt;`_: Fix truncated locals output in verbose mode.



Improved Documentation
----------------------

- `3295 &lt;https://github.com/pytest-dev/pytest/issues/3295&gt;`_: Correct the usage documentation of ``--last-failed-no-failures`` by adding the missing ``--last-failed`` argument in the presented examples, because they are misleading and lead to think that the missing argument is not needed.



Trivial/Internal Changes
------------------------

- `3519 &lt;https://github.com/pytest-dev/pytest/issues/3519&gt;`_: Now a ``README.md`` file is created in ``.pytest_cache`` to make it clear why the directory exists.
   ```
   
  
  
   ### 3.6.4
   ```
   =========================

Bug Fixes
---------

- Invoke pytest using ``-mpytest`` so ``sys.path`` does not get polluted by packages installed in ``site-packages``. (`742 &lt;https://github.com/pytest-dev/pytest/issues/742&gt;`_)


Improved Documentation
----------------------

- Use ``smtp_connection`` instead of ``smtp`` in fixtures documentation to avoid possible confusion. (`3592 &lt;https://github.com/pytest-dev/pytest/issues/3592&gt;`_)


Trivial/Internal Changes
------------------------

- Remove obsolete ``__future__`` imports. (`2319 &lt;https://github.com/pytest-dev/pytest/issues/2319&gt;`_)

- Add CITATION to provide information on how to formally cite pytest. (`3402 &lt;https://github.com/pytest-dev/pytest/issues/3402&gt;`_)

- Replace broken type annotations with type comments. (`3635 &lt;https://github.com/pytest-dev/pytest/issues/3635&gt;`_)

- Pin ``pluggy`` to ``&lt;0.8``. (`3727 &lt;https://github.com/pytest-dev/pytest/issues/3727&gt;`_)
   ```
   
  
  
   ### 3.6.3
   ```
   =========================

Bug Fixes
---------

- Fix ``ImportWarning`` triggered by explicit relative imports in
  assertion-rewritten package modules. (`3061
  &lt;https://github.com/pytest-dev/pytest/issues/3061&gt;`_)

- Fix error in ``pytest.approx`` when dealing with 0-dimension numpy
  arrays. (`3593 &lt;https://github.com/pytest-dev/pytest/issues/3593&gt;`_)

- No longer raise ``ValueError`` when using the ``get_marker`` API. (`3605
  &lt;https://github.com/pytest-dev/pytest/issues/3605&gt;`_)

- Fix problem where log messages with non-ascii characters would not
  appear in the output log file.
  (`3630 &lt;https://github.com/pytest-dev/pytest/issues/3630&gt;`_)

- No longer raise ``AttributeError`` when legacy marks can&#39;t be stored in
  functions. (`3631 &lt;https://github.com/pytest-dev/pytest/issues/3631&gt;`_)


Improved Documentation
----------------------

- The description above the example for ``pytest.mark.skipif`` now better
  matches the code. (`3611
  &lt;https://github.com/pytest-dev/pytest/issues/3611&gt;`_)


Trivial/Internal Changes
------------------------

- Internal refactoring: removed unused ``CallSpec2tox ._globalid_args``
  attribute and ``metafunc`` parameter from ``CallSpec2.copy()``. (`3598
  &lt;https://github.com/pytest-dev/pytest/issues/3598&gt;`_)

- Silence usage of ``reduce`` warning in Python 2 (`3609
  &lt;https://github.com/pytest-dev/pytest/issues/3609&gt;`_)

- Fix usage of ``attr.ib`` deprecated ``convert`` parameter. (`3653
  &lt;https://github.com/pytest-dev/pytest/issues/3653&gt;`_)
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Repo: https://github.com/pytest-dev/pytest/issues
  - Homepage: http://pytest.org
</details>
cmccandless referenced this issue in cmccandless/multisite Aug 20, 2018
This PR updates [pytest](https://pypi.org/project/pytest) from **3.6.4** to **3.7.2**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.7.2
   ```
   =========================

Bug Fixes
---------

- `3671 &lt;https://github.com/pytest-dev/pytest/issues/3671&gt;`_: Fix ``filterwarnings`` not being registered as a builtin mark.


- `3768 &lt;https://github.com/pytest-dev/pytest/issues/3768&gt;`_, `3789 &lt;https://github.com/pytest-dev/pytest/issues/3789&gt;`_: Fix test collection from packages mixed with normal directories.


- `3771 &lt;https://github.com/pytest-dev/pytest/issues/3771&gt;`_: Fix infinite recursion during collection if a ``pytest_ignore_collect`` hook returns ``False`` instead of ``None``.


- `3774 &lt;https://github.com/pytest-dev/pytest/issues/3774&gt;`_: Fix bug where decorated fixtures would lose functionality (for example ``mock.patch``).


- `3775 &lt;https://github.com/pytest-dev/pytest/issues/3775&gt;`_: Fix bug where importing modules or other objects with prefix ``pytest_`` prefix would raise a ``PluginValidationError``.


- `3788 &lt;https://github.com/pytest-dev/pytest/issues/3788&gt;`_: Fix ``AttributeError`` during teardown of ``TestCase`` subclasses which raise an exception during ``__init__``.


- `3804 &lt;https://github.com/pytest-dev/pytest/issues/3804&gt;`_: Fix traceback reporting for exceptions with ``__cause__`` cycles.



Improved Documentation
----------------------

- `3746 &lt;https://github.com/pytest-dev/pytest/issues/3746&gt;`_: Add documentation for ``metafunc.config`` that had been mistakenly hidden.
   ```
   
  
  
   ### 3.7.1
   ```
   =========================

Bug Fixes
---------

- `3473 &lt;https://github.com/pytest-dev/pytest/issues/3473&gt;`_: Raise immediately if ``approx()`` is given an expected value of a type it doesn&#39;t understand (e.g. strings, nested dicts, etc.).


- `3712 &lt;https://github.com/pytest-dev/pytest/issues/3712&gt;`_: Correctly represent the dimensions of an numpy array when calling ``repr()`` on ``approx()``.

- `3742 &lt;https://github.com/pytest-dev/pytest/issues/3742&gt;`_: Fix incompatibility with third party plugins during collection, which produced the error ``object has no attribute &#39;_collectfile&#39;``.

- `3745 &lt;https://github.com/pytest-dev/pytest/issues/3745&gt;`_: Display the absolute path if ``cache_dir`` is not relative to the ``rootdir`` instead of failing.


- `3747 &lt;https://github.com/pytest-dev/pytest/issues/3747&gt;`_: Fix compatibility problem with plugins and the warning code issued by fixture functions when they are called directly.


- `3748 &lt;https://github.com/pytest-dev/pytest/issues/3748&gt;`_: Fix infinite recursion in ``pytest.approx`` with arrays in ``numpy&lt;1.13``.


- `3757 &lt;https://github.com/pytest-dev/pytest/issues/3757&gt;`_: Pin pathlib2 to ``&gt;=2.2.0`` as we require ``__fspath__`` support.


- `3763 &lt;https://github.com/pytest-dev/pytest/issues/3763&gt;`_: Fix ``TypeError`` when the assertion message is ``bytes`` in python 3.
   ```
   
  
  
   ### 3.7.0
   ```
   =========================

Deprecations and Removals
-------------------------

- `2639 &lt;https://github.com/pytest-dev/pytest/issues/2639&gt;`_: ``pytest_namespace`` has been deprecated.

  See the documentation for ``pytest_namespace`` hook for suggestions on how to deal
  with this in plugins which use this functionality.


- `3661 &lt;https://github.com/pytest-dev/pytest/issues/3661&gt;`_: Calling a fixture function directly, as opposed to request them in a test function, now issues a ``RemovedInPytest4Warning``. It will be changed into an error in pytest ``4.0``.

  This is a great source of confusion to new users, which will often call the fixture functions and request them from test functions interchangeably, which breaks the fixture resolution model.



Features
--------

- `2283 &lt;https://github.com/pytest-dev/pytest/issues/2283&gt;`_: New ``package`` fixture scope: fixtures are finalized when the last test of a *package* finishes. This feature is considered **experimental**, so use it sparingly.


- `3576 &lt;https://github.com/pytest-dev/pytest/issues/3576&gt;`_: ``Node.add_marker`` now supports an ``append=True/False`` parameter to determine whether the mark comes last (default) or first.


- `3579 &lt;https://github.com/pytest-dev/pytest/issues/3579&gt;`_: Fixture ``caplog`` now has a ``messages`` property, providing convenient access to the format-interpolated log messages without the extra data provided by the formatter/handler.


- `3610 &lt;https://github.com/pytest-dev/pytest/issues/3610&gt;`_: New ``--trace`` option to enter the debugger at the start of a test.


- `3623 &lt;https://github.com/pytest-dev/pytest/issues/3623&gt;`_: Introduce ``pytester.copy_example`` as helper to do acceptance tests against examples from the project.



Bug Fixes
---------

- `2220 &lt;https://github.com/pytest-dev/pytest/issues/2220&gt;`_: Fix a bug where fixtures overridden by direct parameters (for example parametrization) were being instantiated even if they were not being used by a test.


- `3695 &lt;https://github.com/pytest-dev/pytest/issues/3695&gt;`_: Fix ``ApproxNumpy`` initialisation argument mixup, ``abs`` and ``rel`` tolerances were flipped causing strange comparsion results.
  Add tests to check ``abs`` and ``rel`` tolerances for ``np.array`` and test for expecting ``nan`` with ``np.array()``


- `980 &lt;https://github.com/pytest-dev/pytest/issues/980&gt;`_: Fix truncated locals output in verbose mode.



Improved Documentation
----------------------

- `3295 &lt;https://github.com/pytest-dev/pytest/issues/3295&gt;`_: Correct the usage documentation of ``--last-failed-no-failures`` by adding the missing ``--last-failed`` argument in the presented examples, because they are misleading and lead to think that the missing argument is not needed.



Trivial/Internal Changes
------------------------

- `3519 &lt;https://github.com/pytest-dev/pytest/issues/3519&gt;`_: Now a ``README.md`` file is created in ``.pytest_cache`` to make it clear why the directory exists.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pytest
  - Changelog: https://pyup.io/changelogs/pytest/
  - Homepage: http://pytest.org
</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: parametrize related to @pytest.mark.parametrize type: bug problem that needs to be addressed
Projects
None yet
Development

No branches or pull requests

4 participants