Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more powerful set assertion rewrites for comparisons other than equality == (eg: <=, etc.) #10617

Closed
dougthor42 opened this issue Dec 28, 2022 · 0 comments · Fixed by #11469
Labels
topic: rewrite related to the assertion rewrite mechanism type: enhancement new feature or API change, should be merged into features branch

Comments

@dougthor42
Copy link
Contributor

What's the problem this feature will solve?

Set comparisons that do not check for pure equality do not have nice assertion rewriting.

Examples:

# Set equality assertion message is nice! 👍
>       assert {1, 2, 3} == {1, 2, 4}
E       assert {1, 2, 3} == {1, 2, 4}
E         Extra items in the left set:
E         3
E         Extra items in the right set:
E         4
E         Use -v to get more diff

# But others are not nice... 👎
# inequality
>       assert {1, 2, 3} != {1, 2, 3}
E       assert {1, 2, 3} != {1, 2, 3}

# proper subset
>       assert {1, 2, 3} < {1, 2, 3}
E       assert {1, 2, 3} < {1, 2, 3}

# proper superset
>       assert {1, 2, 3} > {1, 2, 3}
E       assert {1, 2, 3} > {1, 2, 3}

# superset
>       assert {1, 2, 3} >= {1, 2, 4}
E       assert {1, 2, 3} >= {1, 2, 4}

# subset
>       assert {1, 2, 3} <= {1, 2, 4}
E       assert {1, 2, 3} <= {1, 2, 4}

Describe the solution you'd like

When asserting sets are unequal, such as set_a <= set_b, the assertion text should be more intelligent only displaying the values that fail the check.

This might look like:

# inequality
>       assert {1, 2, 3} != {1, 2, 3}
E       assert {1, 2, 3} != {1, 2, 3}
E         Sets are equal

# proper subset
>       assert {1, 2, 3} < {1, 2, 3}
E       assert {1, 2, 3} < {1, 2, 3}
E         Sets are equal

# proper superset
>       assert {1, 2, 3} > {1, 2, 3}
E       assert {1, 2, 3} > {1, 2, 3}
E         Sets are equal

# superset
>       assert {1, 2, 3} >= {1, 2, 4}
E       assert {1, 2, 3} >= {1, 2, 4}
E         Extra items in the right set:
E         4
E         Use -v to get more diff

# subset
>       assert {1, 2, 3} <= {1, 2, 4}
E       assert {1, 2, 3} <= {1, 2, 4}
E         Extra items in the left set:
E         3
E         Use -v to get more diff

Alternative Solutions

I'm not aware of any plugins or other tools that implement this, but I have creating my own using pytest_assertrepr_compare:

def pytest_assertrepr_compare(op, left, right):
    if isinstance(left, set) and isinstance(right, set) and op == "!=":
        return [
            f"{left} {op} {right}",  # N.B.: I know this isn't the _actual_ way to do this, but it's good enough for the example!
            "Sets are equal"
        ]
    if isinstance(left, set) and isinstance(right, set) and op == "<=":
        extra = map(str, left - right)
        return [
            f"{left} {op} {right}",
            "Extra items in left set:",
            *extra,
        ]
    if isinstance(left, set) and isinstance(right, set) and op == "<":
        if left == right:
            return [
                f"{left} {op} {right}",
                "Sets are equal"
            ]
        extra = map(str, left - right)
        return [
            f"{left} {op} {right}",
            "Extra items in left set:",
            *extra,
        ]
    if isinstance(left, set) and isinstance(right, set) and op == ">=":
        extra = map(str, right - left)
        return [
            f"{left} {op} {right}",
            "Extra items in right set:",
            *extra,
        ]
    if isinstance(left, set) and isinstance(right, set) and op == ">":
        if left == right:
            return [
                f"{left} {op} {right}",
                " Sets are equal",
            ]
        extra = map(str, right - left)
        return [
            f"{left} {op} {right}",
            "Extra items in right set:",
            *extra,
        ]

We then get the following:

$ pytest pytest_set_assertion.py 
======================================== test session starts =========================================
platform linux -- Python 3.9.13, pytest-7.2.0, pluggy-1.0.0
rootdir: /usr/local/google/home/dthor/dev/pytest_set_assertrepr
collected 8 items                                                                                    

pytest_set_assertion.py FFFFFFFF                                                               [100%]

============================================== FAILURES ==============================================
___________________________________________ test_equality ____________________________________________

    def test_equality():
>       assert {1, 2, 3} == {1, 2, 4, 5}
E       assert {1, 2, 3} == {1, 2, 4, 5}
E         Extra items in the left set:
E         3
E         Extra items in the right set:
E         4
E         5
E         Use -v to get more diff

pytest_set_assertion.py:2: AssertionError
__________________________________________ test_inequality ___________________________________________

    def test_inequality():
>       assert {1, 2, 3} != {1, 2, 3}
E       assert {1, 2, 3} != {1, 2, 3}
E         Sets are equal

pytest_set_assertion.py:6: AssertionError
____________________________________________ test_subset _____________________________________________

    def test_subset():
>       assert {1, 2, 3} <= {1, 2, 4}
E       assert {1, 2, 3} <= {1, 2, 4}
E         Extra items in left set:
E         3

pytest_set_assertion.py:10: AssertionError
_________________________________________ test_proper_subset _________________________________________

    def test_proper_subset():
>       assert {1, 2, 3} < {1, 2, 4}
E       assert {1, 2, 3} < {1, 2, 4}
E         Extra items in left set:
E         3

pytest_set_assertion.py:14: AssertionError
___________________________________ test_proper_subset_equal_sets ____________________________________

    def test_proper_subset_equal_sets():
>       assert {1, 2, 3} < {1, 2, 3}
E       assert {1, 2, 3} < {1, 2, 3}
E         Sets are equal

pytest_set_assertion.py:18: AssertionError
___________________________________________ test_superset ____________________________________________

    def test_superset():
>       assert {1, 2, 3} >= {1, 2, 4}
E       assert {1, 2, 3} >= {1, 2, 4}
E         Extra items in right set:
E         4

pytest_set_assertion.py:22: AssertionError
________________________________________ test_proper_superset ________________________________________

    def test_proper_superset():
>       assert {1, 2, 3} > {1, 2, 4}
E       assert {1, 2, 3} > {1, 2, 4}
E         Extra items in right set:
E         4

pytest_set_assertion.py:26: AssertionError
__________________________________ test_proper_superset_equal_sets ___________________________________

    def test_proper_superset_equal_sets():
>       assert {1, 2, 3} > {1, 2, 3}
E       assert {1, 2, 3} > {1, 2, 3}
E          Sets are equal

pytest_set_assertion.py:30: AssertionError
====================================== short test summary info =======================================
FAILED pytest_set_assertion.py::test_equality - assert {1, 2, 3} == {1, 2, 4, 5}
FAILED pytest_set_assertion.py::test_inequality - assert {1, 2, 3} != {1, 2, 3}
FAILED pytest_set_assertion.py::test_subset - assert {1, 2, 3} <= {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_subset - assert {1, 2, 3} < {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_subset_equal_sets - assert {1, 2, 3} < {1, 2, 3}
FAILED pytest_set_assertion.py::test_superset - assert {1, 2, 3} >= {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_superset - assert {1, 2, 3} > {1, 2, 4}
FAILED pytest_set_assertion.py::test_proper_superset_equal_sets - assert {1, 2, 3} > {1, 2, 3}
========================================= 8 failed in 0.06s ==========================================

Additional context

  • Python 3.9.13 on Debian
  • pytest 7.2.0
@Zac-HD Zac-HD added type: enhancement new feature or API change, should be merged into features branch topic: rewrite related to the assertion rewrite mechanism labels Jan 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: rewrite related to the assertion rewrite mechanism type: enhancement new feature or API change, should be merged into features branch
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants