-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
0.0.8: pytest is failing in few units #52
Comments
Does specifying PYTHONPATH in 'passenv' in the configuration at https://github.com/fedora-python/tox-current-env/blob/master/tests/fixtures/tox.ini help? |
You should be able to run:
In %check to simulate that. That should get around couple of the failures. |
This seems like a difference in sorting. Maybe tox sorts capital letters first? |
This I don't fully comprehend. |
Partial fix for fedora-python#52
Partial fix for fedora-python#52
Could you please test that assumption in #54 ? |
What you mean |
Just tested tjose three commits from master and tjose units are still failing. ========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_integration.py:136: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
SKIPPED [1] tests/test_integration.py:117: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
SKIPPED [1] tests/test_integration.py:459: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
XFAIL tests/test_integration.py::test_allenvs_print_deps_to_file_print_extras_to_other_file[True-True] - reason: Unsupported combination of parameters
FAILED tests/test_integration.py::test_noquiet_installed_packages[--print-deps-to=-] - AssertionError: assert ['Babel==2.11...==9.3.0', ...] == ['alabaster==...=2.11.0', ...]
FAILED tests/test_integration.py::test_noquiet_installed_packages[--current-env] - AssertionError: assert ['Babel==2.11...==9.3.0', ...] == ['alabaster==...=2.11.0', ...]
FAILED tests/test_integration.py::test_regular_run_native_toxenv - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
FAILED tests/test_integration.py::test_regular_after_first_print_deps_is_supported[--print-deps-to=-] - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
FAILED tests/test_integration.py::test_regular_after_first_print_deps_is_supported[--print-deps-to-file=-] - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
FAILED tests/test_integration.py::test_noquiet_installed_packages[None] - AssertionError: assert 4 == 3
FAILED tests/test_integration.py::test_regular_after_first_print_deps_is_supported[--print-deps-only] - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
=========================================================== 7 failed, 122 passed, 3 skipped, 1 xfailed in 17.44s =========================================================== |
Either
|
What 3 commits from master? There are no commits in master related to any of the failures mentioned here. |
Yes commits from master. |
OK with that ebv variable I have now three failing units ========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_integration.py:136: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
SKIPPED [1] tests/test_integration.py:117: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
SKIPPED [1] tests/test_integration.py:459: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
XFAIL tests/test_integration.py::test_allenvs_print_deps_to_file_print_extras_to_other_file[True-True] - reason: Unsupported combination of parameters
FAILED tests/test_integration.py::test_noquiet_installed_packages[--current-env] - AssertionError: assert ['Babel==2.11...==9.3.0', ...] == ['alabaster==...=2.11.0', ...]
FAILED tests/test_integration.py::test_noquiet_installed_packages[--print-deps-to=-] - AssertionError: assert ['Babel==2.11...==9.3.0', ...] == ['alabaster==...=2.11.0', ...]
FAILED tests/test_integration.py::test_noquiet_installed_packages[None] - AssertionError: assert 4 == 3
=========================================================== 3 failed, 126 passed, 3 skipped, 1 xfailed in 17.29s =========================================================== |
and if you add #54 ? |
O with above input resources Source: %{VCS}/archive/v%{version}/%{name}-%{version}.tar.gz
Patch: %{VCS}/commit/d2f93448.patch#/%{name}-tox-never-gets-capitalized.patch
Patch: %{VCS}/commit/4f97be79.patch#/%{name}-Define-and-use-a-tests-extra.patch
Patch: %{VCS}/commit/2f100547.patch#/%{name}-Update-the-branch-for-tox3-git.patch
Patch: %{VCS}/pull/54.patch#/%{name}-Tests-Dont-assume-tox-sorts-by-lowercase.patch I still have one failing unit __________________________________________________________________ test_noquiet_installed_packages[None] ___________________________________________________________________
[gw25] linux -- Python 3.8.16 /usr/bin/python3
flag = None
@pytest.mark.parametrize("flag", [None, "--print-deps-to=-", "--current-env"])
def test_noquiet_installed_packages(flag):
flags = (flag,) if flag else ()
result = tox("-e", NATIVE_TOXENV, *flags, quiet=False, check=False)
assert f"\n{NATIVE_TOXENV} installed: " in result.stdout
for line in result.stdout.splitlines():
if line.startswith(f"{NATIVE_TOXENV} installed: "):
packages = line.rpartition(" installed: ")[-1].split(",")
break
# default tox produces output sorted by package names
assert packages == sorted(
packages, key=lambda p: p.partition("==")[0].partition(" @ ")[0]
)
# without a flag, the output must match tox defaults
if not flag:
> assert len(packages) == 3
E AssertionError: assert 4 == 3
E + where 4 = len(['py==1.11.0', 'six==1.16.0', 'test @ file:///tmp/pytest-of-tkloczko/pytest-681/popen-gw25/test_noquiet_installed_package0/projdir/.tox/.tmp/package/1/test-0.0.0.zip', 'tox-current-env==0.0.8'])
/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:625: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
GLOB sdist-make: /tmp/pytest-of-tkloczko/pytest-681/popen-gw25/test_noquiet_installed_package0/projdir/setup.py
py38 create: /tmp/pytest-of-tkloczko/pytest-681/popen-gw25/test_noquiet_installed_package0/projdir/.tox/py38
py38 installdeps: six, py
py38 inst: /tmp/pytest-of-tkloczko/pytest-681/popen-gw25/test_noquiet_installed_package0/projdir/.tox/.tmp/package/1/test-0.0.0.zip
py38 installed: py==1.11.0,six==1.16.0,test @ file:///tmp/pytest-of-tkloczko/pytest-681/popen-gw25/test_noquiet_installed_package0/projdir/.tox/.tmp/package/1/test-0.0.0.zip,tox-current-env==0.0.8
py38 run-test-pre: PYTHONHASHSEED='2206847846'
py38 run-test: commands[0] | python -c 'import os, sys; print(os.path.realpath(sys.exec_prefix), "is the exec_prefix")'
/tmp/pytest-of-tkloczko/pytest-681/popen-gw25/test_noquiet_installed_package0/projdir/.tox/py38 is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_integration.py:136: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
SKIPPED [1] tests/test_integration.py:117: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
SKIPPED [1] tests/test_integration.py:459: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
XFAIL tests/test_integration.py::test_allenvs_print_deps_to_file_print_extras_to_other_file[True-True] - reason: Unsupported combination of parameters
FAILED tests/test_integration.py::test_noquiet_installed_packages[None] - AssertionError: assert 4 == 3
=========================================================== 1 failed, 128 passed, 3 skipped, 1 xfailed in 16.97s =========================================================== |
I can reproduce that one only with TOX_TESTENV_PASSENV=PYTHONPATH set but not without. The TOX_TESTENV_PASSENV=PYTHONPATH thing is a workaround, I guess the proper fix is to ignore the warning in the test output. Let me check. |
Don't worry about pytjon 3.8. Nevertheless as you are able to reporoduce that |
Partial fix for fedora-python#52
Could you please try the following combination?
|
Partial fix for fedora-python#52
I do have the exactly same problem:
|
This problem is gone with |
As latest |
Just found that another two units are now failing + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-tox-current-env-0.0.11-3.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-tox-current-env-0.0.11-3.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network' --deselect 'tests/test_integration.py::test_noquiet_installed_packages[None]'
============================= test session starts ==============================
platform linux -- Python 3.8.16, pytest-7.2.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.11, configfile: tox.ini
plugins: xdist-3.2.0
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I / gw12 I / gw13 I / gw14 I / gw15 I / gw16 I / gw17 I / gw18 I / gw19 I / gw20 I / gw21 I / gw22 I / gw23 I / gw24 I / gw25 I / gw26 I / gw27 I / gw28 I / gw29 I / gw30 I / gw31 I / gw32 I / gw33 I / gw34 I / gw35 I / gw36 I / gw37 I / gw38 I / gw39 I / gw40 I / gw41 I / gw42 I / gw43 I / gw44 I / gw45 I / gw46 I / gw47 I
gw0 [259] / gw1 [259] / gw2 [259] / gw3 [259] / gw4 [259] / gw5 [259] / gw6 [259] / gw7 [259] / gw8 [259] / gw9 [259] / gw10 [259] / gw11 [259] / gw12 [259] / gw13 [259] / gw14 [259] / gw15 [259] / gw16 [259] / gw17 [259] / gw18 [259] / gw19 [259] / gw20 [259] / gw21 [259] / gw22 [259] / gw23 [259] / gw24 [259] / gw25 [259] / gw26 [259] / gw27 [259] / gw28 [259] / gw29 [259] / gw30 [259] / gw31 [259] / gw32 [259] / gw33 [259] / gw34 [259] / gw35 [259] / gw36 [259] / gw37 [259] / gw38 [259] / gw39 [259] / gw40 [259] / gw41 [259] / gw42 [259] / gw43 [259] / gw44 [259] / gw45 [259] / gw46 [259] / gw47 [259]
s................................................s...................... [ 27%]
........................................................................ [ 55%]
...........................s...........x................................ [ 83%]
......F...........F........................ [100%]current /home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.11 running in /home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.11
3.26.0 imported from /usr/lib/python3.8/site-packages/tox/__init__.py
registered plugins:
tox-current-env-0.0.11 at /home/tkloczko/rpmbuild/BUILDROOT/python-tox-current-env-0.0.11-3.fc35.x86_64/usr/lib/python3.8/site-packages/tox_current_env/hooks.py
=================================== FAILURES ===================================
________________ test_noquiet_installed_packages[--current-env] ________________
[gw13] linux -- Python 3.8.16 /usr/bin/python3
flag = '--current-env'
@pytest.mark.parametrize("flag", [None, "--print-deps-to=-", "--current-env"])
def test_noquiet_installed_packages(flag):
flags = (flag,) if flag else ()
result = tox("-e", NATIVE_TOXENV, *flags, quiet=False, check=False)
assert f"\n{NATIVE_TOXENV} installed: " in result.stdout
for line in result.stdout.splitlines():
if line.startswith(f"{NATIVE_TOXENV} installed: "):
packages = line.rpartition(" installed: ")[-1].split(",")
break
# default tox produces output sorted by package names
> assert packages == sorted(
packages, key=lambda p: p.partition("==")[0].partition(" @ ")[0].lower()
)
E AssertionError: assert ['Babel==2.11...=0.7.13', ...] == ['alabaster==...==0.3.6', ...]
E At index 0 diff: 'Babel==2.11.0' != 'alabaster==0.7.13'
E Use -v to get more diff
/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.11/tests/test_integration_tox3.py:520: AssertionError
----------------------------- Captured stdout call -----------------------------
current /tmp/pytest-of-tkloczko/pytest-375/popen-gw13/test_noquiet_installed_package0/projdir running in /tmp/pytest-of-tkloczko/pytest-375/popen-gw13/test_noquiet_installed_package0/projdir
py38 create: /tmp/pytest-of-tkloczko/pytest-375/popen-gw13/test_noquiet_installed_package0/projdir/.tox/py38
py38 installed: Babel==2.11.0,Jinja2==3.1.2,MarkupSafe==2.1.2,Pygments==2.14.0,Sphinx==6.1.3,alabaster==0.7.13,attrs==22.2.0,build==0.9.0,charset-normalizer==3.0.1,distlib==0.3.6,distro==1.8.0,docutils==0.19,exceptiongroup==1.0.0,execnet==1.9.0,extras==1.0.0,filelock==3.8.2,fixtures==4.0.0,gpg==1.18.0-unknown,idna==3.4,imagesize==1.4.1,importlib-metadata==6.0.0,iniconfig==2.0.0,libcomps==0.1.19,packaging==23.0,pbr==5.9.0,pep517==0.13.0,pip==22.3.1,platformdirs==2.6.0,pluggy==1.0.0,py==1.11.0,pytest==7.2.1,pytest-xdist==3.2.0,python-dateutil==2.8.2,pytz==2022.4,requests==2.28.2,rpm==4.17.0,setuptools==65.6.3,six==1.16.0,snowballstemmer==2.2.0,sphinxcontrib-applehelp==1.0.2.dev20221204,sphinxcontrib-devhelp==1.0.2.dev20230202,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1.dev20230128,sphinxcontrib-qthelp==1.0.3.dev20230128,sphinxcontrib-serializinghtml==1.1.5,testtools==2.5.0,toml==0.10.2,tomli==2.0.1,tox==3.26.0,tox-current-env==0.0.11,urllib3==1.26.12,virtualenv==20.17.1,wheel==0.38.4,zipp==3.13.0
py38 run-test-pre: PYTHONHASHSEED='1007731709'
py38 run-test: commands[0] | python -c 'import os, sys; print(os.path.realpath(sys.exec_prefix), "is the exec_prefix")'
/usr is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)
----------------------------- Captured stderr call -----------------------------
______________ test_noquiet_installed_packages[--print-deps-to=-] ______________
[gw0] linux -- Python 3.8.16 /usr/bin/python3
flag = '--print-deps-to=-'
@pytest.mark.parametrize("flag", [None, "--print-deps-to=-", "--current-env"])
def test_noquiet_installed_packages(flag):
flags = (flag,) if flag else ()
result = tox("-e", NATIVE_TOXENV, *flags, quiet=False, check=False)
assert f"\n{NATIVE_TOXENV} installed: " in result.stdout
for line in result.stdout.splitlines():
if line.startswith(f"{NATIVE_TOXENV} installed: "):
packages = line.rpartition(" installed: ")[-1].split(",")
break
# default tox produces output sorted by package names
> assert packages == sorted(
packages, key=lambda p: p.partition("==")[0].partition(" @ ")[0].lower()
)
E AssertionError: assert ['Babel==2.11...=0.7.13', ...] == ['alabaster==...==0.3.6', ...]
E At index 0 diff: 'Babel==2.11.0' != 'alabaster==0.7.13'
E Use -v to get more diff
/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.11/tests/test_integration_tox3.py:520: AssertionError
----------------------------- Captured stdout call -----------------------------
current /tmp/pytest-of-tkloczko/pytest-375/popen-gw0/test_noquiet_installed_package0/projdir running in /tmp/pytest-of-tkloczko/pytest-375/popen-gw0/test_noquiet_installed_package0/projdir
py38 create: /tmp/pytest-of-tkloczko/pytest-375/popen-gw0/test_noquiet_installed_package0/projdir/.tox/py38
py38 installed: Babel==2.11.0,Jinja2==3.1.2,MarkupSafe==2.1.2,Pygments==2.14.0,Sphinx==6.1.3,alabaster==0.7.13,attrs==22.2.0,build==0.9.0,charset-normalizer==3.0.1,distlib==0.3.6,distro==1.8.0,docutils==0.19,exceptiongroup==1.0.0,execnet==1.9.0,extras==1.0.0,filelock==3.8.2,fixtures==4.0.0,gpg==1.18.0-unknown,idna==3.4,imagesize==1.4.1,importlib-metadata==6.0.0,iniconfig==2.0.0,libcomps==0.1.19,packaging==23.0,pbr==5.9.0,pep517==0.13.0,pip==22.3.1,platformdirs==2.6.0,pluggy==1.0.0,py==1.11.0,pytest==7.2.1,pytest-xdist==3.2.0,python-dateutil==2.8.2,pytz==2022.4,requests==2.28.2,rpm==4.17.0,setuptools==65.6.3,six==1.16.0,snowballstemmer==2.2.0,sphinxcontrib-applehelp==1.0.2.dev20221204,sphinxcontrib-devhelp==1.0.2.dev20230202,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1.dev20230128,sphinxcontrib-qthelp==1.0.3.dev20230128,sphinxcontrib-serializinghtml==1.1.5,testtools==2.5.0,toml==0.10.2,tomli==2.0.1,tox==3.26.0,tox-current-env==0.0.11,urllib3==1.26.12,virtualenv==20.17.1,wheel==0.38.4,zipp==3.13.0
py38 run-test-pre: PYTHONHASHSEED='2285002353'
six
py
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)
----------------------------- Captured stderr call -----------------------------
=========================== short test summary info ============================
SKIPPED [1] tests/test_integration_tox4.py:24: skipping tests for tox 4
SKIPPED [1] tests/test_integration_tox3.py:55: This test needs all pythons from 3.6 to 3.11 available in $PATH
SKIPPED [1] tests/test_integration_tox3.py:36: This test needs all pythons from 3.6 to 3.11 available in $PATH
SKIPPED [1] tests/test_integration_tox3.py:366: This test needs all pythons from 3.6 to 3.11 available in $PATH
XFAIL tests/test_integration_tox3.py::test_allenvs_print_deps_to_file_print_extras_to_other_file[True-True] - reason: Unsupported combination of parameters
FAILED tests/test_integration_tox3.py::test_noquiet_installed_packages[--current-env]
FAILED tests/test_integration_tox3.py::test_noquiet_installed_packages[--print-deps-to=-]
============= 2 failed, 253 passed, 4 skipped, 1 xfailed in 17.29s ============= |
Partial fix for fedora-python#52
OK .. one sec 😋 |
That PR does nit applies on top of latest version + /usr/bin/tar -xof -
+ /usr/bin/gzip -dc /home/tkloczko/rpmbuild/SOURCES/python-tox-current-env-0.0.11.tar.gz
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd tox-current-env-0.0.11
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ /usr/bin/cat /home/tkloczko/rpmbuild/SOURCES/python-tox-current-env-Tests-Dont-assume-tox-sorts-by-lowercase.patch
+ /usr/bin/patch -p1 -s --fuzz=0 --no-backup-if-mismatch -f
The text leading up to this was:
--------------------------
|From 096d986fd61de9d291eac6756c2385dd43de03f1 Mon Sep 17 00:00:00 2001
|From: =?UTF-8?q?Miro=20Hron=C4=8Dok?= <[email protected]>
|Date: Thu, 8 Dec 2022 11:48:14 +0100
|Subject: [PATCH] Tests: Don't assume tox sorts by lowercase
|
|Partial fix for https://github.com/fedora-python/tox-current-env/issues/52
|---
| tests/test_integration.py | 2 +-
| 1 file changed, 1 insertion(+), 1 deletion(-)
|
|diff --git a/tests/test_integration.py b/tests/test_integration.py
|index 09abd46..a6dd78b 100644
|--- a/tests/test_integration.py
|+++ b/tests/test_integration.py
--------------------------
No file to patch. Skipping patch.
1 out of 1 hunk ignored |
It was rebased (before your comment). |
I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.
python3 -sBm build -w --no-isolation
build
with--no-isolation
I'm using during all processes only locally installed modulesHere is pytest output:
Here is list of installed modules in build env
The text was updated successfully, but these errors were encountered: