Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError on Python 3.11.0b1 with coverage 6.3.3 #1376

Closed
domdfcoding opened this issue May 15, 2022 · 11 comments
Closed

TypeError on Python 3.11.0b1 with coverage 6.3.3 #1376

domdfcoding opened this issue May 15, 2022 · 11 comments
Labels
bug Something isn't working cant reproduce question Further information is requested

Comments

@domdfcoding
Copy link

Describe the bug
When running coverage 6.3.3 on Python 3.11.0b1 the program immediately exits with:

TypeError: startswith first arg must be bytes or a tuple of bytes, not str

To Reproduce
How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

  1. What version of Python are you using? 3.11.0b1 (compiled from source)
  2. What version of coverage.py shows the problem? The output of coverage debug sys is helpful.
-- sys -------------------------------------------------------
               coverage_version: 6.3.3
                coverage_module: /tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/__init__.py
                         tracer: -none-
                        CTracer: available
           plugins.file_tracers: -none-
            plugins.configurers: -none-
      plugins.context_switchers: -none-
              configs_attempted: .coveragerc
                                 setup.cfg
                                 tox.ini
                                 pyproject.toml
                   configs_read: -none-
                    config_file: None
                config_contents: -none-
                      data_file: -none-
                         python: 3.11.0b1 (tags/v3.11.0b1:8d32a5c, May  8 2022, 10:15:51) [GCC 9.4.0]
                       platform: Linux-5.13.0-40-generic-x86_64-with-glibc2.31
                 implementation: CPython
                     executable: /tmp/cov-issue/venv/bin/python3
                   def_encoding: utf-8
                    fs_encoding: utf-8
                            pid: 2729056
                            cwd: /tmp/cov-issue
                           path: /tmp/cov-issue/venv/bin
                                 /home/domdf/python311/lib/python311.zip
                                 /home/domdf/python311/lib/python3.11
                                 /home/domdf/python311/lib/python3.11/lib-dynload
                                 /tmp/cov-issue/venv/lib/python3.11/site-packages
                    environment: HOME = /home/domdf
                                 PYENV_ROOT = /home/domdf/.pyenv
                                 PYTHONSTARTUP = /home/domdf/.pythonrc
                   command_line: /tmp/cov-issue/venv/bin/coverage debug sys
                sqlite3_version: 2.6.0
         sqlite3_sqlite_version: 3.36.0
             sqlite3_temp_store: 0
        sqlite3_compile_options: COMPILER=gcc-9.3.0; ENABLE_COLUMN_METADATA; ENABLE_DBSTAT_VTAB
                                 ENABLE_FTS3; ENABLE_FTS3_PARENTHESIS; ENABLE_FTS3_TOKENIZER
                                 ENABLE_FTS4; ENABLE_FTS5; ENABLE_JSON1
                                 ENABLE_LOAD_EXTENSION; ENABLE_MATH_FUNCTIONS; ENABLE_PREUPDATE_HOOK
                                 ENABLE_RTREE; ENABLE_SESSION; ENABLE_STMTVTAB
                                 ENABLE_UNLOCK_NOTIFY; ENABLE_UPDATE_DELETE_LIMIT; HAVE_ISNAN
                                 LIKE_DOESNT_MATCH_BLOBS; MAX_SCHEMA_RETRY=25; MAX_VARIABLE_NUMBER=250000
                                 OMIT_LOOKASIDE; SECURE_DELETE; SOUNDEX
                                 SYSTEM_MALLOC; TEMP_STORE=1; THREADSAFE=1
                                 USE_URI
  1. What versions of what packages do you have installed? The output of pip freeze is helpful. coverage==6.3.3
  2. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.
  3. What commands did you run? coverage run t.py (t.py is empty.)

Expected behavior
Coverage runs without error.

Additional context

Full output:
Traceback (most recent call last):
  File "/tmp/cov-issue/venv/bin/coverage", line 8, in <module>
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/cmdline.py", line 943, in main
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/cmdline.py", line 659, in command_line
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/cmdline.py", line 827, in do_run
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/control.py", line 589, in start
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/collector.py", line 340, in start
  File "/home/domdf/python311/lib/python3.11/threading.py", line 72, in settrace
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/control.py", line 344, in _should_trace
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/inorout.py", line 323, in should_trace
TypeError: startswith first arg must be bytes or a tuple of bytes, not str
Exception ignored in: <object repr() failed>
Traceback (most recent call last):
  File "/home/domdf/python311/lib/python3.11/threading.py", line 1527, in _shutdown
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/control.py", line 344, in _should_trace
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/inorout.py", line 323, in should_trace
TypeError: startswith first arg must be bytes or a tuple of bytes, not str
Exception ignored in atexit callback: <bound method Coverage._atexit of <coverage.control.Coverage object at 0x7fb504598b10>>
Traceback (most recent call last):
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/control.py", line 602, in _atexit
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/control.py", line 344, in _should_trace
  File "/tmp/cov-issue/venv/lib/python3.11/site-packages/coverage/inorout.py", line 323, in should_trace
TypeError: startswith first arg must be bytes or a tuple of bytes, not str

Line 323 in inorout.py calls original_filename.startswith('<'). Debugging with pdb, I can see that original_filename is b' ' (a bytestring containing only a space).

I can't reproduce this issue if I build coverage from source, only if I install the wheel from PyPI.

@domdfcoding domdfcoding added bug Something isn't working needs triage labels May 15, 2022
@nedbat
Copy link
Owner

nedbat commented May 15, 2022

@domdfcoding That's very odd, I haven't seen something like that. Do you have other files in the current directory? Can you run coverage run --debug=trace,sys t.py and share the output?

@nedbat
Copy link
Owner

nedbat commented May 15, 2022

Also, there are no wheels for 3.11.0b1, so at best you downloaded the sdist from PyPI. Can you provide more details?

@nedbat nedbat added cant reproduce question Further information is requested and removed needs triage labels May 15, 2022
@mvaled
Copy link

mvaled commented Jun 25, 2022

I have the same error. You can see it here: https://gitlab.com/merchise-autrement/xotl.plato/-/jobs/2639831211

The tests are run with tox -e system-unit using the latest docker image python:3.11-rc (I'm just running with 3.11.0b3). The tox command is:

commands=
   unit: coverage run --rcfile=setup.cfg -m pytest -l --hypothesis-show-statistics --color=no []

@mvaled
Copy link

mvaled commented Jun 25, 2022

Using the image python:3.11.0b3 the error doesn't happen any more: https://gitlab.com/merchise-autrement/xotl.plato/-/jobs/2639850727

@csernazs
Copy link

@nedbat
Copy link
Owner

nedbat commented Jul 14, 2022

@csernazs can you provide instructions for me to reproduce the issue?

@csernazs
Copy link

Hi,

For pytest-httpserver, I could reproduce it with creating a .venv directory and installing the dependencies with poetry:

python3.11 -m venv .venv
make dev

Then if I run:

make cov

It reproduces the issue.

The point here is that .venv/bin/python should be 3.11.0b4 (and it seems b3 is also affected). I think it is probably a bug in cpython.

I have a lot of dependencies there (for pytest-httpserver), and if I create a small venv (without pytest-httpserver) I can't reproduce the issue. So the following works for me:

python3.11 -m venv .venv-small
.venv-small/bin/pip install coverage
.venv-small/bin/coverage run p.py

where p.py is a trivial hello world:

def main():
    print("hello!")

if __name__ == "__main__":
    main()

For me it seems that the availability of other packages in the venv might affect the bug.

@csernazs
Copy link

It is also interesting that if I run coverage on the trivial p.py with the coverage from the venv updated by make dev it also triggers the error.
If I add a simple print like this to should_trace:

        if type(original_filename) == bytes:
            print(f"original_filename = {original_filename}")
            return nope(disp, "bytes isn't a file name")

Then it prints a strange pattern to the stdout:

original_filename = b' '
original_filename = b'             '
original_filename = b'    '
original_filename = b'  '
original_filename = b'   '
original_filename = b'     '
original_filename = b'      '
original_filename = b'        '

@csernazs
Copy link

For me, my problem was that poetry installed an older coverage package, and that was not working with python 3.11.
But with the latest one (6.4.2) my tests work with 3.11 (and 3.10)

I'm sorry for the false alarm.

@nedbat
Copy link
Owner

nedbat commented Jul 15, 2022

Yes, poetry installed 6.2 because your project supports Python 3.6, and 6.2 is the latest coverage.py version that also supports 3.6.

@nedbat nedbat closed this as not planned Won't fix, can't repro, duplicate, stale Jul 15, 2022
@ahogen
Copy link

ahogen commented Oct 19, 2022

This dependency spec seems to help, when using pytest-cov as well.

[tool.poetry.group.test.dependencies]
pytest = "^7.0.0"
pytest-cov = "^4.0.0"
coverage = [
    { extras = [
        "toml",
    ], version = "^6.0", python = ">=3.6,<3.7" },
    { extras = [
        "toml",
    ], version = ">=6.5", python = ">=3.7" },
]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working cant reproduce question Further information is requested
Projects
None yet
Development

No branches or pull requests

5 participants