Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify encoding="utf-8" in all open functions for future proofing against Python 3.15+ #2006

Closed
matthewfeickert opened this issue Sep 14, 2022 · 1 comment · Fixed by #2007
Assignees
Labels
chore Other changes that don't modify src or test files

Comments

@matthewfeickert
Copy link
Member

As @henryiii noted in his PyHEP 2022 talk "What's new in Python 3.11"

3.15 will make unicode default! Always specify an encoding with any open function. You can use "native" encoding in 3.11+ (but, why?).

open(..., encoding="utf-8")
Path(...).read_txt(encoding="utf-8")

which means that encoding="utf-8" should be added to all open() and click.open_file() calls in pyhf.

As of 511d8a9 this includes

git grep --name-only "open(\|open_file("
docs/development.rst
docs/examples/notebooks/ImpactPlot.ipynb
docs/examples/notebooks/binderexample/StatisticalAnalysis.ipynb
docs/examples/notebooks/multiBinPois.ipynb
docs/examples/notebooks/multichannel-coupled-histo.ipynb
docs/examples/notebooks/pullplot.ipynb
docs/likelihood.rst
src/pyhf/cli/infer.py
src/pyhf/cli/patchset.py
src/pyhf/cli/rootio.py
src/pyhf/cli/spec.py
src/pyhf/contrib/utils.py
src/pyhf/readxml.py
src/pyhf/schema/loader.py
src/pyhf/writexml.py
tests/conftest.py
tests/contrib/test_contrib_utils.py
tests/contrib/test_viz.py
tests/test_export.py
tests/test_modifiers.py
tests/test_patchset.py
tests/test_pdf.py
tests/test_schema.py
tests/test_scripts.py
tests/test_validation.py
tests/test_workspace.py
validation/makedata.py
validation/manualonoff_roofit/onoff.py
validation/multichan_coupledhistosys_histfactory/makedata.py
validation/multichan_coupledoverall_histfactory/makedata.py
validation/multichannel_histfactory/makedata.py
validation/run_toys.py
validation/standard_hypo_test_demo.py
validation/xmlimport_input2/makedata.py
@matthewfeickert matthewfeickert added the chore Other changes that don't modify src or test files label Sep 14, 2022
@henryiii
Copy link
Member

FYI, this is PEP 597 and:

Using open(filename) to read text files encoded in UTF-8 is a common mistake. It may not work on Windows because UTF-8 is not the default encoding.

You can use -X warn_default_encoding or PYTHONWARNDEFAULTENCODING=1 to find this type of mistake.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore Other changes that don't modify src or test files
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants