Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

setup.py gives UnicodeDecodeError #5

Open
bastings opened this issue Mar 11, 2017 · 1 comment
Open

setup.py gives UnicodeDecodeError #5

bastings opened this issue Mar 11, 2017 · 1 comment

Comments

@bastings
Copy link

setup.py gives a UnicodeDecodeError in Python 3.4.1 because of line 10:

def read(fname):
   return open(os.path.join(os.path.dirname(__file__), fname)).read()`

It is possible to fix for py2/py3 this by using

from io import open

def read(fname):
    return open(os.path.join(os.path.dirname(__file__), fname), encoding='utf-8').read()`
@gwenniger
Copy link

I found two errors with newer versions of python3.

First, there is a problem in pyter.py
The prefix "ur" is no longer supported in newer versions of python, see also:
https://stackoverflow.com/questions/26063899/python-version-3-4-does-not-support-a-ur-prefix
Seemingly, by replacing "ur" with "r" in the second argument of some of these
"re.sub " calls, we can fix this for newer versions of python.

"
### https://stackoverflow.com/questions/26063899/python-version-3-4-does-not-support-a-ur-prefix
### The following lines are problematic, because "ur" is not supported in new versions of python
# s = re.sub(r'\s+', ur' ', s) # スペースの個数正規化 ### Gideon: This line is problematic, gives the error invalid syntax
# s = re.sub(r'(\d) ([.,]) (\d)', ur'\1\2\3', s) # 0 . 1 -> 0.1 ### Gideon: This line is problematic, gives the error invalid syntax
# s = re.sub(r'(Dr|Jr|Prof|Rev|Gen|Mr|Mt|Mrs|Ms) .', ur'\1.', s) # Mr . -> Mr. ### Gideon: This line is problematic, gives the error invalid syntax

### We fix these by replacing "ur" in the second argument with "r"  
s = re.sub(r'\s+', r' ', s)  # スペースの個数正規化    ### Gideon: Fixed by replacing "ur" by "r" as recommended on stacoverflow
s = re.sub(r'(\d) ([.,]) (\d)', r'\1\2\3', s)  # 0 . 1 -> 0.1    ### Gideon: Fixed by replacing "ur" by "r" as recommended on stacoverflow
s = re.sub(r'(Dr|Jr|Prof|Rev|Gen|Mr|Mt|Mrs|Ms) .', r'\1.', s)  # Mr . -> Mr.

"

In similar spirit, there is also a problem with the function "izip", which does not work also in newer
versions of python. It should be replaced with simply "zip" for newer versions.
Ideally, the code should be adapted to check for the python version, and have two cases depending
on the python version.

https://stackoverflow.com/questions/32261698/my-idle-does-not-recognize-itertools-izip-as-a-function

Replacing "iter.zip" with simply "zip" in
pyter/init.py

fixes the problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants