UnicodeEncodeError #39

nmandal · 2017-04-13T00:07:36Z

$ python export_saved.py
Traceback (most recent call last):
File "export_saved.py", line 305, in
main()
File "export_saved.py", line 299, in main
save_saved(reddit)
File "export_saved.py", line 267, in save_saved
process(reddit, seq, "export-saved", "Reddit - Saved")
File "export_saved.py", line 247, in process
csv_rows = get_csv_rows(reddit, seq)
File "export_saved.py", line 194, in get_csv_rows
logging.debug('title: {}'.format(title))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2014' in position 90: ordinal not in range(128)

netfortius · 2017-04-16T03:02:16Z

Similar issue - but with the " character (u201c) found somewhere:

UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position 17: ordinal not in range(128)

Edit 5/2/2017: pulled and tested the latest version - same issue, apparently somewhere else:

UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 83: ordinal not in range(128)

admish · 2017-05-07T20:40:46Z

Use Python 3 and this error will clear up.

If you have multiple versions of python installed use python3 export_saved.py to run. You might also need to install praw as well with pip3 install praw.

rachmadaniHaryono · 2017-05-07T20:59:42Z

Thank you @admish for commenting here. Did you use the recent commit with #38?

This could mean this issue can be closed alongside with already closed issue #37 without waiting response @netfortius and @nmandal.

admish · 2017-05-07T21:08:26Z

@rachmadaniHaryono I just installed this today so have only ever used the latest commit. Running Python 2.7.12 and Python 3.4.2 on macOS 10.12.4. Python3 with praw 4.5 ran flawlessly while Python 2.7 gave the following error:

Version 4.4.0 of praw is outdated. Version 4.5.0 was released 7 days ago.
Traceback (most recent call last):
  File "export_saved.py", line 321, in <module>
    main()
  File "export_saved.py", line 315, in main
    save_saved(reddit)
  File "export_saved.py", line 283, in save_saved
    process(reddit, seq, "export-saved", "Reddit - Saved")
  File "export_saved.py", line 265, in process
    write_csv(csv_rows, file_name + ".csv")
  File "export_saved.py", line 242, in write_csv
    csvwriter.writerow(row)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in position 10: ordinal not in range(128)

rachmadaniHaryono · 2017-05-07T21:25:01Z

Did you come the repo or just download the export_saved file? If you clone the repo can you check which commit you are now?

I will add program version flag so people can easily recognize the version later.

Also python2.7 is actually not officially supported (and we haven't wrote any python requirement in readme, sorry for that).

But I will patch this python2.7 if the latest pr can't fix it.

add program version
add python requirement on Readme

admish · 2017-05-07T21:41:59Z

Yes, I cloned the repo.

By default, OS X comes with Python 2.7. Following the instructions in the README on Macs without knowing this, the script won't work. To anyone else reading this you can change the default version by editing your ~/.bash_profile to include the line alias python='python3' then relaunch your terminal.

If this would help, here is the latest entry in the log.

:~/Workspace/export-saved-reddit$ git log
commit 9841839e273bc8675bc7bb96112650957131fb6c
Merge: 8277d77 5d2ca4b
Author: Christopher Su <[email protected]>
Date:   Wed Apr 26 19:51:43 2017 -0700

    Merge pull request #38 from rachmadaniHaryono/feature/fix-csv-type-error

    fix: usr: TypeError when writing csv field

rachmadaniHaryono · 2017-05-07T23:54:21Z

@admish can you check once again on python2.7 version?

e: checkout the branch on the pr and run it once again.

e2: actually on requirement it is already recommended to use python3

Requirements

Python 3.x

i recommend @netfortius and @nmandal can use python3

the next version i may add an error if python2 is used.

admish · 2017-05-08T00:33:18Z

Same error.

rachmadaniHaryono · 2017-05-08T01:08:39Z

on the same line? 254 or 256?

admish · 2017-05-08T01:36:47Z

Version 4.4.0 of praw is outdated. Version 4.5.0 was released 7 days ago.
Traceback (most recent call last):
  File "export_saved.py", line 329, in <module>
    main()
  File "export_saved.py", line 323, in main
    save_saved(reddit)
  File "export_saved.py", line 287, in save_saved
    process(reddit, seq, "export-saved", "Reddit - Saved")
  File "export_saved.py", line 269, in process
    write_csv(csv_rows, file_name + ".csv")
  File "export_saved.py", line 246, in write_csv
    csvwriter.writerow(row)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201d' in position 68: ordinal not in range(128)

rachmadaniHaryono · 2017-05-10T19:34:53Z

@admish can you try it once again?

the recent code c8cca7e

if there is any error there should be only on line number 254, 256, 263, or 265

admish · 2017-05-10T19:42:54Z

Still the same error.

Version 4.4.0 of praw is outdated. Version 4.5.1 was released 2 days ago.
Traceback (most recent call last):
  File "export_saved.py", line 329, in <module>
    main()
  File "export_saved.py", line 323, in main
    save_saved(reddit)
  File "export_saved.py", line 287, in save_saved
    process(reddit, seq, "export-saved", "Reddit - Saved")
  File "export_saved.py", line 269, in process
    write_csv(csv_rows, file_name + ".csv")
  File "export_saved.py", line 246, in write_csv
    csvwriter.writerow(row)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201d' in position 68: ordinal not in range(128)

The git pull was successful and this was on the latest merge.

## $ git log
commit 64a37f64037c3eec623ba16de962c7a5da35f1d5
Merge: 9841839 9d4d027
Author: rachmadani haryono <[email protected]>
Date:   Mon May 8 06:51:03 2017 +0800

    Merge pull request #42 from rachmadaniHaryono/feature/add-program-version

    chg: dev: add program version.

rachmadaniHaryono · 2017-05-10T19:51:04Z

yeah, but that isn't the pull request branch.

add my fork and checkout the branch.

git remote add rachmadaniHaryono https://github.com/rachmadaniHaryono/export-saved-reddit
git checkout rachmadaniHaryono/feature/fix-py27-writerow

this should be the result from git show

commit c8cca7e5f2734d9afd41a05b75f470aafee0f1ce
Author: rachmadaniHaryono <[email protected]>
Date:   Thu May 11 03:32:14 2017 +0800

admish · 2017-05-10T19:59:17Z

Traceback (most recent call last):
  File "export_saved.py", line 339, in <module>
    main()
  File "export_saved.py", line 333, in main
    save_saved(reddit)
  File "export_saved.py", line 297, in save_saved
    process(reddit, seq, "export-saved", "Reddit - Saved")
  File "export_saved.py", line 279, in process
    write_csv(csv_rows, file_name + ".csv")
  File "export_saved.py", line 256, in write_csv
    csvwriter.writerow(row.encode('utf8', 'ignore'))
AttributeError: 'list' object has no attribute 'encode'

 $ git log
commit c8cca7e5f2734d9afd41a05b75f470aafee0f1ce
Author: rachmadaniHaryono <[email protected]>
Date:   Thu May 11 03:32:14 2017 +0800

    chg: dev: fix variable typo

rachmadaniHaryono · 2017-05-10T20:11:31Z

@admish dbe00b0

admish · 2017-05-10T20:18:02Z

Traceback (most recent call last):
  File "export_saved.py", line 339, in <module>
    main()
  File "export_saved.py", line 333, in main
    save_saved(reddit)
  File "export_saved.py", line 297, in save_saved
    process(reddit, seq, "export-saved", "Reddit - Saved")
  File "export_saved.py", line 279, in process
    write_csv(csv_rows, file_name + ".csv")
  File "export_saved.py", line 256, in write_csv
    csvwriter.writerow([x.encode('utf8', 'ignore') for x in row])
AttributeError: 'int' object has no attribute 'encode'

admish · 2017-05-10T20:18:36Z

Would you like me to setup an OS X VM for you?

rachmadaniHaryono · 2017-05-10T20:21:38Z

yes, if possible

e:@admish fb3a4b5

admish · 2017-05-10T20:27:26Z

Setting up a new one now, but also about to leave work so give me a couple hours. In the meantime email me a public key that I can add into the authorized_keys file, and when I have the login info setup I'll share it so you can login.

netfortius · 2017-07-17T14:37:51Z

Issue is back, at least on OSX. Unicode error, regardless of version of python or praw used. Tried two different machines, one with python2 & praw 4.5.1, and one with python3 and praw 5.0.1. Then upgraded praw on the python2 machine, to cover that combo - consistent error across all: "can't encode character '\xe0' in position 66: ordinal not in range (128)"

Fix for unicode errors and conversion process ref: csu#39

lukepayne · 2017-08-28T09:08:21Z

It looks like this issue wasn't resolved, so I have opened a PR.
@csu could you please take a look / merge if appropriate? Thanks!

rachmadaniHaryono · 2017-08-28T09:26:25Z

Iirc the 3 PRs are actually for this issue, yet nobody mention if the issue is already fixed by one of these

lukepayne · 2017-08-28T09:52:54Z

You're right, it looks like there are 2 other PR requests that are in various states of completeness. It would be good to have a fix merged in for this issue so that others don't have to run into it.

This was referenced May 7, 2017

chg: dev: add program version. #42

Merged

chg: dev: add fix for python2.7 writerow #43

Closed

lukepayne added a commit to lukepayne/export-saved-reddit that referenced this issue Aug 28, 2017

Fix for unicode errors and conversion process

3333d90

Fix for unicode errors and conversion process ref: csu#39

lukepayne mentioned this issue Aug 28, 2017

Fix for unicode errors and conversion process #45

Merged

bmikolaj added a commit to bmikolaj/export-saved-reddit that referenced this issue Sep 29, 2019

Replaced csv with unicodeccsv to fix ISSUE csu#39 and ISSUE csu#49

53033e4

bmikolaj mentioned this issue Sep 29, 2019

Replaced csv with unicodeccsv #55

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UnicodeEncodeError #39

UnicodeEncodeError #39

nmandal commented Apr 13, 2017

netfortius commented Apr 16, 2017 •

edited

Loading

admish commented May 7, 2017

rachmadaniHaryono commented May 7, 2017 •

edited

Loading

admish commented May 7, 2017 •

edited

Loading

rachmadaniHaryono commented May 7, 2017 •

edited

Loading

admish commented May 7, 2017

rachmadaniHaryono commented May 7, 2017 •

edited

Loading

Requirements

admish commented May 8, 2017

rachmadaniHaryono commented May 8, 2017

admish commented May 8, 2017

rachmadaniHaryono commented May 10, 2017

admish commented May 10, 2017

rachmadaniHaryono commented May 10, 2017

admish commented May 10, 2017

rachmadaniHaryono commented May 10, 2017

admish commented May 10, 2017

admish commented May 10, 2017

rachmadaniHaryono commented May 10, 2017 •

edited

Loading

admish commented May 10, 2017

netfortius commented Jul 17, 2017 •

edited

Loading

lukepayne commented Aug 28, 2017

rachmadaniHaryono commented Aug 28, 2017

lukepayne commented Aug 28, 2017

UnicodeEncodeError #39

UnicodeEncodeError #39

Comments

nmandal commented Apr 13, 2017

netfortius commented Apr 16, 2017 • edited Loading

admish commented May 7, 2017

rachmadaniHaryono commented May 7, 2017 • edited Loading

admish commented May 7, 2017 • edited Loading

rachmadaniHaryono commented May 7, 2017 • edited Loading

admish commented May 7, 2017

rachmadaniHaryono commented May 7, 2017 • edited Loading

Requirements

admish commented May 8, 2017

rachmadaniHaryono commented May 8, 2017

admish commented May 8, 2017

rachmadaniHaryono commented May 10, 2017

admish commented May 10, 2017

rachmadaniHaryono commented May 10, 2017

admish commented May 10, 2017

rachmadaniHaryono commented May 10, 2017

admish commented May 10, 2017

admish commented May 10, 2017

rachmadaniHaryono commented May 10, 2017 • edited Loading

admish commented May 10, 2017

netfortius commented Jul 17, 2017 • edited Loading

lukepayne commented Aug 28, 2017

rachmadaniHaryono commented Aug 28, 2017

lukepayne commented Aug 28, 2017

netfortius commented Apr 16, 2017 •

edited

Loading

rachmadaniHaryono commented May 7, 2017 •

edited

Loading

admish commented May 7, 2017 •

edited

Loading

rachmadaniHaryono commented May 7, 2017 •

edited

Loading

rachmadaniHaryono commented May 7, 2017 •

edited

Loading

rachmadaniHaryono commented May 10, 2017 •

edited

Loading

netfortius commented Jul 17, 2017 •

edited

Loading