Remove sys.setdefaultencoding

[rtibbles]

Summary

(filled in by @benjaoming)

As I comment on ba2a383

Sorry, but we can't allow code to be injected in strange places like this. Everything that has to do with Python 2 compatibility has to be put in the right place and properly explained.

Please see various dangers and reasons for not using it...

https://stackoverflow.com/a/3828742/405682

[benjaoming]

@rtibbles just curious, how did you recall this issue?

Also, @ralphiee22 can you clarify why this was needed since your commit comment is "Fly by fixes for kolibri and tasks." ;)

[rtibbles]

@benjaoming it was on the Kolibri Tasks trello board, on the Tech Debt list. I migrated all the cards from there to github issues.

[ralphiee22]

@benjaoming unicode decode/encode hell? Can't recall if because strings were not coming in as unicode or python was not reading them as unicode.

[benjaoming]

Since this is so clearly discouraged, I'll try removing it and see what happens. Then we can add encoding and decoding fixes where they are needed - to have an explicit handling of wrong encodings and possibly track them down to their source.

IMO the only way to deal with encoding issues is at the source, the possible issues that are mentioned in the StackOverflow answer do not sound nice to try and track down... especially in some distant future where maintaining Python 2.7 support is a terrible burden to someone.

I suggest:

1. We isolate all Python 2.7 compatibility handling to a central location (I've suggested this before) - in this example, there's some compat code sitting in the middle of module load time.

2. We try dealing with terminal encoding by having utility functions for all terminal invocation - this will also make it easier to mock stuff for testing purposes

3. All encoding issues from data (CC server etc) should be dealt with at the source. Any problems that occur in this regard are IMO fine, we just need to fix them, when we see them. Since Python 3 has UTF-8 encoding and decoding built into a lot of functions, we won't see these issues in anything but Python 2.7 -- however, if we do see them, we would know that some data we are using was wrongly encoded and has a danger of getting wrongly interpreted at some point.

[rtibbles]

Note that the location of this code has now migrated to: https://github.com/learningequality/kolibri/blob/release-v0.14.x/kolibri/utils/env.py#L63

I think the simplest solution to this issue, may be just to handle it when we eventually drop Python 2 support.

[rtibbles]

Fixed in #11654