Python error when executing :shell or :term

[ir0nh34d]

When I execute :shell or :term I get the following error:

Fatal Python error: config_get_locale_encoding: failed to get the locale encoding: nl_langinfo(CODESET) failed
Python runtime state: preinitialized

How do I correct this?

[ir0nh34d]

I've noticed the encoding is stripped when in the MacVim terminal:

From a Mac terminal:
LANG="en_CA.UTF-8"
LC_COLLATE="en_CA.UTF-8"
LC_CTYPE="en_CA.UTF-8"
LC_MESSAGES="en_CA.UTF-8"
LC_MONETARY="en_CA.UTF-8"
LC_NUMERIC="en_CA.UTF-8"
LC_TIME="en_CA.UTF-8"
LC_ALL=

From within MacVim via the :term command:
LANG="en_CA"
LC_COLLATE="en_CA"
LC_CTYPE="en_CA"
LC_MESSAGES="en_CA"
LC_MONETARY="en_CA"
LC_NUMERIC="en_CA"
LC_TIME="en_CA"
LC_ALL=

[ir0nh34d]

This is related to starting MacVim directly (as an App). If I start MacVim via:

open /Applications/MacVim.app

In a Mac terminal it works fine - so it has to the encoding being stripped when running directly. Any idea how to correct this?

[eirnym]

@ir0nh34d Thank you for an issue

I confirm it, but I don't know where it goes from yet.

[ir0nh34d]

Simply checking:
:echo $LANG
in MacVim shows the issue (that's the delta I could find when comparing the env output in MacVim compared to the terminal). Would it be possible to add a MacVim option to set the encoding (similar in fashion to what the terminal and iTerm2 apps do)?

[eirnym]

I'll check why this is happens in the first place. It's easy to make a fix when you know where things come from

[ir0nh34d]

Fair enough - I think it's due to the fact that macOS sets the language at a system level, but there doesn't seem to be a system setting for encoding, so the $LANG variable passed to applications lacks the encoding value. That said, this is pure guessing on my part.

[ir0nh34d]

I figured out a temporary fix - in my .gvimrc (so as to not affect terminal vim) I set:
let $LANG='en_CA.UTF-8'
Obviously substitute the language per your preference.

[ychin]

Thanks for pointing this out. I need to think about how to solve this but I suspect we need to make the GUI version just inject the .UTF-8 when it's not launched using the Terminal, which is a little hacky but should work.

[ychin]

Actually looking through the code, I think we just need to call setlocale properly to give it the UTF-8 encoding. That said, @ir0nh34d do you know what tool/program is throwing that Python failure? I don't think it's technically "wrong" to just have en_US (without the encoding value specified) as a value for $LANG so I'm wondering what tool is not working when that's the case.

[eirnym]

!locale in MacVim and locale gives the same difference.
As far as I found in mans and in the Internet, locale variables by convention should have encoding at the end.
It's possible that Apple decided, that locale will contain only language, but Terminal.app has compatibility patch to add 'UTF-8'

How iTerm re-computes locale:
gnachman/iTerm2@e0176db

[ir0nh34d]

@ychin , I'm assuming python is used as an interpreter of some sort in :terminal mode (I don't specifically invoke python)? As per the python error, it's expecting the encoding to be specified as part of the locale.

[ir0nh34d]

It turns out one of the commands I have in my .zshrc invokes python (https://github.com/nvbn/thefuck) so that's what is calling python when a shell starts.

[ychin]

This should be fixed and will be released soon.

[ir0nh34d]

I've built from source and can confirm the issue has been addressed - thanks for the fix!

[eirnym]

I want to remind, that iTerm has resigned from this method

[ychin]

What do you mean @eirnym? Care to elaborate?

[eirnym]

Yes, I've posted few comments above the link to a commit how currently iTerm computes LANG variable. I searched there for the term LANG.

I don't know exact reasons why they resign from just appending a .UTF-8, but I believe that the reasons were important enough.

My guess, that they tried to run iTerm2 under root or another user with default or no settings in $HOME, as well as run iTerm2 under recovery mode. But this is only my guess as this is not written in comments. The issues are hidden and probably there were the reasons for this way of setting.

Also they commented out setting of other LC_* environment variables beside LANG and LC_TYPE for a reason.

[ychin]

They are just slapping "UTF-8" in the end of the LANG environmental variable. iTerm allows you to set the encoding you want like native Terminal so you don't see a hard-coded "UTF-8". I don't think there's a lot of value in letting user set their own encoding in the locale var in MacVim. They can always manually call set encoding and set $LANG themselves. The other stuff like calculating country names etc, we are just using a different function to calculate it.

[eirnym]

Thank you, I see now. I'm fine with it, as I never met any other encodings in macOS (I'm a user since 10.6)