“lexical error (UTF-8 decoding error)” in WIndows 7

[bos]

(Imported from Trac #922, reported by hamukazu on 2012-03-02)

Symptom

When I type

cabal install cabal

it fails with the following error:

Distribution\Compat\Exception.hs:0:4: lexical error (UTF-8 decoding error)

Environment

It happens with 64 bit version of Windows 7.

It came to occur after I failed to install HXT. (I am not sure it is related.)

Workaround

By setting

set LANG=C

in command line, cabal comes to work.

I regard this is just a workaround and this issue should be reported as a bug.

Related links

http://stackoverflow.com/questions/9371438/cabal-install-complains-built-in04-lexical-error-utf-8-decoding-error/9533205#9533205
http://trac.haskell.org/network/ticket/43

[bos]

(Imported comment by @dcoutts on 2012-03-02)

Looks like cpp is outputting text that is not ASCII or UTF8. This then blows up because ghc expects .hs files to be UTF8.

Cabal can perhaps work around this cpp quirk by calling ghc/cpp with LANG=C.

The new program launching utils in Cabal should make it easier to set env vars like this.

It'd be useful if someone could confirm this hypothesis.

[bos]

(Imported comment by @kosmikus on 2012-04-12)

Most likely will require a change in the Cabal lib.

[gregorycollins]

@hamukazu is this still happening?

[hamukazu]

Recently I haven't used cabal on Windows, so I checked it just now.

I tested with latest version of Haskell Platform for Windows (2012.4.0.0) and MinGW (mingw-get-inst-20120426). The installation succeeded and UTF8 related error never reproduced even when I tried to install other packages after that.

I regard this is fixed.
Thanks.

[batterseapower]

This actually is a bug and is still happening. Duncan's diagnosis was correct. When you have configured Windows to use something other than English as the "locale for non Unicode programs" then mingw's CPP will output error messages and informational text in that locale. So for example, a C-preprocessed Haskell source file will have this at the top:

# 1 "libraries/filepath\\System\\FilePath.hs"
# 1 "<built-in>"
# 1 "<命令行>"
# 1 "libraries/filepath\\System\\FilePath.hs"

Where the text 命令行 (meaning 'command line interface', BTW) is actually encoded using the Chinese Windows code page rather than UTF-8.

Running the C-preprocessor with LANG=C fixes this because it causes gcc's tools to output messages in English.

[jneira]

Dont have access to a win 7 system to try to reproduce but it does not in windows 10
It seems to be a bug in ghc and its mingw c toolchain so newer ghc versions might not suffer from it.
Definitely not a cabal bug, so closing.