ANSI File encoding

[nrubino]

We have files that are losing characters. It seems to be because we have ANSI encoding (8bits) which this "Replace Tokens" does not support. ASCII is only 7bits.

See example jpg of the characters that are being replaced.

May you also support ANSI?

[qetza]

Hi,
Did you try to use utf8 to see if it could work as a workaround?

On the ANSI encoding, this is not an official encoding and can represents different encoding. Which one would you like to have:

• ISO 8859-1
• Windows 1252
• US-ASCII

Here is the full list of encoding i could support based on the library i'm using:
https://github.com/ashtuchkin/iconv-lite/wiki/Supported-Encodings

[rollercoasterdigital]

I may have come across another instance of this. I'm replacing tokens in a file and the process corrupts the character "ɵ" which is used in another part of the file. I'm struglling to determine the encoding on the file, could you recommend a way to identify the encoding type?

[qetza]

Hi @rollercoasterdigital,
If you use the auto value the script will try to determine the encoding used by looking for a BOM at the start of the file. If there is no BOM then ANSI is used.
You can try opening your file with an editor like vs code or notepad++ which will also try to find the encoding for you.
Also try to force utf-8 in the task and see if it works for you.

[rollercoasterdigital]

Thanks for that. Both VS Code and Notepad++ are identifying this file as UTF-8. Setting the encoding to UTF-8 appears to have resolved the issue.