Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No mapping for the Unicode character exists in the target multi-byte code page #2057

Closed
ItzLevvie opened this issue Mar 31, 2022 · 7 comments · Fixed by #2555
Closed

No mapping for the Unicode character exists in the target multi-byte code page #2057

ItzLevvie opened this issue Mar 31, 2022 · 7 comments · Fixed by #2555
Labels
Issue-Bug It either shouldn't be doing this or needs an investigation.
Milestone

Comments

@ItzLevvie
Copy link

ItzLevvie commented Mar 31, 2022

Brief description of your issue

image

Logs from the DiagOutputDir folder with --verbose-logs:

  1. winget validate = https://pastebin.com/raw/a3WAmzPN
  2. winget install = https://pastebin.com/raw/A3xFjEwQ
  3. winget source update = https://pastebin.com/raw/GBp1un8Q

My settings.json file = https://pastebin.com/raw/q68y6PLX

Steps to reproduce

Install TTKN.CNKIExpress version 0.1.20 from microsoft/winget-pkgs#56247.

PR files: https://github.com/vedantmgoyal2009/winget-pkgs/tree/b00cfb3c7534916f0c75b962235215caf185afc4/manifests/t/TTKN/CNKIExpress/0.1.20

Expected behavior

For the TTKN.CNKIExpress WinGet package to install successfully on my PC or VM.

Actual behavior

I'm seeing the error below on Windows Terminal, Command Prompt, and Windows PowerShell:

An unexpected error occurred while executing the command:
No mapping for the Unicode character exists in the target multi-byte code page.

which prevents me from installing the TTKN.CNKIExpress WinGet package.

Environment

Windows Package Manager (Preview) v1.3.1661-preview
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.25140.1000
System Architecture: X64
Package: Microsoft.DesktopAppInstaller v1.18.1661.0

Logs: %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\DiagOutputDir

Links
---------------------------------------------------------------------------
Privacy Statement   https://aka.ms/winget-privacy
License Agreement   https://aka.ms/winget-license
Third Party Notices https://aka.ms/winget-3rdPartyNotice
Homepage            https://aka.ms/winget
Windows Store Terms https://www.microsoft.com/en-us/storedocs/terms-of-sale

Any other software?

  1. Windows Terminal Preview version 1.14.1452.0
  2. Windows Package Manager Source (winget) version 2022.620.1418.424
  3. Git version 2.37.0.rc1.windows.1
@ghost ghost added the Needs-Triage Issue need to be triaged label Mar 31, 2022
@denelon denelon added Issue-Bug It either shouldn't be doing this or needs an investigation. and removed Needs-Triage Issue need to be triaged labels Mar 31, 2022
@ItzLevvie
Copy link
Author

ItzLevvie commented Apr 1, 2022

I was also able to repro this on microsoft/winget-pkgs#56499 as well.

Looks like it's likely the Chinese characters in the InstallerUrl that's causing this issue because if I change the InstallerUrl to https://www.google.com it doesn't spew out the No mapping for the Unicode character exists in the target multi-byte code page. error.

@ItzLevvie
Copy link
Author

ItzLevvie commented Apr 29, 2022

Azure Pipelines in the winget-pkgs repository is now hitting this error: microsoft/winget-pkgs#59030 (comment)

@florelis
Copy link
Member

It looks like this is triggered by having Unicode characters in the URL, and the culprit is this line:

if (filename.has_stem() && ((filename.string().size() + installerExtension.size()) < MAX_PATH))

Specifically, this part: filename.string()

Looks like changing it to filename.u8string() fixes it. We could send a quick PR to fix that, but maybe we'd want to spend some more time to think about side effects or adding some tests...

@Trenly
Copy link
Contributor

Trenly commented Aug 31, 2022

@denelon - Can we maybe think about pulling this into 1.4? Based on @lechacon's comment above, it seems like it wouldn't be too difficult, and this bug is causing several packages to fail both validation and install for users

@denelon
Copy link
Contributor

denelon commented Aug 31, 2022

Yes, I've already asked engineering to look into this one since it appears to be a relatively small fix. Thanks for linking the PRs! This helps tremendously.

@JohnMcPMS
Copy link
Member

Looks like changing it to filename.u8string() fixes it. We could send a quick PR to fix that, but maybe we'd want to spend some more time to think about side effects or adding some tests...

It should be filename.wstring(), because we are trying to measure the number of characters in the final name, and on Windows that means in UTF-16 (WCS2) characters. I failed when I wrote it, although admittedly I believed before checking it just now that string might be returning the internal representation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue-Bug It either shouldn't be doing this or needs an investigation.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants