Unicode compatibility on Windows

[mrexodia]

Make sure to check if these are still the case. They are hardlinked to a specific commit.

Currently quite a lot of Windows APIs called use the ANSI version of the API is used which means r2 won't work on unicode paths etc. I will add an exhaustive list of all the places here later.

ADVAPI32.dll

CreateServiceA

• https://github.com/radare/radare2/blob/8af36314/libr/io/p/io_r2k_windows.c#L10

LookupPrivilegeValueA

• https://github.com/radare/radare2/blob/1fb422b5/libr/io/p/io_debug.c#L82
• https://github.com/radare/radare2/blob/33e84a9c/libr/debug/p/native/w32.c#L231

OpenSCManagerA

• https://github.com/radare/radare2/blob/8af36314/libr/io/p/io_r2k_windows.c#L8
• https://github.com/radare/radare2/blob/8af36314/libr/io/p/io_r2k_windows.c#L9

OpenServiceA

• https://github.com/radare/radare2/blob/8af36314/libr/io/p/io_r2k_windows.c#L25
• https://github.com/radare/radare2/blob/8af36314/libr/io/p/io_r2k_windows.c#L43

StartServiceA

• https://github.com/radare/radare2/blob/8af36314/libr/io/p/io_r2k_windows.c#L46

KERNEL32.dll

CreateDirectoryA

• https://github.com/radare/radare2/blob/18b8d61a/shlr/sdb/src/disk.c#L13
• https://github.com/radare/radare2/blob/9f05ffce/libr/util/sys.c#L592

CreateFileA

• https://github.com/radare/radare2/blob/8af36314/libr/io/p/io_r2k_windows.c#L71
• https://github.com/radare/radare2/blob/2603cbf3/libr/util/w32-sys.c#L77
• https://github.com/radare/radare2/blob/43532031/libr/io/p/io_w32.c#L54
• https://github.com/radare/radare2/blob/1fb422b5/libr/util/file.c#L601
• https://github.com/radare/radare2/blob/266eee75/shlr/wind/iob_pipe.c#L14

CreateFileMappingA

• https://github.com/radare/radare2/blob/1fb422b5/libr/util/file.c#L651
• https://github.com/radare/radare2/blob/a2cb6b66/libr/util/judy64na.c#L1608
• https://github.com/radare/radare2/blob/33e84a9c/libr/debug/p/native/w32.c#L458

CreateNamedPipeA

• https://github.com/radare/radare2/blob/87724384/libr/lang/p/pipe.c#L181
• https://github.com/radare/radare2/blob/e28ada79/libr/socket/r2pipe.c#L69

CreateProcessA

• https://github.com/radare/radare2/blob/c0c0cba3/shlr/bochs/src/libbochs.c#L211
• https://github.com/radare/radare2/blob/2603cbf3/libr/util/w32-sys.c#L118
• https://github.com/radare/radare2/blob/1fb422b5/libr/io/p/io_debug.c#L146
• https://github.com/radare/radare2/blob/87724384/libr/lang/p/pipe.c#L24
• https://github.com/radare/radare2/blob/e28ada79/libr/socket/r2pipe.c#L58

DeleteFileA

• https://github.com/radare/radare2/blob/87724384/libr/lang/p/pipe.c#L193
• https://github.com/radare/radare2/blob/1fb422b5/libr/util/file.c#L570

FillConsoleOutputCharacterA

• https://github.com/radare/radare2/blob/542e06f6/libr/cons/output.c#L36

FormatMessageA

• https://github.com/radare/radare2/blob/33e84a9c/libr/debug/p/native/w32.c#L256
• https://github.com/radare/radare2/blob/266eee75/shlr/zip/zlib/gzlib.c#L41
• https://github.com/radare/radare2/blob/9f05ffce/libr/util/sys.c#L655

GetEnvironmentVariableA

• https://github.com/radare/radare2/blob/9f05ffce/libr/util/sys.c#L321

GetLogicalDriveStringsA

• https://github.com/radare/radare2/blob/33e84a9c/libr/debug/p/native/w32.c#L490

GetModuleFileNameA

• https://github.com/radare/radare2/blob/266eee75/shlr/tcc/libtcc.c#L60
• https://github.com/radare/radare2/blob/2603cbf3/libr/util/w32-sys.c#L32

GetModuleHandleA

• https://github.com/radare/radare2/blob/c0c0cba3/shlr/bochs/src/libbochs.c#L59
• https://github.com/radare/radare2/blob/33e84a9c/libr/debug/p/native/w32.c#L277
• https://github.com/radare/radare2/blob/33e84a9c/libr/debug/p/native/w32.c#L280
• https://github.com/radare/radare2/blob/5e00b611/shlr/wind/wind.c#L1021

GetShortPathNameA

• https://github.com/radare/radare2/blob/2603cbf3/libr/util/w32-sys.c#L36

GetTempFileNameA

• https://github.com/radare/radare2/blob/1fb422b5/libr/util/file.c#L823

LoadLibraryA

• https://github.com/radare/radare2/blob/1fb422b5/libr/util/lib.c#L20
• https://github.com/radare/radare2/blob/33e84a9c/libr/debug/p/native/w32.c#L305
• https://github.com/radare/radare2/blob/33e84a9c/libr/debug/p/native/w32.c#L317
• https://github.com/radare/radare2/blob/9f05ffce/libr/util/sys.c#L781
• https://github.com/radare/radare2/blob/9f05ffce/libr/util/sys.c#L792
• https://github.com/radare/radare2/blob/5e00b611/shlr/wind/wind.c#L1019

lstrlenA

• https://github.com/radare/radare2/blob/9f05ffce/libr/util/sys.c#L665
• https://github.com/radare/radare2/blob/9f05ffce/libr/util/sys.c#L666

MoveFileExA

• https://github.com/radare/radare2/blob/266eee75/shlr/zip/include/zipint.h#L58
• https://github.com/radare/radare2/blob/266eee75/shlr/zip/zip/zipint.h#L59
• https://github.com/radare/radare2/blob/266eee75/shlr/zip/zip/zipint.h#L59

QueryDosDeviceA

• https://github.com/radare/radare2/blob/33e84a9c/libr/debug/p/native/w32.c#L504

QueryFullProcessImageNameA

• https://github.com/radare/radare2/blob/9f05ffce/libr/util/sys.c#L788
• https://github.com/radare/radare2/blob/33e84a9c/libr/debug/p/native/w32.c#L842

ReadConsoleInputA

• https://github.com/radare/radare2/blob/02499889/libr/cons/input.c#L288
• https://github.com/radare/radare2/blob/8021e7f4/libr/cons/dietline.c#L125

RemoveDirectoryA

• https://github.com/radare/radare2/blob/1fb422b5/libr/util/file.c#L564

SetEnvironmentVariableA

• https://github.com/radare/radare2/blob/2407b939/shlr/spp/r_api.c#L101
• https://github.com/radare/radare2/blob/87724384/libr/lang/p/pipe.c#L180
• https://github.com/radare/radare2/blob/9f05ffce/libr/util/sys.c#L240

USER32.dll

GetClipboardData

• https://github.com/radare/radare2/blob/8021e7f4/libr/cons/dietline.c#L741

[XVilka]

@xarkes just in case you haven't seen this. :)

[josediazfer]

Apply this changes (convert from ANSI to UNICODE) have an impact in radare core functions (you must convert types and functions aka char *to wchar_t, fgets to fgetws...etc):

For example:

R_API bool r_sys_mkdir(const char *dir) {
	if (r_sandbox_enable (0)) {
		return false;
	}
#if __WINDOWS__ && !defined(__CYGWIN__)
	return CreateDirectory (dir, NULL) != 0;
#else
	return mkdir (dir, 0755) != -1;
#endif
}

I think that we have to decide which radare functions pass to unicode before perform this task.
When we have that functions to change then progressively make changes:

For example:

• First step:

R_API bool r_sys_mkdir(const char *dir) {
	if (r_sandbox_enable (0)) {
		return false;
	}
#if __WINDOWS__ && !defined(__CYGWIN__)
        wchar_t *wdir = char2wchar(dir);
        return CreateDirectory (wdir, NULL) != 0;
#else
	return mkdir (dir, 0755) != -1;
#endif
}

• Second step:

R_API bool r_sys_mkdir(const wchar_t *dir) {
   if (r_sandbox_enable (0)) {
   	return false;
   }
#if __WINDOWS__ && !defined(__CYGWIN__)
       return CreateDirectory (dir, NULL) != 0;
#else
	return mkdir (dir, 0755) != -1;
#endif
}

[mrexodia]

No, you should convert to utf8 and only when close to the winapi convert to wchar, see utf8everywhere.org

…

On Wed, 18 Oct 2017 at 18:35, josediazfer ***@***.***> wrote: Apply this changes (convert from ANSI to UNICODE) have an impact in radare core functions (you must convert types and functions aka char *to wchar_t, fgets to fgetws...etc): For example: R_API bool r_sys_mkdir(const char *dir) { if (r_sandbox_enable (0)) { return false; } #if __WINDOWS__ && !defined(__CYGWIN__) return CreateDirectory (dir, NULL) != 0; #else return mkdir (dir, 0755) != -1; #endif } I think that we have to decide which radare functions pass to unicode before perform this task. When we have that functions to change then progressively make changes: For example: - First step: R_API bool r_sys_mkdir(const char *dir) { if (r_sandbox_enable (0)) { return false; } #if __WINDOWS__ && !defined(__CYGWIN__) wchar_t *wdir = char2wchar(dir); return CreateDirectory (wdir, NULL) != 0; #else return mkdir (dir, 0755) != -1; #endif } - Second step: R_API bool r_sys_mkdir(const wchar_t *dir) { if (r_sandbox_enable (0)) { return false; } #if __WINDOWS__ && !defined(__CYGWIN__) return CreateDirectory (dir, NULL) != 0; #else return mkdir (dir, 0755) != -1; #endif } — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7235 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACWCmYYbGsxT5RTu03OIhnaHdWiPj0aWks5stijtgaJpZM4M4oQe> .

[josediazfer]

it's true, you're right. Its not need change "dir" parameter to wchar type and it's not good idea. I'm not sure if we need use SetConsoleOutputCP(CP_UTF8) SetConsoleCP(CP_UTF8) and MultiByteToWideChar() (utf-8 to wchar), I will research it tomorrow.
Thanks!

[josediazfer]

I've already made the changes. (https://github.com/josediazfer/radare2/tree/win32_char_to_unicode)
You can check and test!

Comments and suggestions are welcome

[Maijin]

Please do a Pull request not this.

[XVilka]

@josediazfer @mrexodia so, can be this closed?

[josediazfer]

Yes can be closed it

[mrexodia]

Ehm? I don't think that PR was merged

[Maijin]

527ff71

[mrexodia]

👍

…

On Fri, 3 Nov 2017 at 18:55 Maijin ***@***.***> wrote: 527ff71 <527ff71> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7235 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACWCmeJ_ZhVQXJ3wId4t0iLi03zqV4LAks5sy1OZgaJpZM4M4oQe> .