Support internationalization

[bkeryan]

The DAQmx C API (nicaiu.dll) supports internationalization but not localization:

• User-provided strings have ANSI/MBCS encoding.
• Localized strings (error messages) are in English. Extended error info may also contain user-provided strings.

nidaqmx-python currently assumes that user-provided strings are ASCII and error messages are UTF-8. User-provided strings that use non-ASCII characters produce encoding errors.

Note that there is also a UTF-8 version of the DAQmx C API (nicai_utf8.dll) but nidaqmx-python does not currently use it.

Steps to reproduce:

PS D:\dev\nidaqmx-python> poetry run python
Python 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import nidaqmx.system
>>> d = nidaqmx.system.Device("Gerät")
>>> d
Device(name=Gerät)
>>> d.name
'Gerät'
>>> d.serial_num
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\dev\nidaqmx-python\generated\nidaqmx\system\device.py", line 991, in serial_num
    val = self._interpreter.get_device_attribute_uint32(self._name, 0x632)
  File "D:\dev\nidaqmx-python\generated\nidaqmx\_library_interpreter.py", line 2572, in get_device_attribute_uint32
    error_code = cfunc(
ctypes.ArgumentError: argument 1: <class 'UnicodeEncodeError'>: 'ascii' codec can't encode character '\xe4' in position 3: ordinal not in range(128)

[robin-po]

I have the same error when the device name or task name contains latin characters.
I succeed to make the following code working with some modifications in _library_interpreter.py:

import nidaqmx.system

d = nidaqmx.system.Device("Gerät")
print(d.name)
print(d.serial_num)

I've modified my get_device_attribute_uint32 to convert the device_name to iso-8859-1:

    def get_device_attribute_uint32(self, device_name, attribute):
        value = ctypes.c_uint32()

        cfunc = lib_importer.cdll.DAQmxGetDeviceAttribute
        if cfunc.argtypes is None:
            with cfunc.arglock:
                if cfunc.argtypes is None:
                    cfunc.argtypes = [
                        ctypes_byte_str, ctypes.c_int32]

        error_code = cfunc(
            device_name.encode('iso-8859-1'), attribute, ctypes.byref(value))
        self.check_for_error(error_code)
        return value.value

Hope this helps

[charitylxy]

@bkeryan Just to understand better, meaning we would want user-provided string (in this case, the device_name "Gerät") to be encoded to utf-8? Given the scenario above, is this supposed to be the expected output, instead of the encoding error?

Python 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import nidaqmx.system
>>> d = nidaqmx.system.Device("Gerät")
>>> d.serial_num
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\dev\nidaqmx-python\generated\nidaqmx\system\device.py", line 1143, in serial_num
    val = self._interpreter.get_device_attribute_uint32(self._name, 0x632)
  File "C:\dev\nidaqmx-python\generated\nidaqmx\_library_interpreter.py", line 2657, in get_device_attribute_uint32
    self.check_for_error(error_code)
  File "C:\dev\nidaqmx-python\generated\nidaqmx\_library_interpreter.py", line 6354, in check_for_error
    raise DaqError(extended_error_info, error_code)
nidaqmx.errors.DaqError: Device identifier is invalid.
Device Specified: Gerät

Status Code: -200220

[bkeryan]

Just to understand better, meaning we would want user-provided string (in this case, the device_name "Gerät") to be encoded to utf-8?

@charitylxy The goal is to pass the user-provided string to the C API in an encoding that works.

• Currently, we reject all non-ASCII characters, so only ASCII strings work.
• With nicaiu.dll, I think we can make the MBCS encoding (Latin 1, Shift-JIS, etc.) that corresponds to the current system locale work. However, characters that can't be represented in that encoding (e.g. Japanese characters with a French system locale) won't work.
• With nicai_utf8.dll, we can make all languages work regardless of system locale.

Given the scenario above, is this supposed to be the expected output, instead of the encoding error?

Yes and no.

Given that the user has renamed a device to "Gerät", the expected behavior is that using the device works correctly with no errors.

However, if the user doesn't have a device named "Gerät", the error output you displayed is correct.

[bkeryan]

@charitylxy BTW, this issue does not cover returning localized error messages, since that is localization, not internationalization. The localized error content is available in LabVIEW, but I don't think nicaiu.dll or nicai_utf8.dll returns it.

nidaqmx.errors.DaqError: Der Gerätebezeichner ist ungültig.
Angegebenes Gerät: Gerät

Fehlercode: -200220

[charitylxy]

Closing this issue as it was fixed in the following PRs:

• Add python-decouple Dependency #524
• Support Non-ASCII Characters for User-Provided String #518
• Add Functional Tests for Internationalization #525
• Add Tox Env for Internationalization Testing #532