UnicodeEncodeError: 'charmap' codec can't encode character '\u273f' in position 145: character maps to <undefined>

[xdhmoore]

Describe the bug
shell, trash,copy, etc. actions print stack traces (but still succeed?) for unicode characters in the filenames like ✿, \u273f.

There appears to be a logging bug that causes stack traces to be printed for non-ascii characters. The actions themselves still appear to succeed.

Environment (please complete the following information):

• OS: Windows 10
• Output of organize --version:
  organize 2.2.0

Your config file

rules:
  - name: "test"
    locations: .
    filter_mode: any
    filters:
      - extension: "txt"
    actions:
      - shell: echo {path}
      - trash

And an empty file called file-with-✿-flower.txt
Run with:

organize run .\config.yaml --working-dir .

Sanitized Output:

organize 2.2.0
Config: "flower_test.yaml"

⚙ test ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────/D:\flower_test\
  file-with-✿-flower.txt
    - (shell) $ echo D:\flower_test\file-with-✿-flower.txt
--- Logging error ---
Traceback (most recent call last):
  File "C:\python\lib\logging\__init__.py", line 1103, in emit
    stream.write(msg + self.terminator)
  File "C:\python\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u273f' in position 145: character maps to <undefined>
Call stack:
  File "C:\python\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\python\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "D:\.venv\Scripts\organize.exe\__main__.py", line 7, in <module>
    sys.exit(cli())
  File "D:\.venv\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "D:\.venv\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "D:\.venv\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "D:\.venv\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "D:\.venv\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "D:\.venv\lib\site-packages\organize\cli.py", line 157, in run
    run_local(
  File "D:\.venv\lib\site-packages\organize\cli.py", line 108, in run_local
    core.run(
  File "D:\.venv\lib\site-packages\organize\core.py", line 362, in run
    count = run_rules(rules=rules, tags=tags, skip_tags=skip_tags, simulate=simulate)
  File "D:\.venv\lib\site-packages\organize\core.py", line 322, in run_rules
    is_success = action_pipeline(
  File "D:\.venv\lib\site-packages\organize\core.py", line 231, in action_pipeline
    updates = action.pipeline(args, simulate=simulate)
  File "D:\.venv\lib\site-packages\organize\actions\shell.py", line 66, in pipeline
    logger.info('Executing command "%s" in shell.', full_cmd)
Message: 'Executing command "%s" in shell.'
Arguments: ('echo D:\\flower_test\\file-with-✿-flower.txt',)
    - (trash) Trash "D:\flower_test\file-with-✿-flower.txt"
--- Logging error ---
Traceback (most recent call last):
  File "C:\python\lib\logging\__init__.py", line 1103, in emit
    stream.write(msg + self.terminator)
  File "C:\python\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u273f' in position 133: character maps to <undefined>
Call stack:
  File "C:\python\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\python\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "D:\.venv\Scripts\organize.exe\__main__.py", line 7, in <module>
    sys.exit(cli())
  File "D:\.venv\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "D:\.venv\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "D:\.venv\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "D:\.venv\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "D:\.venv\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "D:\.venv\lib\site-packages\organize\cli.py", line 157, in run
    run_local(
  File "D:\.venv\lib\site-packages\organize\cli.py", line 108, in run_local
    core.run(
  File "D:\.venv\lib\site-packages\organize\core.py", line 362, in run
    count = run_rules(rules=rules, tags=tags, skip_tags=skip_tags, simulate=simulate)
  File "D:\.venv\lib\site-packages\organize\core.py", line 322, in run_rules
    is_success = action_pipeline(
  File "D:\.venv\lib\site-packages\organize\core.py", line 231, in action_pipeline
    updates = action.pipeline(args, simulate=simulate)
  File "D:\.venv\lib\site-packages\organize\actions\trash.py", line 29, in pipeline
    self.trash(path=fs.getsyspath(fs_path), simulate=simulate)
  File "D:\.venv\lib\site-packages\organize\actions\trash.py", line 23, in trash
    logger.info("Moving file %s into trash.", path)
Message: 'Moving file %s into trash.'
Arguments: ('D:\\flower_test\\file-with-✿-flower.txt',)

[xdhmoore]

Based on the line stream.write I think the issue may be coming from specifically the stdout stream log handler. I tried setting in pwsh: $env:PYTHONIOENCODING='utf-8' but it didn't solve the problem.

[xdhmoore]

It looks like if I add encoding: utf-8 to the log file handler configuration, it fixes the issue. Added a PR.