Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not compatible with pytorch-cuda on windows platform #49

Closed
TheAutumnOfRice opened this issue Jul 9, 2022 · 6 comments
Closed

Not compatible with pytorch-cuda on windows platform #49

TheAutumnOfRice opened this issue Jul 9, 2022 · 6 comments
Labels

Comments

@TheAutumnOfRice
Copy link

Describe the bug
As far as I've tested, it is a bug that only existed on windows platform.
If a pytorch (with cuda) module is imported, all the print functions that between counter.update() will malfunction. In detail, if there's only one line print between update(), the last printed text will be replaced by the new one; if there are multiple lines, the progressbar will be replaced and the progressbar will be rewritten at a new line when update is called.

What confused me is that the bug only occurs in cuda-version-pytroch and only on windows platform. In other words, a cpu version of pytorch will not encounter it. I guess the cuda initialization progress may conflict with enlighten.

To Reproduce

No torch, no bug.

code:

import enlighten
import time

manager = enlighten.get_manager()
A = manager.counter(total=10)
for i in range(10):
    print("Processing:",i)
    time.sleep(0.1)
    A.update()
A.close()

output:
image

Torch (cuda version), BUG!

import enlighten
import time
import torch
print(torch.__version__)
manager = enlighten.get_manager()
A = manager.counter(total=10)
for i in range(10):
    print("Processing:",i)
    time.sleep(0.1)
    A.update()
A.close()

output:
image

Torch (cpu version), no bug

(The same code with the last one)
image
Environment (please complete the following information):

  • Enlighten Version: 1.10.2
  • OS and version: win10
  • Console application: cmd, Pycharm Terminal and Powershell

Addition
I've tested it on my remote linux server. It works pretty well under all conditions.

@avylove
Copy link
Contributor

avylove commented Jul 9, 2022

Thanks for reporting. This is going to be difficult because I don't have a like platform to troubleshoot on. I think the issue may be that the console mode is not getting set correctly. Can you add this code and let me know if there are any differences between the cuda version and the non-cuda version?

# Add after manager = enlighten.get_manager()

import sys
from msvcrt import get_osfhandle
import jinxed
from jinxed.win32 import get_console_mode

# Check if stdout is redirected
print(sys.stdout is sys.__stdout__)

# Check for errors in blessed
print(manager.term.errors)

# See what module jinxed is using for terminfo
print(jinxed._terminal.TERM.terminfo.__name__)

# Check console mode
print(get_console_mode(get_osfhandle(sys.stdout.fileno())))

@TheAutumnOfRice
Copy link
Author

Thanks for your reply!

non-cuda version output

1.12.0+cpu
True
["parameters: kind=None, stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, force_styling=False"]
jinxed.terminfo.vtwin10
7

cuda-version output

1.12.0+cu113
False
['parameters: kind=None, stream=<colorama.ansitowin32.StreamWrapper object at 0x000001D9B64BF288>, force_styling=False', 'Output stream is not a default stream']
jinxed.terminfo.vtwin10
7

@avylove
Copy link
Contributor

avylove commented Jul 9, 2022

What that tells us is stdout is getting wrapped by colorama. Try specifying the real stdout when invoking the manager.

manager = enlighten.get_manager(stream=sys.__stdout__)

@TheAutumnOfRice
Copy link
Author

It works! Thank you very very much avylove!

@avylove
Copy link
Contributor

avylove commented Jul 9, 2022

Glad to hear it! I'm wondering if I should make sys.__stdout__ the default rather than sys.stdout. 99% of the time it makes no difference, but in some corner cases, like this, it can make a difference.

@avylove
Copy link
Contributor

avylove commented Sep 29, 2022

Changed default from sys.stdout to sys.__stdout__ in 1.11.0.

@avylove avylove closed this as completed Sep 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants