Running Julia files on Windows in bash shell seems broken

[WestleyArgentum]

To reproduce this issue create some file "foo.jl" and, inside it, write something like:

"some_int = 42
print(some_int)"

Or maybe just a string of junk "fjdsjkd;dfsajeknnsb". I've played with it a bit and gotten a range of behavior.

Next fire up a terminal and say "julia foo.jl"

What comes out is occasionally the correct output but often only a single character and sometimes nothing at all... Also, note that I have no problems when running in repl mode.

[vtjnash]

I can't reproduce this. Can you provide more information about your system configuration? What terminal are you using? Which version of Julia? Does using println instead of print help?

[WestleyArgentum]

This is interesting. I've done more testing and I can repro it in cygwin bash shells and when running the Julia process using a wrapper called QProcess that is part of the Qt SDK. There is no problem running in the regular command prompt.

I'm using the windows binaries that are up on github and I've tried on machines with with and without cygwin and the Qt SDK (including a plain vanilla windows 7 virtual machine).

I believe the process is executing correctly because the output I do get is always the first chunk of the correct output. What seems to happen is that Julia exits before completely flushing standard out...

I see line 206 in client.jl: "atexit(()->flush(stdout_stream))"
It seems really unlikely, but is it possible that atexit() isn't working properly? I've had issues with it in the past (although supposedly those days are long gone).

[vtjnash]

Thanks! I had tried it in MinGW and the regular command prompt and, as you said, it worked fine there.

If you look at the source code for the Windows binaries, I've had to comment out that line since libuv doesn't have a notion of flush.

Unless @loladiro knows how to add a flush method more directly, perhaps we should enter uv_run() at the end with one task that deletes all items that it can reach with uv_walk, then waits for libuv to shutdown the event loop.

[Keno]

We should probably wait for the close_cb on stdout.

[WestleyArgentum]

Ahhh, I can't believe I was looking at client.jl in the main repository instead of this one >.<

Anyway, thanks very much for looking at this... how about we give it a shot? Are you guys able to compile the source from this repo on windows using the main makefile or are there tricks that I have to know about? I've been unsuccessful a couple times but I may just be missing dependencies...

[vtjnash]

Alright, I'll see about adding some code that tries to close everything and waits for libuv to exit more gracefully. I think that is more general then just waiting for close_cb on stdout? (although I may try that first to verify, when I can get some time free)

If you have MinGW with gcc, you should be able to pretty much just run make in the top level and everything should build. I'm not aware of any other dependancies, but perhaps Keno remembers others? I don't know what will happen if you try to build it in Cygwin, but I don't see why it shouldn't be possible (possibly requiring the use of the MinGW compiler and possibly requiring various flags like CC=gcc-mingw-4.5, etc?). If you can add a bug report with the specific error message, it can help us identify what should be added to the README, or fix.

[WestleyArgentum]

I really do hate to pester you like this, but have you had any time to look at this issue since we last talked?

[vtjnash]

Thanks for the reminder, I had almost forgotten. I just took a look and found that it is in fact essential (on Windows) to close open pipes and streams before exiting to ensure the write callbacks get executed by Windows. TTY's are handled differently, so normal consoles worked as expected. I'll try to get a fix out for this soon.

[WestleyArgentum]

Awesome, I look forward to it :)

[WestleyArgentum]

@vtjnash: I just noticed your commit from a couple days ago, does that seem to solve the problem?

[vtjnash]

I showed that the concept was right, but that commit wasn't quite right. Libuv may need to be updated and fixed to actually correct this, but I won't be sure until there is a clear testcase (in c) that can demonstrate the failure.

[WestleyArgentum]

@vtjnash: I'm not sure if we've spoken in person before or not. I'm working at Forio on a project relating to Julia and, well, Windows support has become a very time critical thing for us. Could you please send me an email at [email protected] so that we can discuss this?

[vtjnash]

note: simplest test case for this was ./julia test.jl | cat -

@WestleyArgentum I appreciate your urgency, but this is not my day job. Additionally, I sometimes have to sleep on the issues for a few days to figure out where I went wrong. (the commit from a few days ago was only a fraction of what I eventually tested to no avail)

[WestleyArgentum]

@vtjnash: Sorry, I didn't mean to come across that demanding and I certainly don't mean to pin this on you. I just wasn't quite sure what you needed and, because we were dependent, I wanted to find some way help...

Thanks so much for putting in so much time on this, heh, and I totally know what you mean by needing to sleep on things.

Would you be willing to make and put up windows binaries for this? We can do some pretty extensive testing to make sure everything is alright if you want :)

[vtjnash]

@WestleyArgentum no problem. I tried building binaries for windows, but they are failing tests either because something is getting into the garbage collector uninitialized, or because the stack pointer is getting corrupted by gcc-compiled code, or both. I currently suspect the former. If you have time to try to backtrace this GC value to it's origin (revealed by running julia-debug-basic with gcc), we could probably get the binaries released sooner.

[JeffBezanson]

Does the uninitialized value bug exist on other platforms too?

[vtjnash]

moving discussion to issue #34