"IOError: closed stream" when using pipes (JRuby 9k only)

[jordansissel]

This issue was originally reported in jordansissel/fpm#1194

I think this problem may be an issue or bug in JRuby specifically, but I wanted to file this here as I walk the dependency chain upwards.

Scenario: I use IO.pipe with ChildProcess usually successfully. Recently, someone reported that on JRuby 9k (specifically I tested JRuby 9.1.8.0), the following occurs:

IOError: closed stream
from org/jruby/RubyIO.java:2172:in `flush'

I can reproduce this error with the output to pipe example:

r, w = IO.pipe

proc = ChildProcess.build("echo", "foo")
proc.io.stdout = proc.io.stderr = w
proc.start
w.close

begin
  loop { print r.readpartial(8192) }
rescue EOFError
end

proc.wait

Here's the behavior on a few different rubies:

Ruby 2.4.1

% ruby --version
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]
% ruby x.rb
foo

JRuby 1.7.26 (this version is EOL, but presented as more fun data)

% ruby --version
jruby 1.7.26 (1.9.3p551) 2016-08-26 69763b8 on OpenJDK 64-Bit Server VM 1.8.0_111-b16 +jit [linux-amd64]
% ruby x.rb
<no output>

JRuby 9.1.8.0

% ruby --version
jruby 9.1.8.0 (2.3.1) 2017-03-06 90fc7ab OpenJDK 64-Bit Server VM 25.111-b16 on 1.8.0_111-b16 +jit [linux-x86_64]
IOError: closed stream
         flush at org/jruby/RubyIO.java:2172
          pump at /home/jls/.rbenv/versions/jruby-9.1.8.0/lib/ruby/gems/shared/gems/childprocess-0.7.0/lib/childprocess/jruby/pump.rb:45
  block in run at /home/jls/.rbenv/versions/jruby-9.1.8.0/lib/ruby/gems/shared/gems/childprocess-0.7.0/lib/childprocess/jruby/pump.rb:18

I believe the sample code (and mine) to be correct, and both cause this error in JRuby 9k. I'm not sure if this is a JRuby bug, yet, but will continue digging anyway. Feel free to close if you agree this is a JRuby 9k bug.

[kyleoliveira]

@jordansissel any luck with this / did you bring it up with the JRuby team? I think I might be seeing the same thing.

[sds]

Was going through the issue list and found this one intriguing. After some digging, this appears to boil down to a race condition in the example code provided.

The error returned is complaining about a closed stream:

IOError: closed stream

If you don't call w.close, the program executes and displays the output as expected, it just won't terminate (since the loop doesn't terminate until EOFError is raised, and that won't happen until w is closed since it's reading from the other end of the pipe, r).

You can reproduce this issue with MRI Ruby by calling w.close before proc.start rather than after:

require 'childprocess'
r, w = IO.pipe

proc = ChildProcess.build("echo", "foo")
proc.io.stdout = proc.io.stderr = w
w.close
proc.start

begin
  loop { print r.readpartial(8192) }
rescue EOFError
end

proc.wait

...(backtrace)...
closed stream (IOError)

The reason the original code doesn't fail on MRI is simply because the process starts and completes execution (and writes all output to the pipe) before w.close is called.

The example code can be rewritten so that the pipe is closed after all output is received:

require 'childprocess'

r, w = IO.pipe

proc = ChildProcess.build("echo", "foo")
proc.io.stdout = proc.io.stderr = w
proc.start

Thread.new {
  begin
    loop do
      print r.readpartial(8192)
    end
  rescue EOFError
  end
}

proc.wait
w.close

The above example works on both MRI and JRuby.