Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: deprecate readsfrom/writesto in favor of simply using open on AbstractCmd #6948

Merged
merged 3 commits into from
May 31, 2014

Conversation

stevengj
Copy link
Member

Supersedes #6945. Now you can simply do e.g.

open(readline, `ls`, "r")

to read from a command or e.g.

open(`less`, "w", STDOUT) do io
    for i = 1:1000
          println(io, i)
    end
end

to write to a command.

The old readsfrom(cmd) is replaced by open(cmd, "r"), and writesfrom(cmd) is replaced by open(cmd, "w").

The deprecated readsfrom and writesto commands are currently used only in 5 packages: Graphs (cc: @pozorvlak, @lindahua), AWS (cc: @amitmurthy), ClusterManagers (cc: @nlhepler), ImageView (cc: @timholy), and WinRPM (cc: @ihnorton, @vtjnash).

cc: @StefanKarpinski, @IainNZ, @loladiro

@stevengj stevengj changed the title RFC deprecate readsfrom/writesto in favor of simply using open on AbstractCmd RFC: deprecate readsfrom/writesto in favor of simply using open on AbstractCmd May 24, 2014
@stevengj
Copy link
Member Author

(As discussed in #6945, it would be nice to eventually replace readandwrite(cmd) by open(cmd, "r+"), but this seems to require some deeper changes to Pipe.)

…tCmds, and support: open(cmd, mode) do io ... end
!success(P) && pipeline_error(P)
return ret
catch
kill(P)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need close(io) here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that killing P would be enough, but you're probably right that I should close(io) too. However, this may some rearrangement to avoid calling close(io) twice, avoid killing P if we throw from pipeline_error(P), and so on.

@pozorvlak
Copy link
Contributor

Looks good to me, and it would be trivial to port Graphs.jl to the new interface. Display-via-pager would also be a huge win for large graphs!

Similar to ``open(command, mode, stdio)``, but calls ``f(stream)``
on the resulting read or write stream, then closes the stream
and waits for the process to complete. Returns the value returned
by ``f``.

.. function:: writesto(command)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also needs to be removed.

@JeffBezanson
Copy link
Member

I like it. What about standard error? Maybe readandwrite should handle that?

@stevengj
Copy link
Member Author

Do you mean, add an extra parameter for stderr?

@StefanKarpinski
Copy link
Member

An interesting notion is to bundle all three standard handles into some IOBundle object. Since we tend to write error messages using different function than we print things (i.e. warn, error), this would not be unreasonable. In other words, you could write println(io, "hello") and warn(io, "uh, oh") and the IOBundle object, io, would route the output to the appropriate handle. We're already doing this here for input and output to the process anyway.

@StefanKarpinski
Copy link
Member

Of course this is a little different since its the spawned process that is doing the writing to stderr, not us, but having a composite kind of IO object seems relevant anyway.

@stevengj
Copy link
Member Author

It would be nice to have some sh-like pipe operator to redirect STDERR, because you might want to pipe STDERR into some arbitrary process, not just open.

In any case, that should probably wait for a separate PR. Is this PR good to merge?

stevengj added a commit that referenced this pull request May 31, 2014
RFC: deprecate readsfrom/writesto in favor of simply using open on AbstractCmd
@stevengj stevengj merged commit 75f7732 into JuliaLang:master May 31, 2014
@stevengj
Copy link
Member Author

It seemed like the changes here were uncontroversial, and the only question is future enhancements like stderr and read/write pipes, so I went ahead and merged.

@BobPortmann
Copy link
Contributor

This

open(`less`, "w", STDOUT) do io
    for i = 1:1000
          println(io, i)
    end
end

sort of works but it does not seem to use less. The output is dumped to the screen w/o paging.

@stevengj
Copy link
Member Author

stevengj commented Jun 1, 2014

@BobPortmann, works for me. What OS do you have?

@BobPortmann
Copy link
Contributor

Mac OS X Mavericks 10.9.3

julia> versioninfo()
Julia Version 0.3.0-prerelease+3359
Commit 75f7732 (2014-05-31 18:15 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin13.2.0)
  CPU: Intel(R) Core(TM)2 Duo CPU     T9550  @ 2.66GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY)
  LAPACK: libopenblas
  LIBM: libopenlibm

@stevengj
Copy link
Member Author

stevengj commented Jun 2, 2014

@BobPortmann, I just tried it on 10.9.3 with 0.3.0-prerelease+3381 (the latest nightly), and writing to less works fine for me.

@BobPortmann
Copy link
Contributor

OK. It works for me too. Not sure what I did different before.

@kmsquire
Copy link
Member

kmsquire commented Jun 3, 2014

The old readsfrom(cmd) is replaced by open(cmd, "r"), and writesfrom(cmd) is replaced by open(cmd, "w").

Would be good to document these as well, especially since they return a (stream, process) tuple. Also should remove docs for old readsfrom and writesto.

@kmsquire
Copy link
Member

kmsquire commented Jun 3, 2014

Nevermind. My helpdb.jl was out of date.

@stevengj stevengj deleted the cmdio branch June 3, 2014 12:22
@mhinsch
Copy link

mhinsch commented Jun 9, 2014

Two comments:

  • The example in the docs is displayed as text instead of code.
  • Is there still a way to get the exit code of the executed process?

@mhinsch
Copy link

mhinsch commented Jun 9, 2014

Also, when using the method open(cmd, mode) there seems to be no way to close the returned stream.

Edit:

Ok, I see now that the return value is a tuple (Pipe, Process), so closing as well as exit code are solved. Still, that's a bit inconsistent with the other uses of open I think.

@JeffBezanson
Copy link
Member

Maybe the form of open that accepts a function should return the Process, or a tuple (value, Process).

@StefanKarpinski
Copy link
Member

I think returning the process is a good idea. I don't think it's very common to need the value of a open block, so just the process should be fine – you can always use a local variable instead.

@stevengj
Copy link
Member Author

stevengj commented Jun 9, 2014

On the contrary, I suspect it's extremely common to want the value of an open block if you are opening it for reading, since x = open(...) do .. end is the easiest way to obtain a scalar value from the read.

I don't see the point of returning the Process in the function form since the Process is no longer running when the open(f::Function, ...) returns.

@mhinsch
Copy link

mhinsch commented Jun 9, 2014

I don't see the point of returning the Process in the function form since the Process is no longer running when the open(f::Function, ...) returns.

I think the idea is to make the exit code of the process available this way (in order to know whether it failed). But I agree, I would imagine the return value of the function to be more interesting than the process/exit code in most cases. Couldn't open instead throw an exception if the process failed?

More generally, however, I find it quite weird to have open return completely different categories of values dependent on how it was called. IMHO this suggests that this is an "overunification" of concepts.

@JeffBezanson
Copy link
Member

Ah, I believe open (the block form) does throw an error if the process fails.

@kmsquire
Copy link
Member

kmsquire commented Jun 9, 2014

More generally, however, I find it quite weird to have open return completely different categories of values dependent on how it was called. IMHO this suggests that this is an "overunification" of concepts.

I was thinking the same thing earlier--thanks for stating it in this way, @mhinsch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
io Involving the I/O subsystem: libuv, read, write, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants