Start julia in parallel using a command line option

[ViralBShah]

We need an easy way to start julia in parallel through a command line option. The default method can be addprocs_local. Something like the following will be great.

julia -p 4

or

julia -n 4

[JeffBezanson]

@StefanKarpinski, how can we make command line options available to julia? If I write a #!julia script and try to pass it arguments I get julia's usage help. Also, let's add -p to the allowed options, but just pass it on to the julia-level ARGS array. Then I can implement the option in start_client.

[StefanKarpinski]

Hmm. This used to work. I'm not sure what the deal is. Will look into it. Here's what I'm getting now:

$ julia -e 'println(ARGS)'
Array(Any,(0,))
$ julia -e 'println(ARGS)' foo bar
Array(Any,(0,))

[StefanKarpinski]

I think that passing flags that are meant for the repl through in ARGS is a pretty broken idea. We need to emulate the behavior of perl and ruby, both of which are very carefully designed.

[JeffBezanson]

Another problem is that exec_program bypasses start_client, so parallelism is not available. Same for -e and -E. In general julia needs to get control sooner and possibly handle more of the command line options. All we need in C is the ability to load a file in -b mode.

At this point we can think of the repl as a julia program, since it has control (although some pieces are implemented in C). So there are actually three layers of program startup, each potentially with its own set of switches: core julia, start_client, and a user program. What we're doing is somewhat equivalent to starting julia with the command

julia-exec -J sys.ji start_client.j $*

where julia-exec takes a limited set of options, and start_client handles more options, finally passing on other options to any user program. Does that work?

[StefanKarpinski]

Yes, that sounds right. What do you think the sanest way to get to that point is?

[JeffBezanson]

I'd like to avoid using a shell script, but that creates an ambiguity as to whether options are for core julia or start_client. Can we stop getopt as soon as it hits an unknown option (or a non-option), and pass the rest on to start_client to check? Then we need some basic julia-level getopt functionality.

The options q, e, E, P, and L should be moved to start_client. We should pull start_client and friends out of multi.j, into maybe client.j, and gradually start moving repl functionality there. It's not exclusively a repl thing, since it will also handle running programs in batch mode.

[ViralBShah]

Jeff's suggestion is quite reasonable - also takes us in the right direction with some of that refactoring.

[JeffBezanson]

Adding to 1.0 milestone since the improved client start-up (e.g. enabling parallelism for non-interactive programs) is needed, even if the requested command line option is just icing on that cake.