Restructure-cli

[PhilippeMoussalli]

With the introduction of a new compiler and runner for vertex, I feel the need to restructure the CLI a bit. This is a draft PR (docs and tests still need to be updated).

Running

• Before

fondant run pipeline.py --local

• After

Mode is not longer a positional argument, Pipeline/reference is moved to the end (similar to docker).

 fondant run local pipeline.py 

Help

• Before

Arguments are not isolated per mode which makes it which argument belongs to which mode (e.g. extra volumes not needed for KFP)

❯ fondant run -h                 
usage: fondant run [-h] (--local | --kubeflow) [--output-path OUTPUT_PATH] [--extra-volumes EXTRA_VOLUMES [EXTRA_VOLUMES ...]] [--build-arg BUILD_ARG] [--host HOST] ref

positional arguments:
  ref                   Reference to the pipeline to run, can be a path to a spec file or a module containing the pipeline instance that will be compiled first (e.g. pipeline.py)

options:
  -h, --help            show this help message and exit
  --local
  --kubeflow
  --output-path OUTPUT_PATH, -o OUTPUT_PATH
                        Output directory
  --extra-volumes EXTRA_VOLUMES [EXTRA_VOLUMES ...]
                        Extra volumes to mount in containers
  --build-arg BUILD_ARG
                        Build arguments to pass to `docker build`. Format {key}={value}.
  --host HOST           KubeFlow pipeline host url

• After

❯fondant run -h  

positional arguments:
  {local,kubeflow}
    local           Local runner
    kubeflow        Kubeflow runner
  ref               Reference to the pipeline to run, can be a path to a spec file or a module containing the pipeline instance that will be compiled first (e.g. pipeline.py)

options:
  -h, --help        show this help message and exit

To further deep dive

❯ fondant run local -h                  
usage: fondant run local [-h] [--output-path OUTPUT_PATH] [--extra-volumes EXTRA_VOLUMES [EXTRA_VOLUMES ...]] [--build-arg BUILD_ARG]

options:
  -h, --help            show this help message and exit
  --output-path OUTPUT_PATH, -o OUTPUT_PATH
                        Output path of compiled pipeline
  --extra-volumes EXTRA_VOLUMES [EXTRA_VOLUMES ...]
                        Extra volumes to mount in containers
  --build-arg BUILD_ARG
                        Build arguments for `docker build`

[RobbeSneyders]

Ok, this looks good to me @PhilippeMoussalli.

[mrchtr]

Lgtm, just have one smaller suggestion.

[mrchtr]

Is there a reason for going away from the default docker-compose naming?

Suggested change

	default="docker_compose.yml",
	default="docker-compose.yml",

[PhilippeMoussalli]

Good catch, forgot that it was referenced like this

[mrchtr]

See my comment above.

Suggested change

	default="docker_compose.yml",
	default="docker-compose.yml",