Scripts Usage

This document explains the global and common usage of command line scripts as handled and expected by the library.

For a full example and an help for this tutorial, see the samples/getopts-test.sh script.

Classic command line script usage

The library tries to define the most classic usage of the command line scripts. A global overview of this could be:

path/to/script.sh  -flags((=)arg)  --long-opt(=arg)  (--)  script args
       [1]             [2(a)]            [2(b)]     [2(c)]     [3]

We can distinguish in this usage three entities that are explained below:

1. the program
2. some options with or without argument, required or not
3. some arguments to pass to the script, required or not

The program

It determines the executed script and, on some systems, the shell script to use. We can distinguish here an installed program and a local script file. In the first case, you will just write the program's name, in the second, you may write the full relative path to the executed file.

mycommand ...
# or
./path/to/mycommand ...

The path of the script may exist from you current working directory, or could be stored in a directory that is included in the global $PATH environment variable, which is usually a collection of system's and user's bin/ directories. This is the case of programs.

Depending on the system you are running on, you may precede the program name or the script path calling explicitly the shell binary to use to parse it. On Darwin systems for instance, you may run something like:

bash path/to/script.sh
# or
sh path/to/script.sh

The options

Most of programs or scripts may accept options. An option will, for instance, enable a special environment variable (like verbosity), select a new working directory etc. Options can accept arguments, optional or required, depending on script definitions.

We can here distinguish two types of "options": the short options, also called flags, and the long options. Most of the time, the short options are just aliases of long options, which name will have a real meaning, for quick script's usage.

The short options, or "flags"

A short option is one single letter prefixed by a dash: -a. It can accept an argument that can be written in three equivalent ways:

./path/to/script.sh -a argument
./path/to/script.sh -a=argument
./path/to/script.sh -aargument

The first notation here (a space separator) is not allowed for optional arguments.

The short options can be grouped in a single string like -ab as long as there is no argument inside the group. An argument can be added to the last option of a group.

For instance, the followings are equivalent:

./path/to/script.sh -a -b=arg -c
./path/to/script.sh -ac -b=arg
./path/to/script.sh -acb=arg
./path/to/script.sh -ab=arg -c

The short options are also called "flags" because they are a single letter and they often define a simple behavior, such as rendering the script verbose, quiet, interactive etc.

The long options

A long option is a multi-letters string prefixed by a double-dash: --my-option. It can accept an argument just like the short options, with the difference that the argument MUST be separated from the option name by the equal sign or the space: --my-option=argument.

If a long option accepts an optional argument, the separator must ONLY be the equal sign.

The long options can NOT be grouped.

The end of options

The end of command line options can be explicitly written using the special notation -- (a double dash). This is NOT required but can be useful in some special cases with a long set of options with arguments, to tell the script that the rest of the line will be global arguments and not options' ones.

Note about options arguments

For both short and long options accepting an argument, this argument can be surrounded by quotes:

./path/to/script.sh -a "my arg"

This is NOT required, except in the case where the argument contains a space or a special character (like in the example above). Actually, on certain systems this is not required at all, but it is a good practice to ALWAYS use quotes for complicated arguments to be safely system compliant.

The arguments

Finally, the command line may contains some arguments that will be handled by the script. The arguments may be distinguished from the options or the options arguments. They often represent the "real" things handled by the script's logic while the options are used to define some special behaviors during this logic. The arguments are just some strings written "as is" at the end of the command line. They are NOT required, can be one, two or more.

The case of a previous command output piped

It is common to use the pipe method in command line to process a suite of programs. The library proposes a method to get any piped content from a previous command to use something like:

echo "my piped content" | ./path/to/script.sh

To get the piped content in a script, use the read_from_pipe() method which will load a caught piped content (if so) in the SCRIPT_PIPED_INPUT global variable.

Implementation in the library

Working on the library, I first tried to write global specifications about options and arguments handling following common usages and facilities. These rules are:

• options (short or long) and arguments must be mixable (in any order) keeping in mind that the double-dash (--) means the end of options (but not of arguments)
• of course, an option's argument must be written after its option's name
• for short and long options:
  • if an argument contains special characters or a space, it must be surrounded between quotes (simples or doubles)
• for short options:
  • if the argument is required, it can be written just after the option, separated by an equal sign eventually, or as the next argument, separated by a space
  • if the argument is optional, it must be written just after the option, separated by an equal sign eventually (no space is allowed as the argument is optional)
• for long options:
  • if the argument is required, it can be written separated from the option by an equal sign or a space
  • if the argument is optional, it must be written separated from the option by an equal sign (no space is allowed as the argument is optional)

Re-arrangement of script options

The methods described in this section will load script options in the SCRIPT_OPTS variable array and the arguments in the SCRIPT_ARGS variable array. As it is not possible to reset the global command line positional parameters inside a function, you need to manually redefine them using the set program as shown below. Doing so, you can be safely sure after these lines to have the correct options, loaded with their arguments if so (even if they are multi-words) and the script arguments.

New version

The library can rearrange options and arguments to safely prepare and parse them. To do so, you need to have the new version of the getopt program and write something like:

rearrange_script_options_new "$0" "$@"
[ -n "$SCRIPT_PARAMS" ] && eval set -- "$SCRIPT_PARAMS"

If your system or the one currently in use does not seem to have the new getopt, the rearrange_script_options_new method will fallback to the old rearrange_script_options one (see below).

The old way (atelierspierrot/piwi-bash-library and still valid)

In its first version, the library embedded its own re-arrangement system to safely prepare and parse options and arguments. This "old" method is still existing and valid. To use it, you need to write something like:

rearrange_script_options "$@"
if [ "${#SCRIPT_OPTS[@]}" -gt 0 ] && [ "${#SCRIPT_ARGS[@]}" -eq 0 ];
    then set -- "${SCRIPT_OPTS[@]}";
elif [ "${#SCRIPT_OPTS[@]}" -eq 0 ] && [ "${#SCRIPT_ARGS[@]}" -gt 0 ];
    then set -- "${SCRIPT_ARGS[@]}";
elif [ "${#SCRIPT_OPTS[@]}" -gt 0 ] && [ "${#SCRIPT_ARGS[@]}" -gt 0 ];
    then set -- "${SCRIPT_OPTS[@]}" -- "${SCRIPT_ARGS[@]}";
fi

# or, from version 2.1.0(-xxx) of the library

rearrange_script_options "$@"
[ -n "$SCRIPT_PARAMS" ] && eval set -- "$SCRIPT_PARAMS"

Dealing with arguments

The library defines some methods to get and treat script arguments (global arguments, not options' arguments). It defines and uses a new environment variable named ARGIND which indicates the current argument index, just like OPTIND works for options with the getopts bash builtin.

To get and loop over arguments, you can use the get_next_argument method, which will load the "next" argument (the first one for a first usage) in the ARGUMENT variable and increments ARGIND by one:

# initially
echo $ARGIND
0
echo $ARGUMENT
''

# first usage
get_next_argument
echo $ARGIND
1
echo $ARGUMENT
first-arg

# then, while a next argument is available
get_next_argument
echo $ARGIND
n+1
echo $ARGUMENT
n-arg

For facility, a get_last_argument method is defined to get the last one directly. The ARGIND indexer is not concerned.

On the same model as the internal getopts method, the library defines a getargs method which allows the following while loop usage:

while getargs MYARG; do
    # inside the loop, the `MYARG` variable will contain current argument value
    # and the `ARGIND` will contain current argument index
    echo "> current argument is ${MYARG} with index ${ARGIND}"
done

For instance, if your script is based on positional arguments, you can write something like:

while getargs MYARG; do
    # in the loop, ARGIND is the current argument index + 1 (next argument index)
    case $ARGIND in
        2) write here what to do with first argument with value $MYARG ;;
        3) write here what to do with second argument with value $MYARG ;;
    esac
done

Usage of common options

To use common options in a script, just call the parse_common_options method passing it the script arguments:

#!/bin/bash
source path/to/piwi-bash-library.bash
parse_common_options "$@"

After that, if you want to define and use some custom options, you can write (here for the custom options '-t' and '--test' with arguments):

# any re-arrangement must come before ...

OPTIND=1
while getopts "t:${COMMON_OPTIONS_ALLOWED}" OPTION; do
    OPTARG="${OPTARG#=}"
    case "$OPTION" in
        t) 
            # your script for option 't' with argument $OPTARG
            ;;
        -) case "$OPTARG" in
            test) 
                # your script for option 'test' with argument "${OPTARG#*=}"
                ;;
            esac
        ;;
    esac
done

To get any long option argument, you can use:

parse_long_option "$OPTARG" "${!OPTIND}"

This will load the long option name in the LONGOPTNAME variable, its argument (if so) in the LONGOPTARG variable and update the OPTIND variable if necessary.

If you use a library version less than 2.1.0-alpha, you must use:

LONGOPTARG="$(get_long_option_arg "$OPTARG")"

Options parsing and error reporting

A special parse_common_options_strict method is defined as an alias of the classic parse_common_options but throwing an error for undefined options. To use it, you MUST define both OPTIONS_ALLOWED and LONG_OPTIONS_ALLOWED strings in your script as they are used to define known options names. You can use the library COMMON_OPTIONS_ALLOWED and COMMON_LONG_OPTIONS_ALLOWED and complete them with your own options, like:

OPTIONS_ALLOWED="t:a${COMMON_OPTIONS_ALLOWED}"
LONG_OPTIONS_ALLOWED="test1:,test2::,${COMMON_LONG_OPTIONS_ALLOWED}"

These two variables follow the original getopt program specifications:

• the sort options are listed without any separator
• the long options are listed separated by coma
• when an option waits for a required argument, it is suffixed by one colon
• when an option waits for an optional argument, it is suffixed by two colons

In the example above, the parse_common_options_strict method will consider the -t option as allowed but will throw an error with a -z option that seems not allowed. It works the same way for long options. More, it will throw an error if the test1 long option does not have an argument (as it required) but not if the test2 does not (as it optional). It works the same way for short options.

Full example

New way example

Below is a complete example of a script which first re-arrange the command line call and then parse the common options throwing an error for unknown options (taken from the samples/getopts-test.sh test script).

OPTIONS_ALLOWED="t:a${COMMON_OPTIONS_ALLOWED}"
LONG_OPTIONS_ALLOWED="test1:,test2::,${COMMON_LONG_OPTIONS_ALLOWED}"

rearrange_script_options_new "$0" "$@"
[ -n "$SCRIPT_PARAMS" ] && eval set -- "$SCRIPT_PARAMS"

parse_common_options_strict "$@"

# loop over options
OPTIND=1
while getopts ":${OPTIONS_ALLOWED}" OPTION; do
    OPTARG="$(get_option_arg "${OPTARG:-}")"
    case "$OPTION" in

        # case of long options
        -)  parse_long_option "$OPTARG" "${!OPTIND}"
            case "$LONGOPTNAME" in
                *) echo " - [${OPTIND}] long option '${LONGOPTNAME}' with arg '${LONGOPTARG}'";;
                \?) echo " - [${OPTIND}] unknown long option '${LONGOPTNAME}'";;
            esac
            ;;

        # case of short options
        *) echo " - [${OPTIND}] option '$OPTION' with arg '$OPTARG'";;
        \?) echo " - [${OPTIND}] unknown option '$OPTION'";;

    esac
done

# loop over arguments
while [[ "$ARGIND" -lt "${#SCRIPT_ARGS[@]}" ]]; do
    get_next_argument
    echo " - [${ARGIND}] argument is '$ARGUMENT'"
done

"old way" example

Below is a complete example of a script which first re-arrange the command line call and then parse the common options throwing an error for unknown options (taken from the samples/getopts-test-old.sh test script).

OPTIONS_ALLOWED="t:a${COMMON_OPTIONS_ALLOWED}"
LONG_OPTIONS_ALLOWED="test:,${COMMON_LONG_OPTIONS_ALLOWED}"

rearrange_script_options "$@"
if [ "${#SCRIPT_OPTS[@]}" -gt 0 ] && [ "${#SCRIPT_ARGS[@]}" -eq 0 ];
    then set -- "${SCRIPT_OPTS[@]}";
elif [ "${#SCRIPT_OPTS[@]}" -eq 0 ] && [ "${#SCRIPT_ARGS[@]}" -gt 0 ];
    then set -- "${SCRIPT_ARGS[@]}";
elif [ "${#SCRIPT_OPTS[@]}" -gt 0 ] && [ "${#SCRIPT_ARGS[@]}" -gt 0 ];
    then set -- "${SCRIPT_OPTS[@]}" -- "${SCRIPT_ARGS[@]}";
fi

parse_common_options_strict

# own options usage
OPTIND=1
while getopts ":at:${OPTIONS_ALLOWED}" OPTION; do
    OPTARG="${OPTARG#=}"
    case "$OPTION" in
        t) _echo " - option 't': receiving argument \"${OPTARG}\"";;
        a) echo " - test option A";;
        -)  # for long options with argument, use fct 'get_long_option_arg ( $arg )'
            LONGOPTARG="$(get_long_option_arg "$OPTARG")"
            case "$OPTARG" in
                test*) _echo " - option 'test': receiving argument '${LONGOPTARG}'";;
                ?) echo " - unknown long option '$OPTARG'";;
            esac ;;
        ?) echo " - unknown option '$OPTION'";;
    esac
done

# arguments usage
lastarg=$(get_last_argument)
echo " - last argument is '${lastarg}'"

get_next_argument
echo " - first argument is '$ARGUMENT'"
get_next_argument
echo " - next argument is '$ARGUMENT'"

echo " - final 'ARGIND' is: $ARGIND"