Python CLI

The command-line interface (CLI) is a Python script which is built on top of the GPT4All Python SDK (wiki / repository) and the typer package. The source code, README, and local build instructions can be found here. Required is at least Python 3.8.

In a Nutshell

The CLI is a Python script called app.py. If you're already familiar with Python best practices, the short version is to download app.py into a folder of your choice, install the two required dependencies with some variant of:

pip install gpt4all typer

Then run it with a variant of:

python app.py repl

In case you're wondering, REPL is an acronym for read-eval-print loop.

Installation

Recommendations

Important

Especially if you have several applications/libraries which depend on Python, to avoid descending into dependency hell at some point, you should:

• Always install into some kind of virtual environment.
• On a Unix-like system, don't use sudo for anything other than packages provided by the system package manager. Specifically, never use sudo with pip.

There are several ways and tools available to do this properly, so below are descriptions on how to install with a virtual environment (recommended) or a user installation on all three main platforms.

Different platforms can have slightly different ways to start the Python interpreter itself.

Tip

Typer has an optional dependency for more fanciful output. If you want that, replace typer with typer[all] in the pip-install instructions below.

Virtual Environment Installation

You can name your virtual environment folder for the CLI whatever you like. In the following, gpt4all-cli is used throughout.

macOS

There are at least three ways to have a Python installation on macOS, and possibly not all of them provide a full installation of Python and its tools. When in doubt, try the following:

python3 -m venv --help
python3 -m pip --help

Both should print the help for the venv and pip commands, respectively. If they don't, consult the documentation of your Python installation on how to enable them, or download a separate Python variant, for example try an unified installer package from python.org.

Once ready, do:

python3 -m venv gpt4all-cli
. gpt4all-cli/bin/activate
python3 -m pip install gpt4all typer

Windows

Download the official installer from python.org if Python isn't already present on your system.

A Windows installation should already provide all the components for a virtual environment. Run:

py -3 -m venv gpt4all-cli
gpt4all-cli\Scripts\activate
py -m pip install gpt4all typer

Linux

On Linux, a Python installation is often split into several packages and not all are necessarily installed by default. For example, on Debian/Ubuntu and derived distros, you will want to ensure their presence with the following:

sudo apt-get install python3-venv python3-pip

The next steps are similar to the other platforms:

python3 -m venv gpt4all-cli
. gpt4all-cli/bin/activate
python3 -m pip install gpt4all typer

On other distros, the situation might be different. Especially the package names can vary a lot. You will have to look them up in the documentation, software directory, or package search.

User Installation

A user installation may share dependencies with other Python software.

macOS

There are at least three ways to have a Python installation on macOS, and possibly not all of them provide a full installation of Python and its tools. When in doubt, try the following:

python3 -m pip --help

That should print the help for the pip command. If it doesn't, consult the documentation of your Python installation on how to enable them, or download a separate Python variant, for example try an unified installer package from python.org.

Once ready, do:

python3 -m pip install --user --upgrade gpt4all typer

Windows

Download the official installer from python.org if Python isn't already present on your system. It includes all the necessary components. Run:

py -3 -m pip install --user --upgrade gpt4all typer

Linux

On Linux, a Python installation is often split into several packages and not all are necessarily installed by default. For example, on Debian/Ubuntu and derived distros, you will want to ensure their presence with the following:

sudo apt-get install python3-pip

The next steps are similar to the other platforms:

python3 -m pip install --user --upgrade gpt4all typer

On other distros, the situation might be different. Especially the package names can vary a lot. You will have to look them up in the documentation, software directory, or package search.

Download the CLI

The CLI is a self-contained script called app.py (download). As such, you can save it and run it from anywhere you like, as long as the Python interpreter has access to the mentioned dependencies.

Run the CLI

Note

Different platforms can have slightly different ways to start Python. Whereas below the interpreter command is written as python you typically want to type instead:

• On Unix-like systems: python3
• On Windows: py -3

The simplest way to start the CLI is:

python app.py repl

This automatically selects the Mistral Instruct model and downloads it into the .cache/gpt4all/ folder of your home directory, if not already present.

If you want to use a different model, you can do so with the -m/--model parameter. If only a model file name is provided, it will again check in .cache/gpt4all/ and might start downloading. For models outside that cache folder, use their full path instead. For example:

python app.py repl --model /home/user/my-gpt4all-models/mistral-7b-instruct-v0.1.Q4_0.gguf

When you're done talking to a model and want to end a session, simply type /exit.

To get help and information on all the available commands and options on the command-line, run:

python app.py --help

Additionally, a description for the REPL options can be displayed with:

python app.py repl --help

And while inside the running REPL, write /help.

Finally, if on Windows you see a box instead of an arrow ⇢ as the prompt character, you should change the console font to one which offers better Unicode support.

Shell Aliases

Note that if you've installed the required packages into a virtual environment, you don't need to activate that every time you want to run the CLI. Instead, you can just start it with the Python interpreter in the folder gpt4all-cli/bin/ (Unix-like) or gpt4all-cli/Script/ (Windows).

That also makes it easy to set an alias e.g. in Bash or PowerShell:

• Bash:

  alias gpt4all="'/full/path/to/gpt4all-cli/bin/python' '/full/path/to/app.py' repl"

• PowerShell:

  Function GPT4All-Venv-CLI {"C:\full\path\to\gpt4all-cli\Scripts\python.exe" "C:\full\path\to\app.py" repl}
  Set-Alias -Name gpt4all -Value GPT4All-Venv-CLI

Don't forget to save these in the start-up file of your shell.