Simplify installation experience for Notation CLI

[FeynmanZhou]

Notation CLI only has binary for installation. This is convenient for a few Linux distros but it is cumbersome to install on Win, macOS, and other Linux distros. There are some existing requests as follows:

notaryproject/notation#204
notaryproject/notation#431

To simplify the installation experience, we need to consider providing a native installation package on each platform/OS so that users can install Notation CLI as simply as possible. Here are the popular installation methods for consideration:

macOS

• Homebrew

Windows

• Winget
• Chocolaty (low priority)

Linux / Unix

• Offline installation package: Deb and RPM
• Debian: apt (need investigation)
• Arch: pacman (need investigation)
• Alpine: apk (need investigation)
• Homebrew

Container image (e.g. docker run)

It relies on a container engine like Docker.

[yizha1]

Update the milestone to rc-3 based on the discussion.

[kokamkarsahil]

Hello, I am interested in working on this issue.

I was thinking of an installation script which will cover all Linux distros and macOS based on its architecture, which will download a compactible release from GitHub releases and setup it. And a browser based platform detection to switch to a prompt suggesting to download .exe instead or running the script.

This will reduce the task of maintaining packages for all the different Linux distros.

Similar to rustup[0]

0. https://rustup.rs

Thank You!

[FeynmanZhou]

@kokamkarsahil Developing a generic installation script to cover all Linux distros and macOS sounds good to me. I am thinking if we could provide a script and enable users to install Notation CLI with this command:

curl -L https://notaryproject.dev/download | NOTATION_VERSION=v1.0.0 TARGET_ARCH=x86_64 sh -

For macOS, having a Homebrew formula might be better.

[kokamkarsahil]

Thanks for the reply @FeynmanZhou!

@kokamkarsahil Developing a generic installation script to cover all Linux distros and macOS sounds good to me. I am thinking if we could provide a script and enable users to install Notation CLI with this command:

curl -L https://notaryproject.dev/download | NOTATION_VERSION=v1.0.0 TARGET_ARCH=x86_64 sh -

We can also make script to auto detect Architecture and OS.

By checking $OSTYPE[0] and uname -m[1] from the client.

I have create a demo script to show that:
https://gist.github.com/kokamkarsahil/bc1ac4938f113fb4246613e4e9bf5501

Moreover we can also point it towards main domain and do user-agent[2] check to send the install script.

curl -L https://notaryproject.dev

Script in action

Updated to gif

Please don't try the script in non-sandbox environment as it is not well tested in all platforms

Thank You!

---

Update: Added reference

0. https://www.gnu.org/software/bash/manual/html_node/Bash-Variables.html#index-OSTYPE
1. https://man7.org/linux/man-pages/man1/uname.1.html
2. https://everything.curl.dev/http/requests/user-agent

[FeynmanZhou]

@kokamkarsahil Thanks for providing the demo. I will test it on my machines. We will discuss and triage this issue in the next Notary community meeting.

[kokamkarsahil]

@kokamkarsahil Thanks for providing the demo. I will test it on my machines. We will discuss and triage this issue in the next Notary community meeting.

Sure, will join in to discuss about it.

[shizhMSFT]

The scripts from servers are convenient to use as many vendors are doing so

• Docker: curl https://get.docker.com/ | sh
• Homebrew: /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

However, there is a security concern on running scripts downloaded from the remote server without code review in the production environment as clients might be tricked to run arbitrary code without detection / attention.

Here are some comments from the Docker script:

Docker Engine for Linux installation script.

This script is intended as a convenient way to configure docker's package
repositories and to install Docker Engine, This script is not recommended
for production environments. Before running this script, make yourself familiar
with potential risks and limitations, and refer to the installation manual
at https://docs.docker.com/engine/install/ for alternative installation methods.

The script:

• Requires root or sudo privileges to run.
• Attempts to detect your Linux distribution and version and configure your
  package management system for you.
• Doesn't allow you to customize most installation parameters.
• Installs dependencies and recommendations without asking for confirmation.
• Installs the latest stable release (by default) of Docker CLI, Docker Engine,
  Docker Buildx, Docker Compose, containerd, and runc. When using this script
  to provision a machine, this may result in unexpected major version upgrades
  of these packages. Always test upgrades in a test environment before
  deploying to your production systems.
• Isn't designed to upgrade an existing Docker installation. When using the
  script to update an existing installation, dependencies may not be updated
  to the expected version, resulting in outdated versions.

Source code is available at https://github.com/docker/docker-install/

Usage

To install the latest stable versions of Docker CLI, Docker Engine, and their
dependencies:

1. download the script

$ curl -fsSL https://get.docker.com -o install-docker.sh

2. verify the script's content

$ cat install-docker.sh

3. run the script with --dry-run to verify the steps it executes

$ sh install-docker.sh --dry-run

4. run the script either as root, or using sudo to perform the installation.

$ sudo sh install-docker.sh

[shizhMSFT]

The User-Agent based one is also convenient but has lots of limitations. To ensure integrity, the client still need to verify the downloaded tar file against the target digest / checksum.

[kokamkarsahil]

Thanks for the reply! I will try my best to address all the issues.

However, there is a security concern on running scripts downloaded from the remote server without code review in the production environment as clients might be tricked to run arbitrary code without detection / attention.

The script won't need to be stored in a remote server, it can use git and same static web server used for hosting the website like Netlify. As for tricking clients, it wouldn't be concerns if all files and instructions are clear on the site.

Here are some comments from the Docker script:

Docker Engine for Linux installation script.

This script is intended as a convenient way to configure docker's package
repositories and to install Docker Engine, This script is not recommended
for production environments. Before running this script, make yourself familiar
with potential risks and limitations, and refer to the installation manual
at https://docs.docker.com/engine/install/ for alternative installation methods.

The script:

• Requires root or sudo privileges to run.

Sure we can also run without sudo privileges, it's on my TODO like for e.g. We can store that binary in separate folder like .notation and then source bashrc/zshrc. To avoid using sudo privileges.

• Attempts to detect your Linux distribution and version and configure your
  package management system for you.

The notation script doesn't make use of package management software.

• Doesn't allow you to customize most installation parameters.

We can add customize parameters to it via extra arguments via sh -s -- -args

• Installs dependencies and recommendations without asking for confirmation.

As notation is single binary it doesn't need extra dependenices to be installed like docker and doesn't mess with dependencies..

• Installs the latest stable release (by default) of Docker CLI, Docker Engine,

Sure, notation script does the same, but we can also specify different version via ENV variable.

Docker Buildx, Docker Compose, containerd, and runc. When using this script
to provision a machine, this may result in unexpected major version upgrades
of these packages. Always test upgrades in a test environment before
deploying to your production systems.

• Isn't designed to upgrade an existing Docker installation. When using the
  script to update an existing installation, dependencies may not be updated
  to the expected version, resulting in outdated versions.

Notation script can update version. But will not be its default behavior to avoid any breaking changes from updates.

Source code is available at https://github.com/docker/docker-install/

Usage

To install the latest stable versions of Docker CLI, Docker Engine, and their
dependencies:

1. download the script

$ curl -fsSL https://get.docker.com -o install-docker.sh

2. verify the script's content

$ cat install-docker.sh

3. run the script with --dry-run to verify the steps it executes

$ sh install-docker.sh --dry-run

I had thought if it to add in script itself and prompt user to confirm the steps. But as script does have any step to remove file, it won't add any value. But it can stout the steps it did on terminal.

4. run the script either as root, or using sudo to perform the installation.

$ sudo sh install-docker.sh

Overall they have marked it unsafe because it does a lot of changes in system while the notation script just download a binary without deleting or adding anything else to the system.

Moreover, some projects use it as their default installation method like k3: https://k3s.io/

curl -sfL https://get.k3s.io | sh - 
# Check for Ready node, takes ~30 seconds 
sudo k3s kubectl get node 

The User-Agent based one is also convenient but has lots of limitations. To ensure integrity, the client still need to verify the downloaded tar file against the target digest / checksum.

It's also on my TODO as most users don't verify them the script can auto check it and fail on unmatched checksum.

Sorry if I missed addressing any problem please let me know of it I will address it ASAP.

I will also join the community meeting this week for further, more discussion.

Thank you!