Is nvi supposed to run inside CI?

[commonquail]

It's a little unclear whether nvi is supposed during image building, by installing Node.js into the image, or inside the container itself after build time. In case of the former, I think the paragraph

We really want to use containerized CI, so we considered how other dependencies are injected in our Docker containers, and how to do the same for our Node.js version installer. A common solution is to download an executable, put it on the path, and then build it into the image used by your CI. This is exactly what nvi strives to support.

needs to be rephrased somehow. In case of the latter we should provide more detailed instructions on how to go about that, perhaps even supplying a Dockerfile for demonstrational purposes. Further, we need to investigate image compatibility (for instance, nvi does not work in Alpine, and cannot be trivially made to).

[emiln]

The paragraph is poorly phrased, indeed. Here's a version that seems clearer to me:

We really want to use containerized CI, so we considered how other dependencies are injected in our Docker containers, and how to do the same for our Node.js version installer. A common solution is to add the following steps to the Dockerfile used in building your CI environment:

1. Download a single-file version of the tool.
2. Make the file executable.
3. Put the executable file on the $PATH.

If you install nvi into your CI Docker image like this, you can run your CI scripts inside a container based on this image and call nvi early in the script to have node and npm available in later stages of the script.

I agree that we may need an example Dockerfile. Our current CI environment is unfortunately too privileged to be used as an example, but it can easily be redacted.

Does this alternative phrasing clear things up?

[emiln]

The lack of alpine support is bad. I've seen a lot of interest in alpine around the Node.js community. It seems that prior art is all about downloading the required GNU tools for building Node.js, compiling Node.js from source using GNU tools, and uninstalling the GNU tools. This puts us in a poor spot for supporting installing Node.js for alpine, I fear.

[commonquail]

Yes.

However, that presents some interesting compatibility trials. I think I had trouble with centos:6, too.

[emiln]

We could write a script that attempts to download and use nvi within a bunch of different Docker images that are popular as CI base images. I'm very interested in the compatibility of nvi.

[commonquail]

You might not want to care about Alpine, at least for a start. Alpine uses musl, not glibc, and everything is statically linked. Unfortunately things link with glibc willy-nilly, so many things don't Just Work in Alpine. It's entirely likely the default distribution of Node.js has that problem.

[emiln]

I don't think alpine is a low-hanging fruit, but other distros might be. The first step is probably to automate the process of figuring out compatibility.

[commonquail]

• centos:6:
  • no wget, but curl
  • rm -d dir/ doesn't work. command -r dir/ does but defeats the purpose. rmdir dir/ works
• centos:7:
  • no wget, but curl
• ubuntu:14.04:
  • no wget or curl
• ubuntu:16.04:
  • no wget or curl

You could do something like

FROM ubuntu:16.04

RUN apt update && apt install --yes wget

COPY nvi /
COPY test-nvi /

CMD ["/test-nvi"]

for different distros but the package manager makes it a little difficult to automate this across distros.

I used the test script I showed in #11 but that's insufficient.

Also, I broke BIN_DIR for relative paths. Oops.

[emiln]

The many distros with neither wget nor curl makes it infeasible to provide a fully dependency-free tool, I fear. On the bright side, this is already clear from the README:

In order to run, nvi assumes the following software is available:

• GNU coreutils
• Other GNU software: bash, grep, tar, tput, wget

I think for now we'll just ensure this list is kept up to date. It seems likely that people know how to satisfy these dependencies and whether or not they make nvi undesirable. I think the best approach we can possibly take towards making nvi useful is to be true to its very specific purpose:

A command-line tool to make Node.js binaries available in CI environments.

Perhaps this needs rephrasing at some point, but irt. this issue I think we should keep in mind that:

1. The quoted section in your original issue description needs rephrasing; it must be clear to users how to use nvi in CI environments.
2. We call nvi a Node.js installer. I don't want to broaden the scope to a Node.js compiler. This effectively rejects alpine support and some of the more cumbersome ideas we touched upon here.
3. We should make Node.js in CI environments an easy thing to achieve, which does not imply full compatibility with all possible out-of-the-box base images out there. I think we need to list the popular base images that nvi will just not work in, like alpine. I don't find it problematic that users need to use their package manager to install wget or the likes. Let's treat these nvi-hostile environments as special cases for now and just list them in the README.

I think the actions spawned by the discussion are few, then.

Update the README to clearly communicate 1., 2., and 3. above.

Does that sound reasonable?

[commonquail]

Yes. Although, curl might be a better default than wget, simply because it's already in centos.

[emiln]

My initial reason for depending on wget rather than curl was the conceptually better fit:

From the wget project page:

GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols.

From the curl project page:

A command line tool and library for transferring data with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, GOPHER, TFTP, SCP, SFTP, TELNET, DICT, LDAP, LDAPS, FILE, IMAP, SMTP, POP3, RTSP and RTMP. libcurl offers a myriad of powerful features

In the end I'm interested in utility over purity, however. We could even consider having nvi work when either of them is installed. I don't think that'd amount to a significant increase in complexity.

[commonquail]

This works:

    if hash wget 2>/dev/null
    then
      downloader=(wget --quiet --output-document)
    else
      downloader=(curl --silent --location --output)
    fi

    if ! "${downloader[@]}" "$DOWNLOAD_DEST" "$NODE_TAR"

[emiln]

That looks find to me, but we should probably move it to a new issue.