Package binary case of Toolchain::install

[dherman]

Implement the Tool::PackageBinary case of toolchain::install(tool: Tool):

• fetch package metadata
• fetch tarball to ~/.notion/tools/inventory/packages/$package/
• unpack tarball to ~/.notion/tools/image/packages/$package/
• determine its platform from (package.json || user default)
• build tarball with platform
• create ~/.notion/tools/user/installed/$package/package.toml
• create symlinks in ~/.notion/tools/user/bins/ for each tool in the package to point to ~/.notion/tools/user/installed/$package/
• create shims as hard links to ~/.notion/exe/shim

May need to be implemented simultaneously with #176.

Depends on:

• Staging area in directory layout #172 Staging area in directory layout
• Tool specs #173 Tool specs
• Generalize Toolchain::install #174 Generalize Toolchain::install

[charlespierce]

For the sub-task "build tarball with platform", how do we generalize the "build" operation for any package? Aren't we essentially re-creating npm or yarn at that point, since there are a bunch of hooks and specific operations that go into preparing a package for use?

Another option would be to use npm or yarn to do a "global" install, and then package it up from there, but then we lose control over exactly where everything goes, so that's not ideal either.

[charlespierce]

Also worth noting that on Windows, making a symlink to the "built" file might not be enough, since the executable might just be JS with a #!/usr/bin/env node at the beginning, which won't work on Windows.

There's a package that npm and yarn use to create fake symlinks that are actually .cmd files that call the script correctly by parsing the shebang header. This is all handled internally as part npm install or yarn add (And the global variants), so this make me lean towards defaulting to using one of those implementations to do an installation.

[dherman]

Those are great questions. The flip issue is that if we delegate to an actual global install, we still need a way to make the global package invisible to projects, in particular to make sure its binaries don't end up on the PATH when you run commands inside a project.

[dherman]

There's one part where I'm not sure if we're talking past each other: I didn't ever expect that we'd reimplement the build process. I meant we'd delegate to the package manager to do the build.

What I had in mind was something like:

• fetch the package either from the internet or from the inventory if it's already there
• unpack it off to the side instead of doing a global install
• run npm i or yarn within that side directory to build all its binaries
• then the shims would delegate to the corresponding binaries in its .bin directory

This isn't duplicating most of the functionality of the package manager, but it does duplicate the part that wires up the global binaries. Does that include the fake symlink stuff you're talking about?

Maybe another possibility is to do the global install and then manually "uninstall" it by moving the results off to the side?

[charlespierce]

I think I see where my misunderstanding was. I’m so used to thinking about working with packages from source, where unpacking and running npm i wouldn’t be enough to have a working tool.

But of course we’re not downloading the source, we’re downloading the bundled npm package, which will be ready to go if we just run npm install or yarn, since that will ensure all the dependencies are available and run any postinstall hooks.

The point about fake symlinks may still be needed, however, since the binaries will be in whatever state the package author wanted them in, so we’ll have to figure out how to call them from the shims.

Also I don’t think the top-level binaries will be put into a .bin folder, since they’ll only be referenced by the top-level package.json