0.33.0 - 'npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc' installs both glibc and musl libs #3873

tmcgannon · 2023-11-29T23:34:51Z

Possible install-time or require-time problem

I have read the documentation relating to installation.

Are you using the latest version of sharp?

I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

If you cannot confirm this, please upgrade to the latest version and try again before opening an issue.

If you are using another package which depends on a version of sharp that is not the latest, please open an issue against that package instead.

Are you using a supported runtime?

I am using Node.js 18 with a version >= 18.17.0
I am using Node.js 20 with a version >= 20.3.0
I am using Node.js 21 or later

If you cannot confirm any of these, please upgrade to the latest version and try again before opening an issue.

Are you using a supported package manager?

I am using npm >= 9.6.5
I am using yarn >= 3.2.0
I am using pnpm >= 7.1.0

If you cannot confirm any of these, please upgrade to the latest version and try again before opening an issue.

What is the complete output of running `npm install --verbose --foreground-scripts sharp` in an empty directory?

npm verb cli /usr/local/bin/node /usr/local/bin/npm
npm info using [email protected]
npm info using [email protected]
npm verb title npm install sharp
npm verb argv "install" "--loglevel" "verbose" "--foreground-scripts" "sharp"
npm verb logfile logs-max:10 dir:/Users/tmcgannon/.npm/_logs/2023-11-29T22_24_45_285Z-
npm verb logfile /Users/tmcgannon/.npm/_logs/2023-11-29T22_24_45_285Z-debug-0.log
npm http fetch GET 200 https://registry.npmjs.org/sharp 182ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/semver 47ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/detect-libc 126ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color 134ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-darwin-x64 354ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linuxmusl-x64 369ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-arm 450ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-wasm32 465ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-darwin-arm64 495ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-s390x 609ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-x64 611ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linuxmusl-arm64 611ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linuxmusl-x64 610ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-win32-x64 610ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-darwin-x64 613ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-arm 613ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-win32-ia32 653ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linuxmusl-arm64 662ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-darwin-arm64 666ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-s390x 667ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-x64 669ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-linux-arm64 708ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@img%2fsharp-libvips-linux-arm64 765ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color-convert 50ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@emnapi%2fruntime 266ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/color-name 44ms (cache revalidated)
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-win32-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-win32-ia32
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-wasm32
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@emnapi/runtime
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linuxmusl-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linuxmusl-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-s390x
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-linux-arm
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linuxmusl-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linuxmusl-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-s390x
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-arm64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-linux-arm
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-libvips-darwin-x64
npm verb reify failed optional dependency /Users/tmcgannon/node_modules/@img/sharp-darwin-x64
npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 89ms
npm info run [email protected] install node_modules/sharp node install/check

> [email protected] install
> node install/check

npm http fetch GET 200 https://registry.npmjs.org/xml2js 48ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/axios 59ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/tough-cookie 60ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/jsonwebtoken 62ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/dottie 106ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/@azure%2fms-rest-js 67ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/azurite 91ms (cache revalidated)
npm http fetch GET 200 https://registry.npmjs.org/sequelize 168ms (cache revalidated)
npm info run [email protected] install { code: 0, signal: null }

added 8 packages, and audited 297 packages in 4s

58 packages are looking for funding
  run `npm fund` for details

8 vulnerabilities (7 moderate, 1 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

What is the output of running `npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp`?

➜  test npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp

  System:
    OS: macOS 14.1
    CPU: (8) arm64 Apple M1
    Memory: 83.52 MB / 8.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.10.0 - /usr/local/bin/node
    npm: 10.2.3 - /usr/local/bin/npm
    bun: 1.0.14 - ~/.bun/bin/bun

Problem description

When I run npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc, it installs both glibc and musl versioned libraries:

Expected behavior

Only sharp-libvips-linux-x64 and sharp-linux-x64 should be installed.

The text was updated successfully, but these errors were encountered:

lovell · 2023-11-30T07:59:53Z

Please see npm/cli#6914

adriaanmeuris · 2023-11-30T11:17:48Z

I'm probably missing something, but I'm seeing the same thing happening when using either pnpm or yarn which do allow to set the supported architectures.

Example package.json file

{
  "name": "sharp-binaries",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "4.18.2",
    "sharp": "0.33.0"
  },
  "os": ["linux"],
  "cpu": ["x64"],
  "libc": ["glibc"],
  "pnpm": {
    "supportedArchitectures": {
      "os": ["linux"],
      "cpu": ["x64"],
      "libc": ["glibc"]
    }
  }
}

PNPM

Uses the pnpm property in package.json to setup supported architectures as defined here.

Dockerfile

FROM node:18.18-alpine3.18
RUN corepack enable && corepack prepare pnpm@latest --activate
COPY package*.json ./
RUN pnpm i
COPY index.js ./
CMD ["node", "index.js"]

Build targetting linux/amd64

docker build . --no-cache --platform=linux/amd64 --tag 'sharp-binaries:latest'

Results
I'm seeing 4 packages (using Docker desktop) where I'd expect to see 2 (sharp-libvips-linux-x64 and sharp-linux-x64):

Yarn

Uses os, cpu and libc properties in package.json as described here.

Dockerfile

FROM node:18.18-alpine3.18
COPY package*.json ./
RUN yarn install
COPY index.js ./
CMD ["node", "index.js"]

Build targetting linux/amd64

docker build . --no-cache --platform=linux/amd64 --tag 'sharp-binaries:latest'

Results
I'm seeing 19 packages where I'd expect to see 2 (sharp-libvips-linux-x64 and sharp-linux-x64)

lovell · 2023-11-30T11:47:58Z

@adriaanmeuris For pnpm, please see pnpm/pnpm#7214 - if the latest version of pnpm is installing the wrong dependencies, then please ask at the pnpm repo.

For yarn, it looks like you're using version 1.

$ docker run --rm -it node:18.18-alpine3.18 yarn --version
1.22.19

Please use at least version 3 of yarn, but avoid its "plug'n'play" linker.

Version 1 of yarn is no longer actively developed and is missing so many useful newer features. See also evanw/esbuild#2949

adriaanmeuris · 2023-11-30T14:28:17Z

Thanks @lovell, I can confirm this works with yarn@3 without PnP linker.

For pnpm it indeed seems like an issue with installing the right dependencies as mentioned here.

tmcgannon · 2023-11-30T19:37:05Z

Thanks, I will track the npm issue.

klausbadelt · 2024-01-17T17:17:38Z

For whoever ends up here - the --libc npm feature is now merged npm/cli/pull/6914

tmcgannon added the installation label Nov 29, 2023

tmcgannon changed the title ~~0.33.0 - 'npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc' installs both glibc and muse~~ 0.33.0 - 'npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc' installs both glibc and musl Nov 29, 2023

tmcgannon changed the title ~~0.33.0 - 'npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc' installs both glibc and musl~~ 0.33.0 - 'npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc' installs both glibc and musl libs Nov 29, 2023

adriaanmeuris mentioned this issue Nov 30, 2023

pnpm install multiple bindings of different libc pnpm/pnpm#7362

Closed

4 tasks

tmcgannon closed this as completed Nov 30, 2023

pH200 mentioned this issue Jan 4, 2024

Bug. Both linux & linuxmusl are installed pH200/sharp-layer#2

Closed

klausbadelt mentioned this issue Jan 8, 2024

feat: add --libc option to override platform specific install npm/cli#6914

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

0.33.0 - 'npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc' installs both glibc and musl libs #3873

0.33.0 - 'npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc' installs both glibc and musl libs #3873

tmcgannon commented Nov 29, 2023

lovell commented Nov 30, 2023

adriaanmeuris commented Nov 30, 2023

lovell commented Nov 30, 2023

adriaanmeuris commented Nov 30, 2023

tmcgannon commented Nov 30, 2023

klausbadelt commented Jan 17, 2024

0.33.0 - 'npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc' installs both glibc and musl libs #3873

0.33.0 - 'npm ci --omit=dev --cpu=x64 --os=linux --libc=glibc' installs both glibc and musl libs #3873

Comments

tmcgannon commented Nov 29, 2023

Possible install-time or require-time problem

Are you using the latest version of sharp?

Are you using a supported runtime?

Are you using a supported package manager?

What is the complete output of running npm install --verbose --foreground-scripts sharp in an empty directory?

What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?

Problem description

Expected behavior

lovell commented Nov 30, 2023

adriaanmeuris commented Nov 30, 2023

PNPM

Yarn

lovell commented Nov 30, 2023

adriaanmeuris commented Nov 30, 2023

tmcgannon commented Nov 30, 2023

klausbadelt commented Jan 17, 2024

What is the complete output of running `npm install --verbose --foreground-scripts sharp` in an empty directory?

What is the output of running `npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp`?