Lambda Invalid ELF Header error using argon2 due to installed binaries with incompatible platform bindings (macos vs amazonlinux)

[theseyi]

This is a Bug Report

Description

Nodejs application uses argon2 as a dependency, which is installed via yarn add argon2. argon2 has native module binaries with bindings to the installed os. In this case, macOS and deployed to Lambda which is amazonlinux, so the bindings are incompatible. I've looked around and the typical way is to deploy using a Docker image or a vm that closely matches the target, or as part of a CI/CD pipeline. However, for local dev and quick prototyping it's preferable and possible to build for a different target than the installed environment and package and push to . Unfortunately, the provided solutions with serverless do not seem to be working as expected.

What went wrong?

• The eventual serverless-webpack package still contains an argon2 build for macOS.
• The package step, even when running with the --verbose flag does not give feedback about which scripts were run, unless they fail
• invalid ELF header error gets thrown on lambda

What did you expect should have happened?

The package to contain the argon2 module with bindings for linux platform

What was the config you used?

    webpack: {
      webpackConfig: './webpack.config.js',
      includeModules: true,
      keepOutputDirectory: true,
      packagerOptions: {
        scripts: [
          'npm rebuild argon2 --target_arch=x64 --target_platform=linux --update-binary',
          // Some other scripts I attempted:
          // 'rm -rf node_modules/argon2 && npm install argon2 --target_arch=x64 --target_platform=linux --update-binary',
          // 'if test "$DEPLOY_FROM_MAC"=true ; then npm rebuild  argon2 --arch="x64" --target_arch="x64" --target_platform="linux" ; fi ',
        ],
      },
    },

What stacktrace or error message from your provider did you see?

Eventual package deployed to Lambda still has bindings for macOS causing an invalid ELF header to get thrown, see stack trace snippet below

Similar or dependent issue(s):

• Docker-Lambci Packaging of external native modules #402
• #342
• Unable to import module when using sqlite3 #416

Additional Data

• Serverless-Webpack Version you're using:
  serverless-webpack@npm:5.3.5
• Webpack version you're using:
  webpack@npm:5.13.0
• Serverless Framework Version you're using:
  serverless@npm:2.18.0
• Operating System:

ProductName:	macOS
ProductVersion:	11.1
BuildVersion:	        20C69

• Stack Trace (if available):

{
    "errorType": "Error",
    "errorMessage": "/var/task/node_modules/argon2/lib/binding/napi-v3/argon2.node: invalid ELF header",
    "stack": [
        "Error: /var/task/node_modules/argon2/lib/binding/napi-v3/argon2.node: invalid ELF header",
        "    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1065:18)",
        "    at Module.load (internal/modules/cjs/loader.js:879:32)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:724:14)",
        "    at Module.require (internal/modules/cjs/loader.js:903:19)",
        "    at require (internal/modules/cjs/helpers.js:74:18)",
        "    at Object.<anonymous> (/var/task/node_modules/argon2/argon2.js:9:56)",
        "    at Module._compile (internal/modules/cjs/loader.js:1015:30)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1035:10)",
        "    at Module.load (internal/modules/cjs/loader.js:879:32)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:724:14)"
    ]
}

[theseyi]

For posterity:
My config was missing an additional flag indicating the target libc library. Running the rebuild command as above was failing because node-pre-gyp was fetching the prebuilt binary with an incorrect url resulting in a 404 and a fallback to building locally

Tried to download(404): https://github.com/ranisalt/node-argon2/releases/download/v0.27.1/argon2-v0.27.1-napi-v3-linux-x64-unknown.tar.gz

the problem here was unknown.tar.gz which doesn't exist for the linux platform

However, adding --target_libc=glibc allows it make the correct reference for argon2-v0.27.1-napi-v3-linux-x64-glibc.tar.gz et voilà!