[bug] The new json-logic-js dependency from node-zwave-js isn't being picked up in a custom docker image #669

rccoleman · 2021-02-21T05:05:10Z

Before submitting a bug please read: https://zwave-js.github.io/zwavejs2mqtt/#/troubleshooting/bug_report

Version

Build/Run method

Docker
PKG
Manually built (git clone - npm install - npm run build )

Manually built docker image as per instructions at https://zwave-js.github.io/zwavejs2mqtt/#/troubleshooting/docker_custom_builds

zwavejs2mqtt version: 1.2.2
zwavejs version: 6.4.0

Describe the bug

Pulling the latest "master" branch of node-zwave-js and zwavejs2mqtt and building based on the instructions at https://zwave-js.github.io/zwavejs2mqtt/#/troubleshooting/docker_custom_builds yields a container that constantly restarts with the following error in the docker logs:

Error: Cannot find module 'json-logic-js'
Require stack:
- /usr/src/app/node_modules/@zwave-js/config/build/Logic.js
- /usr/src/app/node_modules/@zwave-js/config/build/Devices.js
- /usr/src/app/node_modules/@zwave-js/config/build/ConfigManager.js
- /usr/src/app/node_modules/@zwave-js/config/build/index.js
- /usr/src/app/node_modules/zwave-js/build/lib/commandclass/MultilevelSensorCC.js
- /usr/src/app/node_modules/zwave-js/build/lib/commandclass/index.js
- /usr/src/app/node_modules/zwave-js/CommandClass.js
- /usr/src/app/node_modules/zwave-js/index.js
- /usr/src/app/lib/ZwaveClient.js
- /usr/src/app/node_modules/app-root-path/lib/app-root-path.js
- /usr/src/app/node_modules/app-root-path/index.js
- /usr/src/app/bin/www
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
    at Function.Module._load (internal/modules/cjs/loader.js:667:27)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at require (internal/modules/cjs/helpers.js:74:18)
    at Object.<anonymous> (/usr/src/app/node_modules/@zwave-js/config/src/Logic.ts:1:1)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Module.require (internal/modules/cjs/loader.js:887:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/usr/src/app/node_modules/@zwave-js/config/build/Logic.js',
    '/usr/src/app/node_modules/@zwave-js/config/build/Devices.js',
    '/usr/src/app/node_modules/@zwave-js/config/build/ConfigManager.js',
    '/usr/src/app/node_modules/@zwave-js/config/build/index.js',
    '/usr/src/app/node_modules/zwave-js/build/lib/commandclass/MultilevelSensorCC.js',
    '/usr/src/app/node_modules/zwave-js/build/lib/commandclass/index.js',
    '/usr/src/app/node_modules/zwave-js/CommandClass.js',
    '/usr/src/app/node_modules/zwave-js/index.js',
    '/usr/src/app/lib/ZwaveClient.js',
    '/usr/src/app/node_modules/app-root-path/lib/app-root-path.js',
    '/usr/src/app/node_modules/app-root-path/index.js',
    '/usr/src/app/bin/www'
  ]
}

I suspect that this is related to the recent addition of conditional logic in the JSON config files, but I can't figure out what's missing.

To Reproduce

Steps to reproduce the behavior:

Build a custom container based on the master branches and the instructions above and attempt to deploy it.

Expected behavior

The container should start and run properly.

Additional context

The dependabot also broke Dockerfile.contrib again by trying to move it to Alpine from Debian Buster. This was the fix for the last time this happend: b048fcb#diff-4ddc2e10398d5c169795b7937f893ec72d2619b395259c592156a5dd0b476c9f

The text was updated successfully, but these errors were encountered:

scyto · 2021-02-21T05:17:26Z

this fixes the alpine issue, but not the other issues #666

AlCalzone · 2021-02-21T23:32:55Z

Does this also happen if you run yarn install after pulling the latest changes from Github?

rccoleman · 2021-02-21T23:54:21Z

Yes, it does.

robertsLando · 2021-02-22T09:31:31Z

@AlCalzone ?

AlCalzone · 2021-02-22T10:00:49Z

I'm honestly confused here. json-logic-js is a dependency of @zwave-js/config, there is no package-lock.json that says otherwise and json-logic-js is included in yarn.lock.

rccoleman · 2021-02-22T17:58:06Z

Are either of you able to repro it? It was just a matter of following the instructions for generating the container build, with the slight complication that dependabot keeps breaking Dockerfile.contrib and needs to be manually fixed. I'll run through a clean install later today and capture the results. I'm certainly open to the possibility that I'm doing something wrong, but others are reporting exactly the same failure. In my case, simply running a test container based on the built image shows that that package isn't present in node_modules.

AlCalzone · 2021-02-22T18:56:49Z

Yeah, I can reproduce

rccoleman · 2021-02-22T19:08:39Z

That's good, I guess. I'm now getting a new error when I create a test dir and clone master of both repos from scratch:

#14 27.04 [3/4] Linking dependencies...
#14 27.05 warning " > @xstate/[email protected]" has unmet peer dependency "xstate@^4.9.0".
#14 27.05 warning "@xstate/test > @xstate/[email protected]" has unmet peer dependency "xstate@^4.9.0".
#14 27.06 warning "lerna > @lerna/version > @lerna/github-client > @octokit/rest > @octokit/[email protected]" has unmet peer dependency "@octokit/core@>=3".
#14 39.11 [4/4] Building fresh packages...
#14 39.73 error /home/node/node-zwave-js/node_modules/node-expat: Command failed.
#14 39.73 Exit code: 1
#14 39.73 Command: node-gyp rebuild
#14 39.73 Arguments:
#14 39.73 Directory: /home/node/node-zwave-js/node_modules/node-expat
#14 39.73 Output:
#14 39.73 gyp info it worked if it ends with ok
#14 39.73 gyp info using [email protected]
#14 39.73 gyp info using [email protected] | linux | x64
#14 39.73 gyp ERR! find Python
#14 39.73 gyp ERR! find Python Python is not set from command line or npm configuration
#14 39.73 gyp ERR! find Python Python is not set from environment variable PYTHON
#14 39.73 gyp ERR! find Python checking if "python" can be used
#14 39.73 gyp ERR! find Python - "python" is not in PATH or produced an error
#14 39.73 gyp ERR! find Python checking if "python2" can be used
#14 39.73 gyp ERR! find Python - "python2" is not in PATH or produced an error
#14 39.73 gyp ERR! find Python checking if "python3" can be used
#14 39.73 gyp ERR! find Python - "python3" is not in PATH or produced an error
#14 39.73 gyp ERR! find Python
#14 39.73 gyp ERR! find Python **********************************************************
#14 39.73 gyp ERR! find Python You need to install the latest version of Python.
#14 39.73 gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
#14 39.73 gyp ERR! find Python you can try one of the following options:
#14 39.73 gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
#14 39.73 gyp ERR! find Python   (accepted by both node-gyp and npm)
#14 39.73 gyp ERR! find Python - Set the environment variable PYTHON
#14 39.73 gyp ERR! find Python - Set the npm configuration variable python:
#14 39.73 gyp ERR! find Python   npm config set python "/path/to/pythonexecutable"
#14 39.73 gyp ERR! find Python For more information consult the documentation at:
#14 39.73 gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
#14 39.73 gyp ERR! find Python **********************************************************
#14 39.73 gyp ERR! find Python
#14 39.73 gyp ERR! configure error
#14 39.73 gyp ERR! stack Error: Could not find any Python installation to use
#14 39.73 gyp ERR! stack     at PythonFinder.fail (/home/node/node-zwave-js/node_modules/node-gyp/lib/find-python.js:307:47)
#14 39.73 gyp ERR! stack     at PythonFinder.runChecks (/home/node/node-zwave-js/node_modules/node-gyp/lib/find-python.js:136:21)
#14 39.73 gyp ERR! stack     at PythonFinder.<anonymous> (/home/node/node-zwave-js/node_modules/node-gyp/lib/find-python.js:179:16)
#14 39.73 gyp ERR! stack     at PythonFinder.execFileCallback (/home/node/node-zwave-js/node_modules/node-gyp/lib/find-python.js:271:16)
#14 39.73 gyp ERR! stack     at exithandler (child_process.js:315:5)
#14 39.73 gyp ERR! stack     at ChildProcess.errorhandler (child_process.js:327:5)
#14 39.73 gyp ERR! stack     at ChildProcess.emit (events.js:314:20)
#14 39.73 gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:274:12)
#14 39.73 gyp ERR! stack     at onErrorNT (internal/child_process.js:470:16)
#14 39.73 gyp ERR! stack     at processTicksAndRejections (internal/process/task_queues.js:84:21)
#14 39.73 gyp ERR! System Linux 5.4.0-65-generic
#14 39.73 gyp ERR! command "/usr/local/bin/node" "/home/node/node-zwave-js/node_modules/.bin/node-gyp" "rebuild"
#14 39.73 gyp ERR! cwd /home/node/node-zwave-js/node_modules/node-expat
#14 39.73 gyp ERR! node -v v12.20.2
#14 39.73 gyp ERR! node-gyp -v v5.1.1
#14 39.73 gyp ERR! not ok
#14 39.73 info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
------
executor failed running [/bin/sh -c yarn install --network-timeout=${YARN_NETWORK_TIMEOUT}]: exit code: 1

Looks like that showed up with the move from erbium-buster to erbium-buster-slim originally from here: f4ff24a. I switched back and at least the container build succeeds.

Same issue, though. Shellling in with docker run -it zwavejs2mqtt sh and doing node app.js produces the same json-logic-js error, and it's not present in node_modules.

scyto · 2021-02-22T19:16:31Z

i built a repro this morning using buster-slim, image is at scyto/zwavejs2mqtt:patch-2 if anyone wants it.
~~also building with buster as the build i was testing a couple of nights ago with buster appeared not to have the issue, but not sure.~~

AlCalzone · 2021-02-22T19:31:22Z

I have two theories why this is happening:

a) The problem has to do with the version magic the dockerfile does. This is the end of the npm ls command inside the container @scyto built:

`-- UNMET PEER DEPENDENCY [email protected] invalid
  +-- @alcalzone/[email protected] extraneous
  +-- @sentry/[email protected] extraneous
  +-- @sentry/[email protected] extraneous
  +-- UNMET DEPENDENCY @zwave-js/config@^6.4.0
  +-- UNMET DEPENDENCY @zwave-js/core@^6.4.0
  +-- UNMET DEPENDENCY @zwave-js/serial@^6.4.0
  +-- UNMET DEPENDENCY @zwave-js/shared@^6.2.0
  +-- [email protected] extraneous
  +-- [email protected]
  +-- [email protected] extraneous
  +-- [email protected] extraneous
  +-- [email protected] deduped
  +-- [email protected]
  | +-- [email protected]
  | `-- [email protected] deduped
  +-- [email protected] deduped
  `-- [email protected] extraneous

npm ERR! peer dep missing: zwave-js@^6.4, required by @zwave-js/[email protected]
npm ERR! extraneous: @zwave-js/[email protected] /usr/src/app/node_modules/@zwave-js/config
npm ERR! extraneous: @zwave-js/[email protected] /usr/src/app/node_modules/@zwave-js/core
npm ERR! extraneous: @zwave-js/[email protected] /usr/src/app/node_modules/@zwave-js/serial
npm ERR! extraneous: @zwave-js/[email protected] /usr/src/app/node_modules/@zwave-js/shared

According to this theory, the local version has been changed to the prerelease version [email protected], but zwavejs2mqtt or the server wants zwave-js@^6.4, which [email protected] does not satisfy.

b) zwave-js is a peer dependency. Not sure what this implies but it could be a problem.

I'm investigating a) now.

rccoleman · 2021-02-22T19:33:31Z

(a) looks like it may be an artifact of the custom build. All version numbers get (I think) the commit number added to them, and that shows up in the UI. I'm currently running a released container, but here's the version info from a previous run with a custom container:

App Version 1.1.1+2ea9944.2ea9944
Zwavejs Version 6.4.0-ad7995f8

AlCalzone · 2021-02-22T19:59:33Z

I've removed the version magic and it still fails.

scyto · 2021-02-24T01:51:37Z

are these still failing for you?
my github actions build using dockerfile.contrib just worked and gave me a working build with

App Version 1.3.0+fec72b7.fec72b7
Zwavejs Version 6.5.0-e0b52b1
Zwavejs-server Version 1.0.0-beta.9

rccoleman · 2021-02-24T01:59:47Z

I'm getting the same python error with the latest pull from both repos that I posted here. I was able to fix it before by not using the -slim image.

If I apply this patch, the build is successful and I do see json-logic-js in node_modules and I can run node app.js from the shell without an error now.

diff --git a/docker/Dockerfile.contrib b/docker/Dockerfile.contrib
index 4e1b9c8..236d2f1 100644
--- a/docker/Dockerfile.contrib
+++ b/docker/Dockerfile.contrib
@@ -13,7 +13,7 @@ RUN git clone -b ${ZWJ_BRANCH} --depth 1 https://github.com/zwave-js/node-zwave-
 RUN git clone -b ${Z2M_BRANCH} --depth 1 https://github.com/zwave-js/zwavejs2mqtt

 ##### LOCAL SOURCE #####
-FROM node:erbium-buster-slim AS local-copy-src
+FROM node:erbium-buster AS local-copy-src
 COPY --chown=node node-zwave-js /home/node/node-zwave-js
 COPY --chown=node zwavejs2mqtt /home/node/zwavejs2mqtt

And the image appears to work now. It's still going through the initial query, but I don't see the constant restarts with the missing module error anymore.

@AlCalzone

Fixes #669 cc @AlCalzone

@AlCalzone

* chore: fix brokern Dockerfile.contrib Fixes #669 cc @AlCalzone * fix: restore from image * fix: disable automerge of docker bumps

rccoleman added the bug Something isn't working label Feb 21, 2021

rccoleman assigned robertsLando Feb 21, 2021

rccoleman changed the title ~~[bug] Dockerfile.contrib is broken again (incorrectly moved to Alpine) and the new json-logic-js isn't being picked up~~ [bug] The new json-logic-js dependency from node-zwave-js isn't being picked up in a custom docker image Feb 21, 2021

robertsLando added a commit that referenced this issue Feb 24, 2021

chore: fix brokern Dockerfile.contrib

01ae974

Fixes #669 cc @AlCalzone

robertsLando mentioned this issue Feb 24, 2021

chore: fix broken Dockerfile.contrib #726

Merged

scyto mentioned this issue Feb 24, 2021

Error in driver ZWaveError: Failed to initialize the driver: core_1.shouldLogNode is not a function zwave-js/node-zwave-js#1881

Closed

18 tasks

robertsLando closed this as completed in #726 Feb 24, 2021

robertsLando added a commit that referenced this issue Feb 24, 2021

chore: fix broken Dockerfile.contrib (#726)

906f932

* chore: fix brokern Dockerfile.contrib Fixes #669 cc @AlCalzone * fix: restore from image * fix: disable automerge of docker bumps

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bug] The new json-logic-js dependency from node-zwave-js isn't being picked up in a custom docker image #669

[bug] The new json-logic-js dependency from node-zwave-js isn't being picked up in a custom docker image #669

rccoleman commented Feb 21, 2021 •

edited

Loading

scyto commented Feb 21, 2021 •

edited

Loading

AlCalzone commented Feb 21, 2021

rccoleman commented Feb 21, 2021

robertsLando commented Feb 22, 2021

AlCalzone commented Feb 22, 2021

rccoleman commented Feb 22, 2021 •

edited

Loading

AlCalzone commented Feb 22, 2021

rccoleman commented Feb 22, 2021 •

edited

Loading

scyto commented Feb 22, 2021 •

edited

Loading

AlCalzone commented Feb 22, 2021

rccoleman commented Feb 22, 2021 •

edited

Loading

AlCalzone commented Feb 22, 2021

scyto commented Feb 24, 2021

rccoleman commented Feb 24, 2021 •

edited

Loading

[bug] The new json-logic-js dependency from node-zwave-js isn't being picked up in a custom docker image #669

[bug] The new json-logic-js dependency from node-zwave-js isn't being picked up in a custom docker image #669

Comments

rccoleman commented Feb 21, 2021 • edited Loading

Version

Describe the bug

To Reproduce

Expected behavior

Additional context

scyto commented Feb 21, 2021 • edited Loading

AlCalzone commented Feb 21, 2021

rccoleman commented Feb 21, 2021

robertsLando commented Feb 22, 2021

AlCalzone commented Feb 22, 2021

rccoleman commented Feb 22, 2021 • edited Loading

AlCalzone commented Feb 22, 2021

rccoleman commented Feb 22, 2021 • edited Loading

scyto commented Feb 22, 2021 • edited Loading

AlCalzone commented Feb 22, 2021

rccoleman commented Feb 22, 2021 • edited Loading

AlCalzone commented Feb 22, 2021

scyto commented Feb 24, 2021

rccoleman commented Feb 24, 2021 • edited Loading

rccoleman commented Feb 21, 2021 •

edited

Loading

scyto commented Feb 21, 2021 •

edited

Loading

rccoleman commented Feb 22, 2021 •

edited

Loading

rccoleman commented Feb 22, 2021 •

edited

Loading

scyto commented Feb 22, 2021 •

edited

Loading

rccoleman commented Feb 22, 2021 •

edited

Loading

rccoleman commented Feb 24, 2021 •

edited

Loading