Building Jupyterlab plugin from source does not pull working directory changes

[sc1f]

Previously, when I built the perspective-jupyterlab plugin, it would pull in changes from other packages in Perspective—thus I could make a change in perspective-viewer and see it reflected in Jupyterlab, etc.

Since #1231, building the Jupyterlab plugin no longer pulls in changes from the local working directory, an issue which I fixed here as part of #1289, before the Jlab 3 PR (#1294) was merged. The issue with it not resolving working directory changes was due to externals: [/^[a-z0-9@]/], which meant that perspective was not being bundled into the Jupyterlab plugin, and so the plugin could only pull the disted version of Perspective (assuming my understanding of Webpack in this case is correct).

I tried to implement the same fix on top of #1294, but jupyter lab build fails with this error:

[webpack-cli] ModuleNotFoundError: Module not found: Error: Can't resolve 'fs' in '/Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/@finos
/perspective-jupyterlab/dist'
    at /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/webpack/lib/Compilation.js:1570:28
    at /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/webpack/lib/NormalModuleFactory.js:648:13
    at eval (eval at create (/Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:8:1)
    at /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/webpack/lib/NormalModuleFactory.js:233:22
    at eval (eval at create (/Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:7:1)
    at /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/webpack/lib/NormalModuleFactory.js:357:22
    at /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/webpack/lib/NormalModuleFactory.js:116:11
    at /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/webpack/lib/NormalModuleFactory.js:577:24
    at /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/webpack/lib/NormalModuleFactory.js:721:5
    at finishWithoutResolve (/Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/Resolver.js:296:11)
resolve 'fs' in '/Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/@finos/perspective-jupyterlab/dist'
  Parsed request is a module
  using description file: /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/@finos/perspective-jupyterlab/package.json (relative path: ./dist)
    Field 'browser' doesn't contain a valid alias configuration
    resolve as module
      /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/@finos/perspective-jupyterlab/dist/node_modules doesn't exist or is not a directory
      /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/@finos/perspective-jupyterlab/node_modules doesn't exist or is not a directory
      /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/@finos/node_modules doesn't exist or is not a directory
      /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/node_modules doesn't exist or is not a directory
      looking for modules in /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules
        single file module
          using description file: /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/package.json (relative path: ./node_modules/fs)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/fs doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
              /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/fs.js doesn't exist
            .json
              Field 'browser' doesn't contain a valid alias configuration
              /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/fs.json doesn't exist
            .wasm
              Field 'browser' doesn't contain a valid alias configuration
              /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/fs.wasm doesn't exist
        /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/staging/node_modules/fs doesn't exist
      /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/lab/node_modules doesn't exist or is not a directory
      /Library/Frameworks/Python.framework/Versions/3.7/share/jupyter/node_modules doesn't exist or is not a directory
      /Library/Frameworks/Python.framework/Versions/3.7/share/node_modules doesn't exist or is not a directory
      /Library/Frameworks/Python.framework/Versions/3.7/node_modules doesn't exist or is not a directory
      /Library/Frameworks/Python.framework/Versions/node_modules doesn't exist or is not a directory
      /Library/Frameworks/Python.framework/node_modules doesn't exist or is not a directory
      /Library/Frameworks/node_modules doesn't exist or is not a directory
      /Library/node_modules doesn't exist or is not a directory
      /node_modules doesn't exist or is not a directory
error Command failed with exit code 2.

I have added node: {fs: "empty"} to plugin.config.js and the same issue happens, and we are definitely not importing 'fs' anywhere in our disted assets - @timkpaine not sure if you have insight into this issue/have a good direction to look.

[rvillalba-novetta]

Not sure if the same thing. But I'm unable to install the jupyter labextension because I get a similar can't resolve fs error

[sc1f]

Not sure if the same thing. But I'm unable to install the jupyter labextension because I get a similar can't resolve fs error

Are you installing the extension from NPM, i.e. jupyter labextension install @finos/perspective-jupyterlab, or from your local filesystem? I am assuming it is the former but this information would be helpful in narrowing down the issue. I was under the impression this should only be an issue if building the plugin from a local working directory.

[rvillalba-novetta]

Yeah I'm on jupyterlab 3.0.5 doing jupyter labextension install @finos/perspective-jupyterlab. And I get
[webpack-cli] ModuleNotFoundError: Module not found: Error: Can't resolve 'fs' in '.../anaconda3/envs/test/share/jupyter/lab/staging/node_modules/@finos/perspective-jupyterlab/dist'

[timkpaine]

This was broken prior to the release of 0.6.0, but your install wouldn't work anyway as we didn't support jupyterlab 3 until after 0.6.0.

[rvillalba-novetta]

Could you clarify for me. Shouldn't 0.6.0 work for jupyterlab 3? If so, those are the versions I'm trying to install. Are these not the same issue?

[sc1f]

The next version we dist will be compatible for Jupyterlab 3 - 0.6.0 remains compatible with Jupyterlab 1 and 2 only. The issue I'm working on refers to changes from local working directories not picked up during the development build process, which does not affect whether the plugin will build on Jlab 3 once the new compatible version is disted. Sorry if there was any confusion!