Make dependencies between files explicit via require/import

[dritchie]

I frequently find myself wanting to include various bits of existing deterministic Javascript code in my WebPPL programs: for example, THREE.js, or some miscellaneous utility functions only relevant to my current experiment. It feels cumbersome to me to write a webppl package just to include this code. Do other people have a similar experience, and if so, would it be useful to talk about a principled way to improve the experience?

Ideally, I'd like to just be able to call node.js's require function from webppl code. As things stand, this isn't possible because in the run function in src/main.js

function run(code, k, verbose) {
  var compiledCode = compile(code, verbose);
  if (verbose && console.time) {
    console.time('run');
  }
  eval.call(global, compiledCode)({}, k, '');
  if (verbose && console.timeEnd) {
    console.timeEnd('run');
  }
}

using eval.call makes require unavailable. It turns out that require is not a member of the global object, but rather a special function that's locally available to every module (https://nodejs.org/api/globals.html#globals_require). Apparently, the this argument to eval.call clobbers the module. Using eval directly works, and I checked that all the webppl tests pass when using it instead of eval.call. Is there a specific reason why we use eval.call here?

Even with require available, I end up having to call it like this

var thing = require.call(null, 'filename');

to circumvent the CPS transform. Would people be opposed to omitting require from the CPS transform or exposing it some other way (perhaps via a globally-available object, like webppl.nodeRequire)?

[null-a]

Regarding the current set-up: All npm packages are valid WebPPL packages, so you should be able to --require things like three.js directly. The library is made available via a global, the name of which is the package name camelCased.

[stuhlmueller]

In addition to what Paul said, I do think we should consider making require available within webppl. This would make structuring packages easier (#209, and perhaps #139), and would generally be a more pleasant way to indicate dependencies than passing them in via command-line arguments.

We'd have to think about what the semantics would be. We would definitely want it to be possible to require webppl (and hence node) packages, maybe with the same semantics as the current command line arguments. This means that we can't just make node require available, since it wouldn't correctly load the webppl and webppl-header files associated with webppl modules.

In addition to packages, we'd also want to enable requiring individual js and wppl files (which may in turn recursively require other files). Again, the ability to load webppl files would require an extension beyond node require.

Overall, it would be great if we could come up with a solution that (to users) feels as close as possible to how node require works, while doing "the right thing" when webppl files and packages are loaded.

A related consideration is whether we should introduce module.exports for webppl files. This would likely be necessary if we wanted to make our module experience closely analogous to node.

[null-a]

We could also consider building on the ES6 modules syntax. (Related #239.)

I wonder whether at some point we might need to consider using a different syntax to call out to JS so that member expressions are available for use as namespaces for modules.

[stuhlmueller]

Yes, import/export is probably better than require. In general, we can look towards other Javascript-based languages and systems, and see how they are handling issues like this. Meteor is using ES6 modules now, and transparently allows import of jsx files (e.g., import Foo from 'somefile.jsx'). We could similarly allow imports from both plain Javascript and webppl files and transpile as necessary based on file extension.

[null-a]

exposing it some other way (perhaps via a globally-available object, like webppl.nodeRequire)?

I don't see a problem with doing something like this until we make progress with a better package system. (This came up again in #379.) Alternatively, we could tackle #278.