Streamline 1.0 issues (many of them) #298

aseemk · 2015-11-04T14:14:09Z

I'm excited to try out Streamline 1.0, but I haven't been able to get it working.

—

First, I got these npm peer dependency warnings and Babel errors.

$ npm install streamline
npm WARN peerDependencies The peer dependency babel@>=5.8.0 included from streamline will no
npm WARN peerDependencies longer be automatically installed to fulfill the peerDependency 
npm WARN peerDependencies in npm 3+. Your application will need to depend on it explicitly.
npm WARN peerDependencies The peer dependency babel@>=5.8.0 included from babel-plugin-streamline will no
npm WARN peerDependencies longer be automatically installed to fulfill the peerDependency 
npm WARN peerDependencies in npm 3+. Your application will need to depend on it explicitly.

> [email protected] install /path/to/project/node_modules/streamline/node_modules/fibers
> node build.js || nodejs build.js

`darwin-x64-v8-3.28` exists; testing
Binary is fine; exiting

npm WARN prefer global [email protected] should be installed with -g
npm WARN prefer global [email protected] should be installed with -g

[email protected] node_modules/babel

[email protected] node_modules/streamline
├── [email protected]
├── [email protected] ([email protected])
├── [email protected]
├── [email protected] ([email protected])
├── [email protected]
├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected], [email protected])
├── [email protected]
└── [email protected] ([email protected])

$ npm test

> [email protected] test /path/to/project
> mocha test

[STREAMLINE] transforming (fibers): /path/to/project/test/_auth._coffee

/path/to/project/node_modules/streamline/node_modules/babel/index.js:1
(function (exports, require, module, __filename, __dirname) { throw new Error(
                                                                    ^
Error: The node API for `babel` has been moved to `babel-core`.
    at Object.<anonymous> (/path/to/project/node_modules/streamline/node_modules/babel/index.js:1:7)
    at Module._compile (module.js:460:26)
    at Module._compile (/path/to/project/node_modules/streamline/lib/register.js:52:12)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (/path/to/project/node_modules/coffee-script/lib/coffee-script/register.js:45:36)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.exports.transform (/path/to/project/node_modules/streamline/lib/transformSync.js:25:10)
    at /path/to/project/node_modules/streamline/lib/transformSync.js:42:18
    at Object.exports.get (/path/to/project/node_modules/streamline/lib/cacheSync.js:58:15)
    at exports.transformFileSync (/path/to/project/node_modules/streamline/lib/transformSync.js:40:19)
    at Object.._coffee (/path/to/project/node_modules/streamline/lib/register.js:64:12)
    at Module.load (/path/to/project/node_modules/coffee-script/lib/coffee-script/register.js:45:36)
    ...

npm ERR! Test failed.  See above for more details.

The root cause seems to be that Streamline asks for Babel >=5.8.0 — as opposed to a safer ^5.8.0 — and Babel 6 indeed introduced breaking changes:

http://babeljs.io/blog/2015/10/29/6.0.0/

I can fix by explicitly installing Babel 5.8.x first:

$ npm uninstall streamline
$ npm uninstall babel    # 6.x got automatically installed alongside Streamline
$ npm install babel@^5.8.0
$ npm install streamline

—

Even when I fix that (as well as npm install streamline-runtime), my code (which ran fine on Streamline 0.x) won't run.

For starters, I'm surprised it's using Fibers! (And even then, I get this error.) Why?

[STREAMLINE] transforming (fibers): /path/to/project/test/fixtures/index._coffee

/path/to/project/node_modules/babel/node_modules/babel-core/lib/transformation/file/index.js:662
      if (err.stack) {
             ^
Error: No element indexed by 1
    at ArraySet_at [as at] (/path/to/project/node_modules/streamline/node_modules/source-map-support/node_modules/source-map/lib/source-map/array-set.js:83:11)
    at SourceMapConsumer_parseMappings [as _parseMappings] (/path/to/project/node_modules/streamline/node_modules/source-map-support/node_modules/source-map/lib/source-map/source-map-consumer.js:220:44)
    at SourceMapConsumer.Object.defineProperty.get (/path/to/project/node_modules/streamline/node_modules/source-map-support/node_modules/source-map/lib/source-map/source-map-consumer.js:160:14)
    at SourceMapConsumer_originalPositionFor [as originalPositionFor] (/path/to/project/node_modules/streamline/node_modules/source-map-support/node_modules/source-map/lib/source-map/source-map-consumer.js:311:43)
    at mapSourcePosition (/path/to/project/node_modules/streamline/node_modules/source-map-support/source-map-support.js:168:42)
    at wrapCallSite (/path/to/project/node_modules/streamline/node_modules/source-map-support/source-map-support.js:303:20)
    at /path/to/project/node_modules/streamline/node_modules/source-map-support/source-map-support.js:338:26
    at Array.map (native)
    at Function.prepareStackTrace (/path/to/project/node_modules/streamline/node_modules/source-map-support/source-map-support.js:337:24)
    at Function.Error.prepareStackTrace (/path/to/project/node_modules/streamline/node_modules/streamline-runtime/lib/util.js:71:37)
    at Function.Error.prepareStackTrace (/path/to/project/node_modules/streamline-runtime/lib/util.js:71:37)
    at File.wrap (/path/to/project/node_modules/babel/node_modules/babel-core/lib/transformation/file/index.js:662:14)
    at Pipeline.transform (/path/to/project/node_modules/babel/node_modules/babel-core/lib/transformation/pipeline.js:165:17)
    at Object.exports.transform (/path/to/project/node_modules/streamline/lib/transformSync.js:25:27)
    at /path/to/project/node_modules/streamline/lib/transformSync.js:42:18
    at Object.exports.get (/path/to/project/node_modules/streamline/lib/cacheSync.js:58:15)
    at exports.transformFileSync (/path/to/project/node_modules/streamline/lib/transformSync.js:40:19)
    ...

Here's my test/mocha.opts (which worked without issue in Streamline 0.x):

--compilers coffee:coffee-script/register,_coffee:streamline/register
--reporter spec
--timeout 5000
--slow 500

And I have no .streamline.json file (neither in this project directory or my home directory). (And in general, I don't use Fibers, so I doubt I've specified it somewhere.)

—

Even if I add a .streamline.json file and manually specify {"runtime":"callbacks"}, I then get this error about the regenerator module.

[STREAMLINE] transforming (callbacks): /path/to/project/test/_auth._coffee

module.js:338
    throw err;
          ^
Error: Cannot find module 'regenerator/runtime'
    at Function.Module._resolveFilename (module.js:336:15)
    at Function.Module._load (module.js:278:25)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/path/to/project/test/_auth._coffee:3:58)
    at Module._compile (module.js:460:26)
    at Module._compile (/path/to/project/node_modules/streamline/lib/register.js:52:12)
    at Object.._coffee (/path/to/project/node_modules/streamline/lib/register.js:65:10)
    at Module.load (/path/to/project/node_modules/coffee-script/lib/coffee-script/register.js:45:36)
    ...

Indeed, if I compile my code manually, I see that the generated JS has this at the top:

var regeneratorRuntime = typeof require === 'function' ? require('regenerator/runtime') : Streamline.require('regenerator/runtime');

So I guess I need to manually npm install regenerator too? The docs don't mention that; only that it's used for the compilation. Any reason streamline-runtime couldn't encapsulate this?

—

Finally, even when I do manually install regenerator, now I'm back to the same error as with fibers:

[STREAMLINE] transforming (callbacks): /path/to/project/test/fixtures/index._coffee
/path/to/project/node_modules/babel/node_modules/babel-core/lib/transformation/file/index.js:662
      if (err.stack) {
             ^
Error: No element indexed by 1
    at ArraySet_at [as at] (/path/to/project/node_modules/streamline/node_modules/source-map-support/node_modules/source-map/lib/source-map/array-set.js:83:11)

I debugged this by trying to compile that file manually, which gives a more helpful error:

[STREAMLINE] transforming (callbacks): /path/to/project/test/fixtures/index._coffee
SyntaxError: /path/to/project/test/fixtures/index._coffee: Line 85: unexpected `_` argument: enclosing function does not have an `_` parameter.
  83 |         _results = [];
  84 |         for (i = _i = 0; 0 <= numNodes ? _i < numNodes : _i > numNodes; i = 0 <= numNodes ? ++_i : --_i) {
> 85 |           _results.push(this.createTestNode(suite, _));
     |                         ^
  86 |         }
  87 |         return _results;
  88 |       }).call(_this);
    at File.errorWithNode (/path/to/project/node_modules/babel/node_modules/babel-core/lib/transformation/file/index.js:489:13)
    at NodePath.Plugin.visitor.CallExpression (/path/to/project/node_modules/streamline/node_modules/babel-plugin-streamline/lib/plugin.js:355:53)
    at NodePath.call (/path/to/project/node_modules/babel/node_modules/babel-core/lib/traversal/path/context.js:56:28)
    at NodePath.visit (/path/to/project/node_modules/babel/node_modules/babel-core/lib/traversal/path/context.js:90:8)
    at TraversalContext.visitMultiple (/path/to/project/node_modules/babel/node_modules/babel-core/lib/traversal/context.js:108:16)
    at TraversalContext.visit (/path/to/project/node_modules/babel/node_modules/babel-core/lib/traversal/context.js:146:19)
    at Function.traverse.node (/path/to/project/node_modules/babel/node_modules/babel-core/lib/traversal/index.js:76:17)
    at NodePath.visit (/path/to/project/node_modules/babel/node_modules/babel-core/lib/traversal/path/context.js:107:26)
    at TraversalContext.visitSingle (/path/to/project/node_modules/babel/node_modules/babel-core/lib/traversal/context.js:132:12)
    at TraversalContext.visit (/path/to/project/node_modules/babel/node_modules/babel-core/lib/traversal/context.js:148:19)
    at Function.traverse.node (/path/to/project/node_modules/babel/node_modules/babel-core/lib/traversal/index.js:76:17)
    at NodePath.visit (/path/to/project/node_modules/babel/node_modules/babel-core/lib/traversal/path/context.js:107:26)
    at TraversalContext.visitMultiple (/path/to/project/node_modules/babel/node_modules/babel-core/lib/traversal/context.js:108:16)
    ...

Here's the source CoffeeScript for that line of JS (which worked in Streamline 0.x):

nodes = (@createTestNode suite, _ for i in [0...numNodes])

And here's the JS that compiles to:

nodes = (function() {
  var j, ref, results1;
  results1 = [];
  for (i = j = 0, ref = numNodes; 0 <= ref ? j < ref : j > ref; i = 0 <= ref ? ++j : --j) {
    results1.push(this.createTestNode(suite, _));
  }
  return results1;
}).call(_this);

I think you previously had special-case checks for these CoffeeScript idioms?

—

For now, I rewrote those comprehensions to use explicit loops, so my code finally compiles now! =)

Unfortunately, one more break — with Node, this at the top level refers to module.exports, so I frequently use that in all my CoffeeScript modules to define my public exports:

@expectPrefix = (str, prefix) ->
    # ...

// AKA exports.expectPrefix = ...
this.expectPrefix = function (str, prefix) {
    // ...
};

Unfortunately, it looks like Streamline 1.0 compilation now breaks this!

/path/to/project/test/util/helpers.js:15
  this.expectPrefix = function (str, prefix) {
                    ^
TypeError: Cannot set property 'expectPrefix' of undefined
    at /path/to/project/test/util/helpers.js:15:21
    ...

Indeed, manually compiling with Streamline and looking at the bottom of the file:

var _filename = '/path/to/project/test/util/helpers._coffee';
(function () {
  // ...
}).call(undefined);

Why the undefined? Why not maintain whatever this happens to be at the top level? (E.g. whether exports in Node or window in the browser.)

—

I worked around that by manually replacing my @s with exports. everywhere... and finally, my code ran!! =)

Unfortunately, over half of my tests are failing now, and they're all failing by timing out — which suggests something related to Streamline and callbacks not getting called. =/

I'm exhausted at this point, so haven't debugged this further. But I wanted to file at least these issues.

Hope this helps, suggestions welcome, etc. Thanks @bjouhier.

The text was updated successfully, but these errors were encountered:

aseemk · 2015-11-04T14:52:59Z

Okay! I found the issue behind half my tests failing — I had replaced one too many @. Sorry for that false alarm (but literally everything else was still legit).

My tests are now almost back to 100%! I'll continue debugging the last few failures, but wanted to note one other thing: Mocha's reporter now doesn't output the list of errors at the end properly:

    ...
    ✓ should properly handle fatal database errors 
    4) should properly handle non-fatal errors on the first query
    ✓ should properly handle fatal client errors on an auto-commit first query 
    ✓ should properly handle fatal database errors on the first query 
    5) should properly handle errors with batching
    - should support streaming (TODO)
    ✓ (delete test graph) 

  99 passing (4s)
  12 pending
  5 failing

  6) (delete test graph)

(Normally, after the summary, Mocha prints all the errors + full stack traces.)

My tests are written in the standard Mocha + Streamline style (which has always worked):

describe 'Transactions', ->

    it 'should support foo', (_) ->
        # ...

    # ...

    it '(delete test graph)', (_) ->
        fixtures.deleteTestGraph module, _

I think Streamline + Mocha's done callback is generally working (as evidenced by tests successfully running, and a few failures being reported as failures right away rather than timing out), but I wonder if there's something funky happening with the runner at the end.

It's obv. not your domain to debug Mocha, but FYI. I'm on Mocha 2.0.1 (and Node 0.12); I'll see if upgrading to the latest Mocha 2.3.3 helps.

bjouhier · 2015-11-04T20:58:14Z

@aseemk
Thanks for the detailed report. Given the scope of the changes a beta would have been very helpful but I rushed a bit the release because of a dependency on some internal project (which is now running smoothly with streamline 1.0 - but we are not using everything).

Here are some first answers, with links to the issues that I have created to track their resolution.

Streamline 1.0 issues (many of them) #298. The NPM warnings are expected. I'm going to upgrade to babel 6 soon and I will try to fix them in this second pass.
Incorrect babel dependency - streamline fails to run #300. The incorrect babel dependency is a serious blunder. I'll fix it ASAP and republish to NPM.
The fibers default is intentional. I put a note about it but it is a bit buried at the end of the FAQ. The main reason is performance and debuggability. At least for us, the fibers runtime is a winner.
Regenerator dependency should be internal to streamline-runtime #301. You are right about the regenerator dependency. The intent was to encapsulate it in streamline-runtime but I must have missed something.
Streamline 1.0 fails on CS comprehension idioms #302. I'll need to check the CS comprehension issue. Normally streamline 1.0 should handle the CS idioms but my testing is very limited because we only have one CS module in our stack. Our unit tests pass on this module but that's not very thorough. Was this related to the extra @ or not?
Top level this is replaced by undefined #303. The this/undefined issue is an oversight. I'm going to fix it.
Mocha reporter broken by streamline 1.0 #304. The moccha issue needs investigation.

aseemk · 2015-11-04T22:12:37Z

Thanks for being so receptive to feedback as always, @bjouhier. Much appreciated!

Btw, the CS comprehension issue was a legit issue; not related to extra @. Thanks.

aseemk · 2015-11-04T22:24:10Z

Also, thanks for the heads-up that fibers is the new default. Agreed it'd be good to document that explicitly in the runtimes section.

bjouhier · 2015-11-05T10:18:51Z

I just published 1.0.9 to NPM. So I'm closing this one and I'll be following up in the specific issues.

aseemk · 2015-11-05T12:45:14Z

Awesome, I'll (try to) try again later today and let you know how it goes. Thanks again!

bjouhier closed this as completed Nov 5, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Streamline 1.0 issues (many of them) #298

Streamline 1.0 issues (many of them) #298

aseemk commented Nov 4, 2015

aseemk commented Nov 4, 2015

bjouhier commented Nov 4, 2015

aseemk commented Nov 4, 2015

aseemk commented Nov 4, 2015

bjouhier commented Nov 5, 2015

aseemk commented Nov 5, 2015

Streamline 1.0 issues (many of them) #298

Streamline 1.0 issues (many of them) #298

Comments

aseemk commented Nov 4, 2015

aseemk commented Nov 4, 2015

bjouhier commented Nov 4, 2015

aseemk commented Nov 4, 2015

aseemk commented Nov 4, 2015

bjouhier commented Nov 5, 2015

aseemk commented Nov 5, 2015