Skip to content

Latest commit

 

History

History
744 lines (504 loc) · 50.1 KB

README.md

File metadata and controls

744 lines (504 loc) · 50.1 KB
Raw

NPM

NoDent

NoDent is a small module for Nodejs that implements the JavaScript ES7 keywords async and await. These make writing, reading and understanding asynchronous and callback methods more implicit and embedded in the language. It works by (optionally) transforming JavaScript when it is loaded into Node.

This README assumes you're using Nodent v3.x.x - see Upgrading if your upgrading from an earlier version, and keep an eye out for the asides that look like..

v2.x users - changes between NoDent v2 and v3 are highlighted like this

Contents

Online demo

You can now see what Nodent does to your JS code with an online demo at here. Within the examples in this README, click on TRY-IT to see the code live.

Basic Use and Syntax

Declare an asynchronous function (one that returns "later").

async function tellYouLater(sayWhat) {
	// Do something asynchronous and terminal, such as DB access, web access, etc.
	return result ;
}

TRY-IT

Call an async function:

result = await tellYouLater("Hi there") ;

TRY-IT

To use NoDent, you need to:

require('nodent')() ;

This must take place early in your app, and need only happen once per app - there is no need to require('nodent') in more than one file, once it is loaded it will process any files ending in use nodent directive at the top of a .js file.

v2.x users, the '.njs' extension is still supported, but not recommended

You can't use the directive, or any other Nodent features in the file that initially require("nodent")(). If necessary, have a simple "loader.js" that requires Nodent and then requires your first Nodented file, or start your app with nodent from the command line:

./nodent.js myapp.js

That's the basics.

Why Nodent?

  • Performance - on current JS engines, Nodent is between 2x and 5x faster than other solutions in common cases, and up to 10x faster on mobile browsers.
  • Simple, imperative code style. Avoids callback pyramids in while maintaining 100% compatibility with existing code.
  • No dependency on ES6, "harmony"
  • No run-time overhead for Promises, Generators or any other feature beyond ES5 - works on most mobile browsers & IE (although Nodent can use Promises and Generators if you want it to!)
  • No execution framework needed as with traceur, babel or regenerator
  • No 'node-gyp' or similar OS platform requirement for threads or fibers
  • ES7 async and await on ES5 (most browsers and nodejs)
  • Compatible with ES6 too - nodent passes ES6 constructs unchanged for use with other transpilers and Node > 4.x
  • For more about ES7 async functions and await see:

Installation

npm install --save nodent

Command-Line usage

You can invoke and run a nodented JavaScript file from the command line (although for Node apps it's much easier using the JS transpiler). To load, compile and run your JS file (containing async and await), use:

./nodent.js myNodentedFile.js

You can also simply compile and display the output, without running it. This is useful if you want to pre-compile your scripts:

./nodent.js --out myNodentedFile.js

If you are using nodent as part of a toolchain with another compiler, you can output the ES5 or ES6 AST is ESTree format:

./nodent.js --ast myNodentedFile.js

...or read an AST from another tool

./nodent.js --fromast --out estree.json // Read the JSON file as an ESTree AST, and output the nodented JS code

To generate a source-map in the output, use --sourcemap.

./nodent.js --sourcemap --out myNodentedFile.js

The testing options --parseast and --minast output the source as parsed into the AST, before transformation and the minimal AST (without position information) respectively. The option --pretty outputs the source formatted by nodent before any syntax transformation. You can read the Javascript or JSON from stdin (i.e. piped) by omitting or replacing the filename with -.

The full list of options is:

option Description
--fromast Input is a JSON representation of an ESTree
--parseast Parse the input and output the ES7 specification ESTree as JSON
--pretty Parse the input and output the JS un-transformed
--out Parse the input and output the transformed ES5/6 JS
--ast Parse the input and output the transformed ES5/6 ESTree as JSON
--minast Same as --ast, but omit all the source-mapping and location information from the tree
--exec Execute the transformed code
--sourcemap Produce a source-map in the transformed code
--runtime Include the nodent runtime in the output

Code generation options:

option Description
--use=mode Ignore any "use nodent" directive in the source file, and force compilation mode to be es7,promises,generators. engine or default
--wrapAwait Allow await with a non-Promise expression more info...
--lazyThenables Evaluate async bodies lazily in 'es7' mode. See the Changelog for 2.4.0 for more information
--noruntime Compile code (in -promises or -engine mode) for execution with no runtime requirement at all
--es6target Compile code assuming an ES6 target (as of v3.0.8, this only requires support for arrow functions)
--noextensions Don't allow nodent's extensions to the ES7 specification more info...

Node.js usage

Automatic transpilation

There is no need to use the command line at all if you want to do is use async and await in your own scripts then just require('nodent')(). Files are transformed if they have a use nodent directive at the top, or have the extension ".njs". Existing files ending in '.js' without a use nodent... directive are untouched and are loaded and executed unchanged.

ES7 and Promises

Nodent can generate code that implements async and await using basic ES5 JavaScript, Promises (via Nodent's built-in library, a third party library or module, or an ES5+/6 platform) or Generators (ES6). Using the directive:

'use nodent';

The use nodent directive uses a default set of compilation options called 'default', which can be modifed in your package.json.

Within your package.json, you can have named sets of pre-defined options, which individual files can refer to if necessary. There are four pre-defined sets of options: promises, es7, generators and engine.

'use nodent-promises';
'use nodent-es7';
'use nodent-generators';
'use nodent-engine';

Which one should you use?

All the implementations work with each other - you can mix and match. If you're unsure as to which will suit your application best, or want to try them all out 'use nodent'; will use a 'default' configuration you can determine in your application's package.json. See Advanced Configuration for details.

Shipping a self-contained app to a browser, Node <=0.10.x or other unknown environment

use nodent-es7 - it's the most compatible as it doesn't require any platform support such as Promises or Generators, and works on a wide range of desktop and mobile browsers.

Shipping an app or module within Node, npm or modern browsers supporting Promises

use nodent-promises provides the most compatibility between modules and apps. If your module or library targets Node earlier than v4.1.x, you should install a Promise library (e.g. rsvp, when, bluebird) or use nodent.Thenable to expose the Promise API. In promises mode, there is no need for a runtime at all. Specifying the option use nodent-promises {"noRuntime":true} will generate pure ES5 code at the cost of some loss in performance and increase in code size.

v2.x users: nodent.EagerThenable() is still defined, but as of v3 is the same as the Thenable implementation.

Generators

use nodent-generators generates code which is reasonably easy to follow, but is best not used for anything beyond experimentation as it requires an advanced browser on the client-side, or Node v4.x.x. The performance and memory overhead of generators is poor - currently (Node v6.6.0) averaging 3.5 times slower compared to the es7 with 'lazyThenables'.

Engine

use nodent-engine does not transpile standard ES7 async/await constructs, but only transpiles the additional non-standard features provided by nodent - await anywhere, async getters, async return and throw. At the time of writing, not many runtimes implement async and await - Chrome v53 does with command line flags, and Edge 14 are examples. On Chrome, performance is better than generators, but not quite as good as Promises, and still less than half the speed of ES7 mode. In promises mode, there is no need for a runtime at all. Specifying the option use nodent-engine {"noRuntime":true} will generate pure ES5 code at the cost of some loss in performance and increase in code size.

Programmatical usage within scripts

To compile code programmatically you should use require(nodent-compiler). This module is a standalone compiler that has:

  • no require hook (it does not intercept .js files as they are loaded into Node)
  • no runtime (it does not implement any functions required to run transpiled code)
  • no pollution (because it has no runtime, it does not add async-specific bindings on Function.prototype, a global error handler, stack source-mapping)

Example usage:

var NodentCompiler = require('nodent-compiler');
var compiler = new NodentCompiler() ;
var es5ReadySourceCode = compiler.compile(sourceCode, filename, { sourcemap:false, promises: true, noRuntime: true, es6target: true });

Use within a browser

You can use async and await within a browser by auto-parsing your scripts when Nodejs serves them to your clients.

The exported function generateRequestHandler(path, matchRegex, options) creates a node/connect/express compatible function for handling requests for nodent-syntax files that are then parsed and served for use within a stanadrd browser environment, complete with a source map for easy debugging.

For example, with connect:

var nodent = require('nodent')() ;
...
var app = connect() ;
...
app.use(nodent.generateRequestHandler(
	"./static-files/web",	// Path to where the files are located
	/\.njs$/,				// Only parse & compiles ending in ".njs"
	options					// Options (see below)
)) ;

The regex can be omitted, in which case it has the value above.

The currently supported options are:

enableCache: <boolean>		// Caches the compiled output in memory for speedy serving.
runtime: <boolean>			// Set to precede the compiled code with the runtime support required by Nodent
extensions: <string-array>	// A set of file extensions to append if the specified URL path does not exist.
htmlScriptRegex: <optional regex> // If present, Nodent will attempt to read and parse <script> tags within HTML files matching the specified regex
compiler:{					// Options for the code generator
	es7:<boolean>,			// Compile in es7 mode (like 'use nodent-es7')
	promises:<boolean>,		// Compile in Promises mode (like 'use nodent-promises')
	generators:<boolean>,	// Compile in generator mode (like 'use nodent-generators')
	engine:<boolean>,		// Compile in engine mode (like 'use nodent-engine')
	sourcemap:<boolean>,	// Create a sourcemap for the browser's debugger
	wrapAwait:<boolean>,		// Allow 'await' on non-Promise expressions
	lazyThenables:<boolean>,	// Evaluate async bodies lazily in 'es7' mode. See the Changelog for 2.4.0 for more information
	noRuntime:<boolean>,  	// Only compatible with promises & engine. Generate pure ES5 code for an environment that support Promises natively or as a global declaration. Currently about 15% slower that using the built-in runtime $asyncbind. Default is false.
	es6target:<boolean>		// Compile code assuming an ES6 target (as of v3.0.8, this only requires support for arrow functions)
}
setHeaders: function(response) {}	// Called prior to outputting compiled code to allow for headers (e.g. cache settings) to be sent

Note that parsing of script tags within HTML is relatively simple - the parsing is based on regex and is therefore easily confused by JS strings that contain the text 'script', or malformed/nested tags. Ensure you are parsing accurate HTML to avoid these errors. Scripts inline in HTML do not support source-mapping at present.

If you're using a modern browser with Promise support (or including a third-party promise library), you can specify the compiler filag noRuntime: true, which will generate pure ES5 code at the cost of some loss in performance and increase in code size. Otherwise, you'll need to provide some support routines at runtime (i.e. in the browser):

  • Function.prototype.$asyncbind

  • Function.prototype.$asyncspawn if you're using generators

  • Object.$makeThenable if you're using the wrapAwait option and not using promises (see await with a non-Promise

  • window.$error if you use await outside of an async function, to catch unhandled errors, for example:

    // Called when an async function throws an exception during // asynchronous operations and the calling synchronous function has returned. window.$error = function(exception) { // Maybe log the error somewhere throw ex ; };

This are generated automatically in the transpiled files when you set the runtime option, and declared when Nodent is loaded (so they are already avaiable for use within Node).

Further information on using Nodent in the browser can be found at #2.

Other options: Babel & Browserify

You can also invoke nodent from browserify as a plugin, or as an alternative, faster implementation than than Babel's transform-async-to-generator

Async and Await syntax and usage

You can find out more about defining and calling async functions here. There's plenty on the web too.

Gotchas and ES7 compatibility

Async programming with Nodent (or ES7) is much easier and simpler to debug than doing it by hand, or even using run-time constructs such as Promises, which have a complex implementation of the their own when compiled to ES5. However, a couple of common cases are important to avoid.

Differences from the ES7 specification

You can continue to use all the Nodent extensions with async/await capable engines. In the use nodent-engine mode, all ES7 standard async/await constructs are passed through unchanged, and only functions that use a Nodent extension are transformed.

You can disable the extensions (but not the known differences) by compiling with flag parser:{ noNodentExtensions: true} or the command line option --noextensions. Nodent will pass the code unalterted to the parser (acorn) which will fail the syntactic extensions. In this scenario, the dependency acorn-es7-plugin is not required (but will still be installed by default). The option only works with acorn >4.x.

Extensions to the specification:

  • async getters and ** static async** class members:

Nodent permits a class or object definition to define async getters:

async get data() { ... }
get async data() { ... }

class MyClass {
	static async name() { ... }
}
  • await outside async

The ES7 async-await spec states that you can only use await inside an async function. This generates a warning in nodent, but is permitted. The synchronous return value from the function is compilation mode dependent. In practice this means that the standard, synchronous function containing the await does not have a useful return value of it's own.

  • async return/throw

The statements async return <expression> and async throw <expression> are proposed extensions to the ES7 standard (see tc39/proposal-async-await#38). The alternative to this syntax is to use a standard ES5 declaration returning a Promise. See below for details.

Known differences from the specification:

  • AsyncFunction

The AsyncFunction type is not defined by default, but is returned via the expression require('nodent')(...).require('asyncfunction'). The AsyncFunction constructor allows you to create async functions on the fly, just as the standard Function constructor does.

  • case without break

As of the current version, case blocks without a break; that fall thorugh into the following case do not transform correctly if they contain an await expression. Re-work each case to have it's own execution block ending in break, return or throw. Nodent logs a warning when it detects this situation.

  • await non-Promise

The ES7 specification allows an application to await on a non-Promise value (this occurs because the template implementation wraps every generated value in a Promise). So the statement:

	var x = await 100 ; // 100

...is valid. Nodent, by default, does not allow this behaviour (you'll get a run-time error about '100.then is not a function'. Generally, this is not a problem in that you obviously only want to wait on asynchronous things (and not numbers, strings or anything else). However, there is one unpleasant edge case, which is where an expression might be a Promise (my advice is to never write code like this, and avoid code that does).

	var x = await maybeThisIsAPromise() ;

In this case, the expression will need wrapping before it is awaited on by Nodent. You can emulate this behaviour by specifying the code-generation flag 'wrapAwait' in your package.json or after the nodent directive:

	'use nodent {"wrapAwait":true}';

Wrapping every value in a Promise increases the time taken to invoke an async function by about 20%. An alternative to wrapping everything is to only wrap expression where this might be the case explicitly:

	var x = await Promise.resolve(maybeThisIsAPromise()) ;

or

	var isThenable = require('nodent').isThenable ;
		...
	var x = maybeThisIsAPromise() ;
	if (isThenable(x))
  		x = await x ;

The second implementation avoids the expense (20%) of wrapping every return value in a Promise, with the extra code for testing if it is a Promise before awaiting on it.

  • lazyThenables

v2.x users - lazyThenables are only available in -es7 mode in v3, and the nodent.Thenable implementation is not lazy, as it was in v2.x.

Invoking an async function without a preceding await (simply by calling it) executes the function body but you can't get the result. This is useful for initiating 'background' things, or running async functions for their side effects. This is in compliance with the ES7 specification.

However, this has a performance overhead. For maximum performance, you can specify this code generation option in use nodent-es7 {"lazyThenables":true} mode. In this mode, if you call the async function the body is not actually executed until resolved with an await (or a .then()). If you know your code always uses await, you can use this option to improve performance.

In use nodent-promises mode, it is the implementation of the Promise that determines the execution scheduling and performance. The table below is a summary of modes and execution semantics. You can test the performance on your own hardware with the following command. Note the relative performance is a worst case, since the test does nothing other than make async calls in a loop.

	./nodent.js tests tests/semantics/perf.js
Mode Flags / Implementation Lazy / Eager Possibly sync resolution Performance (relative)
es7 lazyThenable Lazy Yes 1.0
es7 (none) Eager No 1.7x slower
promises nodent Eager No 1.7x slower
promises node 6.6 native Eager No 5.2x slower
promises bluebird 3.4.6 Eager No 2.0x slower
promises rsvp 3.3.1 Eager No 2.2x slower
promises when 3.7.7 Eager No 1.6x slower
generators nodent Eager No 7.5x slower
generators node 6.6 native Eager No 15.0x slower
generators bluebird 3.4.6 Eager No 8.5x slower
generators rsvp 3.3.1 Eager No 7.6x slower
generators when 3.7.7 Eager No 8.3x slower

All other JavaScript ES5/6/2015 constructs will be transformed as necessary to implement async and await.

v2.x users - note the timings and execution semantics for Thenable (and EagerThenable) have changed: they are now fully Promise/A+ compliant, meaning they resolve asynchronously and evaluate eagerly. Only -es7 lazyThenable mode might resolve synchronously.

Exiting async functions from callbacks

Specifically in Nodent (not specified by ES7), you can interface an ES7 async function with a old style callback-based function. For example, to create an async function that sleeps for a bit, you can use the standard setTimeout function, and in its callback use the form async return <expression> to not only return from the callback, but also the surrounding async function:

async function sleep(t) {
    setTimeout(function(){
    	// NB: "async return" and "async throw" are NOT ES7 standard syntax
    	async return undefined;
    },t) ;
}

TRY-IT

Similarly, async throw <expression> causes the inner callback to make the container async function throw an exception. The async return and async throw statements are NOT ES7 standards (see tc39/proposal-async-await#38). If you want your code to remain compatible with standard ES7 implementations when the arrive, use the second form above, which is what nodent would generate and is therefore ES5/6/7 compatible.

Advanced Configuration

Nodent has two sets of configuration values:

  • one controls the runtime environment - catching unhandled errors, handling warnings from Nodent and stack mapping, etc.
  • the other controls code generation - whether to generate code that uses Promises or generators (or not), whether to await on non-Promise values, whether to include the runtime code, etc.

The first is defined once per installation (Nodent contained as dependencies within dependencies have their own, per-installation, instances). You can 'redefine' the values, but the effect is to overwrite existing settings. These are specified as the first argument when you require('nodent')(options). Details of the options are below.

The second set is defined per-file for each file that Nodent loads and compiles. The options are:

Member Type
es7 boolean set by any use nodent... directive
promises boolean set by the directive use nodent-promises and use nodent-generators
generators boolean set by the directive use nodent-generators
engine boolean set by the directive use nodent-engine
wrapAwait boolean default: false - allow await followed by a non-Promise more info...
sourcemap boolean default: true - generate a source-map in the output JS
noRuntime boolean default: false - generate pure ES5 code with external dependencies. The code is bigger and slower, and only works with -promises or -engine
es6target boolean default: false - use ES6 constructs to improve code speed and size (as of v3.0.8, this only requires support for arrow functions)
parser object default: {sourceType:'script'} - passed to Acorn to control the parser
mapStartLine int default: 0 - initial line number for the source-map
generatedSymbolPrefix string default '$': string used to disambiguate indentifiers created by the compiler

The members $return, $error, $arguments, $asyncspawn, $asyncbind, $makeThenable represent the symbols generated by the compiler. You could change them to avoid name clashes, but this is not recommended.

When determining what options to use when compiling an individual file, nodent follows the sequence:

  • Use the set specified after the use nodent- directive. For example use nodent-promises uses a predefined set called 'promises'. Other predefined sets are 'es7', 'generators' and 'engine'. If the use nodent doesn't have a name, the internal name "default" is used.

  • Apply any modifications contained within the package.json containing your module or application. The package.json can (optionally) contain a 'nodent' section to define your own sets of options. For example, to create a set to be used by files containing a use nodent-myapp directive:

      "nodent":{
  	"directive":{
  		"myapp":{
  			"promises":true,
  			"wrapAwait":true
  		}
  	}
  }

You can also change options for the pre-defined sets here (default, es7, promises, generators, engine).

v2.x users - Until v2.5.4, Nodent would typically look for your application's package.json. All later versions use the installation location of the calling code, so an application, module and sub-module can all have their own settings and defaults.

  • Finally, nodent applies any options specified within the directive, but after the name. The options are strict JSON and cannot be an expression. This is useful for quickly testing options, but is probably a bad idea if applied to very many files. One exception is rare use of the wrapAwait option, which has a performance overhead and few genuine use-cases. For example, to create the same effect as the 'myapp' set above:

      'use nodent-promises {"wrapAwait":true}';

You can programmatically set these options before creating the nodent compiler (but after requiring nodent) by using the setDefaultCompileOptions() and setCompileOptions() API calls, however it is more flexible and less likely to clash with another module if you use the techniques above.

Within a nodented file, the special symbol __nodent is expanded out to the current option set. It is not a variable and cannot be assigned to - it is an object literal. This has few useful use-cases, except for testing. An example is here

API

Create an instance of a nodent compiler:

var nodent = require('nodent')(options);

Options:

Member Type
dontMapStackTraces boolean default: false
augmentObject boolean Adds asyncify(PromiseProvider) and isThenable() to Object.prototype, making expressions such as var client = new DB().asyncify(Promise) and if (abc.isThenable()) await abc() less verbose
extension string extension for files to be compiled (default: '.njs'). Note that this is unused if the file has a use nodent- directive.
log(msg) function Called when nodent has a warning or similar to show. By default they are passed to console.warn(). Set this member to change how to record logging, or to false to disable logging.

v2.x The flag 'asyncStackTrace' has been removed as modern debuggers can do this better than nodent can. You can specify it, but it is ignored.

Return: a 'nodent' compiler object with the following properties:

Member Type
version string The currently installed version
asyncify (PromiseProvider) function Return a function to convert an object with functions with callbacks to ones with async function members. asyncify is also a meta-property (see below)
Thenable(function) EagerThenable()(function) function Nodent's in-built Promise implementation. Thenable is also a meta-property (see below)
require(moduleName,options) object Import an async helper module
generateRequestHandler(path, matchRegex, options) function Create a function use with Express or Connect that compiles files for a browser on demand - like a magic version of the 'static' middleware
isThenable (object) boolean Return boolean if the supplied argument is Thenable (i.e. has an executable then member). All Promises, nodent.EagerThenable() and nodent.Thenable return true
$asyncbind function Runtime required in -promises and -engine mode if not compiled with noRuntime:true
$asyncspawn function Runtime required in generator mode

Note the nodent object has other members used for implementation - these are subject to change and are not part of the API.

v2.x users - nodent.Thenable and nodent.EagerThenable() are now full Promises/A+-compliant Promise implementations. There is no external access to the synchronous 'Thenable' used in -es7-lazyThenables mode.

Meta-API

You can over-ride certain defaults and access values that are global to the process (as opposed to module by module) by instantiating nodent without an argument:

var nodentMeta = require('nodent') ;

The available meta-properties are:

Member Type
Thenable function Nodent's built in Promise/A+ implementation
asyncify object Method to transform methods from callbacks to async functions by wrapping in Promises
setDefaultCompileOptions (compiler[,env]) function Set the defaults for the compiler and environment. This should be called before the first compiler is created. The default environment options (log augmentObject extension dontMapStackTraces asyncStackTrace) will be used when the corresponding option is missing when the compiler is created. The compiler options (sourcemap and default symbol names) must be set before the first compiler is created. The other compilation options (es7 promises generators engine) are set by the corresponding directive
setCompileOptions (name,compiler) function Set the compilation options for a named directive for the compiler. This should be called before the first compiler is created. 
// Turn off sourcemap generation:
nodentMeta.setDefaultCompileOptions({sourcemap:false},{asyncStackTrace:true})

// Access values that are global to all nodent compiler instances
var Promise = global.Promise || nodentMeta.Thenable ;	// Set a Promise provider for this module
nodentMeta.asyncify

You still need to (at least once) create the compiler:

var nodent = nodentMeta(options) ;

The return ('compiler') has the additional, instance specific properties specified in the API above.

Built-in conversions and helpers

Nodent has a small set of covers for common Node modules. More information can be found here

Testing

Nodent has a test suite (in ./tests) which is itself a node package. Since it requires a bunch of Promise implementations to test against, it is NOT installed when 'nodent' is installed. The Promise implementations are option - you can run the tests using the nodent.Thenable and native Promises (if available) without installing any other modules. To run the tests:

npm test

If you want to run the tests against some popular Promise libraries:

cd tests
npm install
cd ..
./nodent.js tests

The test runner in tests/index.js accepts the following options:

./nodent.js tests [OPTIONS] [test-files]

--quick      	Don't target a specific execute time, just run each test once
--nogenerators 	Performance test syntax transformations only, not generators
--genonly	 	Only run the performance tests for generator mode
--noengine		Performance test the underlying engine's support for async and await (e.g. Chrome v53, node.js v7 with flags)
--syntax	 	Check the parser/output code before running semantic tests
--syntaxonly	Only run syntax tests
--notStrict		Run the tests without a 'use strict' inserted at the top of every test file (NB: the reverse --forceStrict was the default until v3.0.11)
--noNodentExtensions	Compile the tests disallowing nodent's extensions to ES7 specification. This will generate test failures

v2.x users - The flag --generators has been replaced by --nogenerators, which has the opposite sense.

Performance

Run the test script without the --quick option to see how nodent code performs in ES7 mode, Promises, generators and engine on your platform. The specific 'perf' test just calls and awaits in a tight loop:

./nodent.js tests tests/semantics/perf.js

Additionally, a try the following links to test performance against Babel and Traceur.

nodent 356ms

babel 1072ms - more than 3x slower

traceur 1175ms - more than 3x slower

The example timings are from Chrome v53 on Mac OSX. I get even wider results with Firefox, and dramatically wider results on mobiles (nodent ES7 mode is upto 10x faster than generators and transpilers).

The test is a simple set of nested loops calling async functions that don't do much. The purpose is to illustrate the overhead generated in the transpilation by each compiler. In reality, you'd be crazy to use async calls for everything, but very well advised to use them for I/O bound operations (network, disks, streams, etc). In these cases, you can be reasonably certain that the overhead generated by the compilers would be small in comparison to the actual operation....but it's nice to know you're not wasting cycles, right? For those who want to know why, the real reason is the use of generators (the suggested implementation in the ES7 async/await specification), which are inefficient natively (about 50% slower than using 'nodent-promises'), and even worse when transcompiled into ES5.

Upgrading

v3.0.0

Nodent v3 is a significant update. If your project isn't using any internal functions or monkey patching, you should just be able to upgrade by changing your package.json - with very few exceptions the external API and features are the same. Major changes are:

  • The Promise implementation used by Nodent by default is based on the most excellent Zousan Promise library. After a lot of looking around and testing, Zousan proved to be one of the smallest and fastest around - small enough to fit into Nodent's runtime and faster in most practical cases than Bluebird and when. The speed is enhanced compared to most libraries as Nodent rarely generates code that relies on Promise chaining or having multiple .then() calls on the same Promise (v2 never did this, v3 does). Zousan is optimal in these cases as only creates the required data lazily.

  • Promises are now the default execution mode. Only -es7 with lazyThenables uses the synchronous Thenable protocol. This is only retained for speed in exceptional cases. In almost all practical applications (i.e. using async functions to handle IO of some sort), the overhead is around 20% per call, meaning it is trivial compared to the IO operation. Nodent syntax-transformation remains around 3-4 times faster than both native generators and libraries like regenerator.

  • The use of Promises/A+ compliant execution means the recursive loop asynchronisation used in Nodent v2 can be unwound. Specifically, for, while and do...while loops in Nodent v2 were recursive if the loop didn't yield control to an async operation. This meant that you could run out of stack on relatively trivial loop. For example:

for (var i=0; i<cache.length; i++) {
	if (cache[i].entry === null)
		cache[i].entry = await readNetworkResult() ;
}

This loop would appear to run fine while the cache hadn't been emptied, or was no more than a few hundred items long. However, if the test for 'emptiness' was never met, and the cache was thousands of items long, because the loop could be asynchronous, it would recurse deeply and cause the process to exit with a stack overflow.

Nodent v3 doesn't use recursion (it uses a trampoline), but in doing so it requires Promise chaining, which was not supported by nodent Thenables.

Nodent will still generate potentially recursive loops if you specify use nodent-es7 {"lazyThenables":true} since the basic lazy Thenable (while small and very fast) doesn't support chaining.

This execution case was pointed out by https://github.com/jods4 - many thanks.

  • The new code-generation option noRuntime will generate pure-ES5 code if you are using -promises, and pure-ES7 if you are using -engine. Other modes (-es7, -generators) still require a small runtime.

  • The new code-generation option es6target allows nodent to use some ES6 features to optimize code size and speed (for example, arrow functions in preference to calling bind(this)). As of v3.0.0 no differences are implemented, but if features are used they will be documented. This option is intended for targets such as node >=4.0.0 or modern browsers. It is unlikely to be worthwhile specifying an es6target if you intend to transpile that es6 to es5 (e.g. with Babel) - in that case don't specify the option and compile straight to ES5.

  • The helper function noDentify which was deprecated in v2 has now been removed from the main Nodent library. It can be required by Function.prototype.noDentify = nodent().require('nodentify') if needed.

Changelog

12-Jul-18 v3.2.8 (3.2.7)

  • Modify the implementation of asyncify so that it doesn't try and asyncify getters, even if they return a function. In node v10 this recurses as fs.ReadStream is a getter that does a lazy load, and for some reason it fails on Ubuntu (maybe all linux). Also, always push the promised callback, rather than forcing it into the final argument position, as it breaks logic used by fs.stat to determine its optional arguments. It is possible this change will break other function members that use unusual methods for determining the number and type of arguments.

02-May-18 v3.2.6 (3.2.5)

30-Apr-18 v3.2.4

  • Fix issue where catch body was incompletely transformed (see #109)
  • Fix issue where unreachable continuation generated an illegal Identifer after try-catch (see MatAtBread/fast-async#56)

24-Apr-18 v3.2.2

  • Fix the lifetime of let and const loop variable declarations (the previous implementation correctly limited the scope, but not the lifetime of the loop declarations, meaning they would be re-used if referenced in a callback)
  • Move compiler-only tests into a different sub-directory since they are now duplicates of the tests in nodent-compiler.

18-Apr-18 v3.2.0

  • Use nodent-compiler @3.2.0, which in turn uses the [email protected], which is the base AST transformer (with no parser or code generator), which is also used from Babel 7-beta (see babel/babel#7076) and forms the basis of leaner (external) fast-async Babel plugin.

10-Apr-18 v3.1.3-8

  • Various minor fixes and compatability changes to ensure operation with a variety of versions of acorn, babel, node, etc.

21-Aug-17 v3.1.2

  • Correctly walk destructuring assignment initializers

18-Aug-17 v3.1.1

  • Permit the compiler options es6target, noRuntime, promises and engine to be defined with the value "host", in which case they are determined from the execution environment the code was compiled in (ie. the version of nodejs). This allows transpiled code to gracefully compile across node >0.10 using additional features of each platform as they are available. An addition named set of options, also called "host" is defined to use these values, allowing the directive "use nodent-host" to compile code for the current platform. The new value can also be used to set (or reset) names sets in the package.json. For example, to set "host" as the default values:
  "nodent":{
    "directive":{
      "default":{
		"es6target":"host",
		"promises":"host",
		"engine":"host",
		"noRuntime":"host"
      }
    }
  }

12-Jul-17 v3.1.0

This version splits the CLI, require hook and options parsing (use nodent directive, package.json options) from the core compiler.

If your code monkey patches the internals of nodent, it may no longer work, as the core compiler has been moved into a submodule.

  • The core compiler is now in nodent-compiler.
  • The runtime remains in nodent-runtime
  • The CLI (which invokes the compiler), require hook (which intercepts .js files loaded into node and sets up the runtime environment) remiain here, in nodent.

If you are not requiring individual components within nodent or otherwise dependent on the internal structure, no changes will be required.

04-Jul-17 v3.0.20

  • Fix issue with '.' after a template string.
  • Fix edge cases with try-catch-finally (v3.0.18..19)

29-Mar-17 v3.0.17

28-Mar-17 v3.0.16

  • Update dependencies to work with Acorn v5

07-Mar-17 v3.0.14

  • Fix case where object key arguments is incorrectly mapped within async functions (Babel trees/fast-async only)
  • Fix mapping of identifier arguments in arrow functions (name is mapped, but no local arguments is created)
  • Fix case where destructured declarations containing an assigment lost their identifier when initialzed within an await
  • Fix case where initialized declarations were deferred until after an await leading to var url = '/', r = await get(url) failing due to url being not yet assigned.
  • Fix cases where assignments within expression lists (function parameters, array initializers, template strings, comma operators, object definitions) were initialised out of order if one of them contained an await
  • Fix case where an exception after a try block has completed incorrectly invokes the catch block it contains, rather than the one it is contained in

14-Feb-17 v3.0.12

  • Correctly handle async return|throw extension if nested more than one function level deep (see tests/semantics/nested-async-exits.js)
  • Provide default value for engine in configuration object so it can be programmatically over-ridden
  • Enforce use strict in tests unless --notStrict is specified
  • Correct handling of strict mode nested function hoisting to mimic the semantics of recent browsers
  • Run tests against NodeJS 0.10, 0.12, 4, 5, 6 and 7 in both strict and sloppy modes

19-Jan-17 V3.0.11

  • Fix case where try/catch is not contained in a block (e.g, case, conditional or label)
  • Fix case where throw is incorrectly mapped inside a sync try to an exit, rather than the matching catch [NB: this mimics V8's incorrect behaviour. After the fix it meets the specification and also Babel's implementation]
  • Fix issue that prevented the correct package.json being located for setting compiler options (3.0.10)

12-Jan-17 v3.0.9

  • Change internal variable name to avoid an issue with rollup (see #70)

20-Dec-16 v3.0.8

  • Use arrow functions when the es6target option is specified. This generates async call sequences that run approximately 20% faster (on on V8 v5.4.x) as calls to bind(this) and $asyncbind(this) are omitted from the generated code, especially when noRuntime is also specified.

23-Oct-16 v3.0.5, v3.0.4

  • 'engine' mode: don't map 'super' in all async members, onlt async getters (v3.0.5)
  • 'engine' mode: fix case where async class method contains a callback containing a reference to super
  • 'engine' mode: fix case where finally contains an exit (does not need transformation)
  • Return undefined from top-level trampoline to avoid long Promise chains
  • Run tests against nodejs 7 nightly and Chrome Canary to ensure engine-mode compliance

15-Oct-16 v3.0.3

  • Fix issue when handling multiple files on the command line, and "use" option is missing
  • Don't hoist function expressions (upsets Babel and ES6 compliant scope rules)

10-Oct-16 v3.0.2

  • Move runtime into a separate module nodent-runtime
  • Remove hidden '$' properties from AST before exit

06-Oct-16 v3.0.0 see above

28-Sep-16 v2.6.10

  • Fix issue with scoping on let/const declarations in for-loop initializers
  • Maintain location information when replacing/appending nodes
  • Add tests for dual-loop-scope and dual-while-throw

25-Sep-16 v2.6.9

  • Update acorn-es7-plugin to handle async(()=>0)
  • Fix case where generator mode generated an illegal anonymous FunctionDefintion from an ArrowFunctionExpression that was never in an expression

Older changes...

Credits

Thanks to the people who reported bugs, provided test cases and fixes while I was working on v3. Here's a few that I remember, in alpha order: