The async
/await
release!
There are a lot of new features and subtle breaking changes in this major version, but the biggest feature is that most Async methods return a Promise if you omit the callback, meaning you can await
them from within an async
function.
const results = await async.mapLimit(urls, 5, async url => {
const resp = await fetch(url)
return resp.body
})
- Most Async methods return a Promise when the final callback is omitted, making them
await
-able! (#1572) - We are now making heavy use of ES2015 features, this means we have dropped out-of-the-box support for Node 4 and earlier, and many old versions of browsers. (#1541, #1553)
- Calling
callback(false)
will cancel an async method, preventing further iteration and callback calls. This is useful for preventing memory leaks when you break out of an async flow by calling an outer callback. (#1064, #1542) during
anddoDuring
have been removed, and insteadwhilst
,doWhilst
,until
anddoUntil
now have asynchronoustest
functions. (#850, #1557)limits
of less than 1 now cause an error to be thrown in queues and collection methods. (#1249, #1552)memoize
no longer memoizes errors (#1465, #1466)
- Async generators are now supported in all the Collection methods. (#1560)
- Added
cargoQueue
, a queue with bothconcurrency
andpayload
size parameters. (#1567) - Queue objects returned from
queue
now have aSymbol.iterator
method, meaning they can be iterated over to inspect the current list of items in the queue. (#1459, #1556)
- Better handle arbitrary error objects in
asyncify
(#1568, #1569)
- Removed Lodash as a dependency (#1283, #1528)
- Miscellaneous docs fixes (#1393, #1501, #1540, #1543, #1558, #1563, #1564, #1579, #1581)
- Miscellaneous test fixes (#1538)
- Updated lodash to prevent
npm audit
warnings. (#1532, #1533) - Made
async-es
more optimized for webpack users (#1517) - Fixed a stack overflow with large collections and a synchronous iterator (#1514)
- Various small fixes/chores (#1505, #1511, #1527, #1530)
- Added missing aliases for many methods. Previously, you could not (e.g.)
require('async/find')
or useasync.anyLimit
. (#1483) - Improved
queue
performance. (#1448, #1454) - Add missing sourcemap (#1452, #1453)
- Various doc updates (#1448, #1471, #1483)
- Added
concatLimit
, theLimit
equivalent ofconcat
(#1426, #1430) concat
improvements: it now preserves order, handles falsy values and theiteratee
callback takes a variable number of arguments (#1437, #1436)- Fixed an issue in
queue
where there was a size discrepancy betweenworkersList().length
andrunning()
(#1428, #1429) - Various doc fixes (#1422, #1424)
- Added
tryEach
, for running async functions in parallel, where you only expect one to succeed. (#1365, #687) - Improved performance, most notably in
parallel
andwaterfall
(#1395) - Added
queue.remove()
, for removing items in aqueue
(#1397, #1391) - Fixed using
eval
, preventing Async from running in pages with Content Security Policy (#1404, #1403) - Fixed errors thrown in an
asyncify
ed function's callback being caught by the underlying Promise (#1408) - Fixed timing of
queue.empty()
(#1367) - Various doc fixes (#1314, #1394, #1412)
- Added support for ES2017
async
functions. Wherever you can pass a Node-style/CPS function that uses a callback, you can also pass anasync
function. Previously, you had to wrapasync
functions withasyncify
. The caveat is that it will only work ifasync
functions are supported natively in your environment, transpiled implementations can't be detected. (#1386, #1390) - Small doc fix (#1392)
- Added
groupBy
, and theSeries
/Limit
equivalents, analogous to_.groupBy
(#1364) - Fixed
transform
bug whencallback
was not passed (#1381) - Added note about
reflect
toparallel
docs (#1385)
- Fix
auto
bug when function names collided with Array.prototype (#1358) - Improve some error messages (#1349)
- Avoid stack overflow case in queue
- Fixed an issue in
some
,every
andfind
where processing would continue after the result was determined. - Cleanup implementations of
some
,every
andfind
- Make bundle size smaller
- Create optimized hotpath for
filter
in array case.
- Fixed a stackoverflow bug with
detect
,some
,every
on large inputs (#1293).
retry
andretryable
now support an optionalerrorFilter
function that determines if thetask
should retry on the error (#1256, #1261)- Optimized array iteration in
race
,cargo
,queue
, andpriorityQueue
(#1253) - Added alias documentation to doc site (#1251, #1254)
- Added BootStrap scrollspy to docs to highlight in the sidebar the current method being viewed (#1289, #1300)
- Various minor doc fixes (#1263, #1264, #1271, #1278, #1280, #1282, #1302)
- Significantly optimized all iteration based collection methods such as
each
,map
,filter
, etc (#1245, #1246, #1247).
Lots of changes here!
First and foremost, we have a slick new site for docs. Special thanks to @hargasinski for his work converting our old docs to jsdoc
format and implementing the new website. Also huge ups to @ivanseidel for designing our new logo. It was a long process for both of these tasks, but I think these changes turned out extraordinary well.
The biggest feature is modularization. You can now require("async/series")
to only require the series
function. Every Async library function is available this way. You still can require("async")
to require the entire library, like you could do before.
We also provide Async as a collection of ES2015 modules. You can now import {each} from 'async-es'
or import waterfall from 'async-es/waterfall'
. If you are using only a few Async functions, and are using a ES bundler such as Rollup, this can significantly lower your build size.
Major thanks to @Kikobeats, @aearly and @megawac for doing the majority of the modularization work, as well as @jdalton and @Rich-Harris for advisory work on the general modularization strategy.
Another one of the general themes of the 2.0 release is standardization of what an "async" function is. We are now more strictly following the node-style continuation passing style. That is, an async function is a function that:
- Takes a variable number of arguments
- The last argument is always a callback
- The callback can accept any number of arguments
- The first argument passed to the callback will be treated as an error result, if the argument is truthy
- Any number of result arguments can be passed after the "error" argument
- The callback is called once and exactly once, either on the same tick or later tick of the JavaScript event loop.
There were several cases where Async accepted some functions that did not strictly have these properties, most notably auto
, every
, some
, filter
, reject
and detect
.
Another theme is performance. We have eliminated internal deferrals in all cases where they make sense. For example, in waterfall
and auto
, there was a setImmediate
between each task -- these deferrals have been removed. A setImmediate
call can add up to 1ms of delay. This might not seem like a lot, but it can add up if you are using many Async functions in the course of processing a HTTP request, for example. Nearly all asynchronous functions that do I/O already have some sort of deferral built in, so the extra deferral is unnecessary. The trade-off of this change is removing our built-in stack-overflow defense. Many synchronous callback calls in series can quickly overflow the JS call stack. If you do have a function that is sometimes synchronous (calling its callback on the same tick), and are running into stack overflows, wrap it with async.ensureAsync()
.
Another big performance win has been re-implementing queue
, cargo
, and priorityQueue
with doubly linked lists instead of arrays. This has lead to queues being an order of magnitude faster on large sets of tasks.
- Async is now modularized. Individual functions can be
require()
d from the main package. (require('async/auto')
) (#984, #996) - Async is also available as a collection of ES2015 modules in the new
async-es
package. (import {forEachSeries} from 'async-es'
) (#984, #996) - Added
race
, analogous toPromise.race()
. It will run an array of async tasks in parallel and will call its callback with the result of the first task to respond. (#568, #1038) - Collection methods now accept ES2015 iterators. Maps, Sets, and anything that implements the iterator spec can now be passed directly to
each
,map
,parallel
, etc.. (#579, #839, #1074) - Added
mapValues
, for mapping over the properties of an object and returning an object with the same keys. (#1157, #1177) - Added
timeout
, a wrapper for an async function that will make the task time-out after the specified time. (#1007, #1027) - Added
reflect
andreflectAll
, analagous toPromise.reflect()
, a wrapper for async tasks that always succeeds, by gathering results and errors into an object. (#942, #1012, #1095) constant
supports dynamic arguments -- it will now always use its last argument as the callback. (#1016, #1052)setImmediate
andnextTick
now support arguments to partially apply to the deferred function, like the node-native versions do. (#940, #1053)auto
now supports resolving cyclic dependencies using Kahn's algorithm (#1140).- Added
autoInject
, a relative ofauto
that automatically spreads a task's dependencies as arguments to the task function. (#608, #1055, #1099, #1100) - You can now limit the concurrency of
auto
tasks. (#635, #637) - Added
retryable
, a relative ofretry
that wraps an async function, making it retry when called. (#1058) retry
now supports specifying a function that determines the next time interval, useful for exponential backoff, logging and other retry strategies. (#1161)retry
will now pass all of the arguments the task function was resolved with to the callback (#1231).- Added
q.unsaturated
-- callback called when aqueue
's number of running workers falls below a threshold. (#868, #1030, #1033, #1034) - Added
q.error
-- a callback called whenever aqueue
task calls its callback with an error. (#1170) applyEach
andapplyEachSeries
now pass results to the final callback. (#1088)
- Calling a callback more than once is considered an error, and an error will be thrown. This had an explicit breaking change in
waterfall
. If you were relying on this behavior, you should more accurately represent your control flow as an event emitter or stream. (#814, #815, #1048, #1050) auto
task functions now always take the callback as the last argument. If a task has dependencies, theresults
object will be passed as the first argument. To migrate old task functions, wrap them with_.flip
(#1036, #1042)- Internal
setImmediate
calls have been refactored away. This may make existing flows vulnerable to stack overflows if you use many synchronous functions in series. UseensureAsync
to work around this. (#696, #704, #1049, #1050) map
used to return an object when iterating over an object.map
now always returns an array, like in other libraries. The previous object behavior has been split out intomapValues
. (#1157, #1177)filter
,reject
,some
,every
,detect
and their families like{METHOD}Series
and{METHOD}Limit
now expect an error as the first callback argument, rather than just a simple boolean. Passnull
as the first argument, or usefs.access
instead offs.exists
. (#118, #774, #1028, #1041){METHOD}
and{METHOD}Series
are now implemented in terms of{METHOD}Limit
. This is a major internal simplification, and is not expected to cause many problems, but it does subtly affect how functions execute internally. (#778, #847)retry
's callback is now optional. Previously, omitting the callback would partially apply the function, meaning it could be passed directly as a task toseries
orauto
. The partially applied "control-flow" behavior has been separated out intoretryable
. (#1054, #1058)- The test function for
whilst
,until
, andduring
used to be passed non-error args from the iteratee function's callback, but this led to weirdness where the first call of the test function would be passed no args. We have made it so the test function is never passed extra arguments, and only thedoWhilst
,doUntil
, anddoDuring
functions pass iteratee callback arguments to the test function (#1217, #1224) - The
q.tasks
array has been renamedq._tasks
and is now implemented as a doubly linked list (DLL). Any code that used to interact with this array will need to be updated to either use the provided helpers or support DLLs (#1205). - The timing of the
q.saturated()
callback in aqueue
has been modified to better reflect when tasks pushed to the queue will start queueing. (#724, #1078) - Removed
iterator
method in favour of ES2015 iterator protocol which natively supports arrays (#1237) - Dropped support for Component, Jam, SPM, and Volo (#1175, ##176)
- Improved handling of no dependency cases in
auto
&autoInject
(#1147). - Fixed a bug where the callback generated by
asyncify
withPromises
could resolve twice (#1197). - Fixed several documented optional callbacks not actually being optional (#1223).
- Added
someSeries
andeverySeries
for symmetry, as well as a complete set ofany
/anyLimit
/anySeries
andall
//allLmit
/allSeries
aliases. - Added
find
as an alias fordetect. (as well as
findLimitand
findSeries`). - Various doc fixes (#1005, #1008, #1010, #1015, #1021, #1037, #1039, #1051, #1102, #1107, #1121, #1123, #1129, #1135, #1138, #1141, #1153, #1216, #1217, #1232, #1233, #1236, #1238)
Thank you @aearly and @megawac for taking the lead on version 2 of async.
- Allow using
"constructor"
as an argument inmemoize
(#998) - Give a better error messsage when
auto
dependency checking fails (#994) - Various doc updates (#936, #956, #979, #1002)
- Fix issue with
pause
inqueue
with concurrency enabled (#946) while
anduntil
now pass the final result to callback (#963)auto
will properly handle concurrency when there is no callback (#966)auto
will no. properly stop execution when an error occurs (#988, #993)- Various doc fixes (#971, #980)
- Added
transform
, analogous to_.transform
(#892) map
now returns an object when an object is passed in, rather than array with non-numeric keys.map
will begin always returning an array with numeric indexes in the next major release. (#873)auto
now accepts an optionalconcurrency
argument to limit the number o. running tasks (#637)- Added
queue#workersList()
, to retrieve the lis. of currently running tasks. (#891) - Various code simplifications (#896, #904)
- Various doc fixes 📜 (#890, #894, #903, #905, #912)
- Ensure coverage files don't get published on npm (#879)
- Add in overlooked
detectLimit
method (#866) - Removed unnecessary files from npm releases (#861)
- Removed usage of a reserved word to prevent 💥 in older environments (#870)
asyncify
now supports promises (#840)- Added
Limit
versions offilter
andreject
(#836) - Add
Limit
versions ofdetect
,some
andevery
(#828, #829) some
,every
anddetect
now short circuit early (#828, #829)- Improve detection of the global object (#804), enabling use in WebWorkers
whilst
now called with arguments from iterator (#823)during
now gets called with arguments from iterator (#824)- Code simplifications and optimizations aplenty (diff)
New Features:
- Added
constant
- Added
asyncify
/wrapSync
for making sync functions work with callbacks. (#671, #806) - Added
during
anddoDuring
, which are likewhilst
with an async truth test. (#800) retry
now accepts aninterval
parameter to specify a delay between retries. (#793)async
should work better in Web Workers due to betterroot
detection (#804)- Callbacks are now optional in
whilst
,doWhilst
,until
, anddoUntil
(#642) - Various internal updates (#786, #801, #802, #803)
- Various doc fixes (#790, #794)
Bug Fixes:
cargo
now exposes thepayload
size, andcargo.payload
can be changed on the fly after thecargo
is created. (#740, #744, #783)
Bug Fix:
- Small regression with synchronous iterator behavior in
eachSeries
with a 1-element array. Before 1.1.0,eachSeries
's callback was called on the same tick, which this patch restores. In 2.0.0, it will be called on the next tick. (#782)
New Features:
- Added
timesLimit
(#743) concurrency
can be changed after initialization inqueue
by settingq.concurrency
. The new concurrency will be reflected the next time a task is processed. (#747, #772)
Bug Fixes:
- Fixed a regression in
each
and family with empty arrays that have additional properties. (#775, #777)
Bug Fix:
- Small regression with synchronous iterator behavior in
eachSeries
with a 1-element array. Before 1.1.0,eachSeries
's callback was called on the same tick, which this patch restores. In 2.0.0, it will be called on the next tick. (#782)
New Features:
cargo
now supports all of the same methods and event callbacks asqueue
.- Added
ensureAsync
- A wrapper that ensures an async function calls its callback on a later tick. (#769) - Optimized
map
,eachOf
, andwaterfall
families of functions - Passing a
null
orundefined
array tomap
,each
,parallel
and families will be treated as an empty array (#667). - The callback is now optional for the composed results of
compose
andseq
. (#618) - Reduced file size by 4kb, (minified version by 1kb)
- Added code coverage through
nyc
andcoveralls
(#768)
Bug Fixes:
forever
will no longer stack overflow with a synchronous iterator (#622)eachLimit
and other limit functions will stop iterating once an error occurs (#754)- Always pass
null
in callbacks when there is no error (#439) - Ensure proper conditions when calling
drain()
after pushing an empty data set to a queue (#668) each
and family will properly handle an empty array (#578)eachSeries
and family will finish if the underlying array is modified during execution (#557)queue
will throw if a non-function is passed toq.push()
(#593)- Doc fixes (#629, #766)
No known breaking changes, we are simply complying with semver from here on out.
Changes:
- Start using a changelog!
- Add
forEachOf
for iterating over Objects (or to iterate Arrays with indexes available) (#168 #704 #321) - Detect deadlocks in
auto
(#663) - Better support for require.js (#527)
- Throw if queue created with concurrency
0
(#714) - Fix unneeded iteration in
queue.resume()
(#758) - Guard against timer mocking overriding
setImmediate
(#609 #611) - Miscellaneous doc fixes (#542 #596 #615 #628 #631 #690 #729)
- Use single noop function internally (#546)
- Optimize internal
_each
,_map
and_keys
functions.