Modularization

[megawac]

A work in progress to bring modular support and lodash implementations of certain methods.

Previous discussion be found here #984.

Resolves #984

See https://github.com/caolan/async/tree/modularization/build for the generated bundles

---

TODOS

• script releases:
  • generate modular builds/files
  • copy ES modules to build/es
  • add git commit
  • npm test
  • smoke test build files
  • cd build/
  • publish npm release
  • revert modular build changes, returning async to normal folder structure
  • git tag, git push
• possibly? create npm release for each module
• async modularized #984 (comment)
• use baseOrderBy in the implementation of internal/flatten?
• move documentation to jsdoc format (bonus?) save for later
• test package with npm pack before first 2.0 publish

[megawac]

I have managed to remove most of the string parsing logic included by lodash -- the only string parsing code remaining is toNumber included by baseRange. @jdalton may consider moving the toNumber logic to range and having baseRange assume it is provided numeric start, end and step?

Also @jdalton it'd be nice if there was a stable ref I could link for a lodash-es submodule -- it's pretty frustrating to update the submodule ref every couple days

[jdalton]

may consider moving the toNumber logic to range and having baseRange assume it is provided numeric start, end, and step?

Good idea!

it'd be nice if there was a stable ref I could link for a lodash-es submodule -- it's pretty frustrating to update the submodule ref every couple days

Lodash v4 will be released on the 12th.

[megawac]

Cool, so there will be a stable es-4.0.0 ref in v4?
On Jan 4, 2016 9:32 AM, "John-David Dalton" [email protected]
wrote:

may consider moving the toNumber logic to range and having baseRange
assume it is provided numeric start, end, and step?

Good idea!

it'd be nice if there was a stable ref I could link for a lodash-es
submodule -- it's pretty frustrating to update the submodule ref every
couple days

Lodash v4 will be released on the 12th.

—
Reply to this email directly or view it on GitHub
#996 (comment).

[jdalton]

Cool, so there will be a stable es-4.0.0 ref in v4?

Yep, just as there's a stable 3.10.1 ref.

[aearly]

What does the testing situation look like? Do we just test the bundle? It might be handy to test the es6 modules directly somehow as well, just to verify that Async works in both forms.

[megawac]

@aearly nope, we test lib/index.js see https://github.com/caolan/async/blob/modularization/test/test-async.js#L7

[jdalton]

Updated the es branch with toNumber removed from baseRange.
The gzipped minified size is now 6.17 kB so a 1.9 kB difference which I think is fine.

[megawac]

Nice! 👍 that is totally reasonable in my opinion, thoughts @aearly?

[jdalton]

Found another cheap win which will bring the size down to 5.73 kB; the diff to 1.4 kB
5.87 kB; the diff of 1.6 kB :P

[aearly]

Really excellent 👍 . Lodash is totally worth 1.4kb 😛 .

The only danger I see is relying on an internal lodash function, but those are pretty stable, right?

[jdalton]

The only danger I see is relying on an internal lodash function, but those are pretty stable, right?

It depends. When relying on internals I usually avoid the ^ range and go with the ~.
I can review the deps and report back if I see anything of concern. Offhand I think the internals used here are stable.

[jdalton]

Just checked and the base methods used are totally stable. You should have no problems with them.
I also simplified the deps of toNumber.

[aearly]

I'm thinking of a clean way to publish the individual method files without polluting the base directory.

My initial thought is to copy the package.json/readme/changelog and whatever else we need to the build/ directory and run npm publish from there. We wouldn't have to use the files property of the package.json because we would just publish everything in that folder. This has some drawbacks:

1. We can't use xyz any more since it relies on npm test, which needs to be run from the root directory. We'd have to use a custom script to run the tests, bump the versions, and publish.
2. The package.json will be out of sync with its location. The scripts will be assuming it's still in the root, but the actual publish artifacts will be assumed to be in build/. For example, main would be just async-cjs.js rather than build/async-cjs.js, which could be confusing to people submitting PRs.
3. We need to make sure that bower still pulls the correct file. The bower.json will need to have its main remain as dist/async.js, and we'll still have to commit that build artifact. The package.json and bower.json main fields will now be different.
4. I'd like to publish the ES modules as well as the CJS modules. I assume we'll copy lib to build and include "esnext:main": "es/index.js" or something similar.

Anyone have better ideas?

[megawac]

When do you want to release v2? and do we want to do a pre-release?

[martinheidegger]

I am jumping on this bandwaggon waaay too late but: Why all this effort of splitting it out to own packages? This will make npm just slower. npm supports require('async/something') for files within the async repo. Async only needed to distribute the modules as.js` files in the tree:

- package.json
- parallel.js
- each.js
- serial.js
- ....

Then you could do require('async/parallel') would automatically just load the code required for parallel. This would significantly faster in every respect, wouldn't it?

[megawac]

Why all this effort of splitting it out to own packages?

We're not doing that at the moment, but the benefit is the download for an individual method is smaller than the entire async package

[jdalton]

Though with npm's cache, packages of a given version are only downloaded once.
Subsequent requests hit the local npm cache on your system.

[aearly]

I want to release 2.0 when all the issues in the milestone are complete. (All the breaking changes). I'd also like to get the docs migrated to jsdoc and the tests ported to Mocha, but those aren't required.

[martinheidegger]

@megawac I do understand the need for splitting out the modules in own files/modules as this will reduce the size if used by packages such as a browserify but when downloading the files through npm this method should slow down things (considering the package.json and other overhead). Browserify (and babel afaik) do support deep binding require("<package>/<module>") so the bundled files would already become a lot smaller.

[megawac]

rowserify (and babel afaik) do support deep binding require("/") so the bundled files would already become a lot smaller.

Sure, assuming that none of the user's depencies require async and do require("async").

Also, at the moment babel does nothing in regards to import <package>/<module> besides transpile it to require("<package>/<module>"). Bundlers such as webpack, browserify, and rollup do preform this action

@aearly I think it makes sense to set a package.json field for "browser": "index.js", the reason being is webpack and rollup support tree shaking. This would allow the bundler to exclude unused modules (as if the user did require('async/some-module')

[martinheidegger]

@megawac Users likely will always be able to do require("async") and blow up their dependencies.

[megawac]

For reference, I measured the require time of using index.js vs dist/async in v2.

Env	dist/async.js	async/index.js
v0.12	8 ms	69ms
v4.4.7	9 ms	58ms
v6.2.2	11 ms	46ms

Methodology of test

I used time-require to make these measurements. I ran a tiny script

  require('time-require');

  var async = require('async');
  var async1 = require('async/index');

I ran this in v0.12, v4, and v6 installed via nvm 5 times each and took the average (tended not to deviate much) on my laptop

Computer

Costs involved loading async/index (node 6)

Note: I snipped a couple irrelevant parts of this log

Start time: (2016-07-13 22:37:29 UTC) [treshold=1%]
 #  module                          time  %
 5  ./isFunction (...sFunction.js)   1ms  ▇ 2%
 6  ./toNumber (no.../toNumber.js)   1ms  ▇ 2%
 7  ./toFinite (no.../toFinite.js)   1ms  ▇ 2%
 8  ./toInteger (n...toInteger.js)   1ms  ▇ 2%
 9  lodash/rest (n...dash/rest.js)   2ms  ▇▇ 4%
10  ./internal/app...applyEach.js)   3ms  ▇▇ 6%
11  ./_getLength (...getLength.js)   1ms  ▇ 2%
12  lodash/isArray...ArrayLike.js)   1ms  ▇ 2%
13  ./getIterator...tIterator.js)    1ms  ▇ 2%
14  ./isArray (nod...h/isArray.js)   1ms  ▇ 2%
15  ./_indexKeys (...indexKeys.js)   2ms  ▇▇ 4%
16  lodash/keys (n...dash/keys.js)   2ms  ▇▇ 4%
17  ./iterator (no.../iterator.js)   4ms  ▇▇▇ 8%
18  ./eachOfLimit...chOfLimit.js)    5ms  ▇▇▇ 10%
19  ./internal/doP...llelLimit.js)   6ms  ▇▇▇▇ 12%
20  ./mapLimit (no.../mapLimit.js)   6ms  ▇▇▇▇ 12%
21  ./map (node_mo...async/map.js)   6ms  ▇▇▇▇ 12%
22  ./applyEach (n...applyEach.js)   9ms  ▇▇▇▇▇▇ 18%
23  ./applyEachSer...achSeries.js)   1ms  ▇ 2%
24  ./asyncify (no.../asyncify.js)   1ms  ▇ 2%
25  ./_baseFor (no.../_baseFor.js)   1ms  ▇ 2%
26  lodash/_baseFo...aseForOwn.js)   1ms  ▇ 2%
27  ./auto (node_m...sync/auto.js)   1ms  ▇ 2%
28  lodash/_arrayM..._arrayMap.js)   1ms  ▇ 2%
29  ./_checkGlobal...eckGlobal.js)   1ms  ▇ 2%
30  ./_root (node_...ash/_root.js)   1ms  ▇ 2%
31  ./_Symbol (nod...h/_Symbol.js)   1ms  ▇ 2%
32  ./_baseToStrin...eToString.js)   1ms  ▇ 2%
33  ./_stringToArr...ngToArray.js)   1ms  ▇ 2%
34  lodash/trim (n...dash/trim.js)   2ms  ▇▇ 4%
35  ./autoInject (...utoInject.js)   4ms  ▇▇▇ 8%
36  ./internal/que...nal/queue.js)   1ms  ▇ 2%
37  ./cargo (node_...ync/cargo.js)   2ms  ▇▇ 4%
38  ./eachOfLimit...chOfLimit.js)    1ms  ▇ 2%
39  ./eachOfSeries...hOfSeries.js)   1ms  ▇ 2%
40  ./reduce (node...nc/reduce.js)   1ms  ▇ 2%
41  ./seq (node_mo...async/seq.js)   1ms  ▇ 2%
42  ./compose (nod...c/compose.js)   1ms  ▇ 2%
43  ./internal/doP...oParallel.js)   1ms  ▇ 2%
44  ./concat (node...nc/concat.js)   1ms  ▇ 2%
45  ./internal/doS.../doSeries.js)   1ms  ▇ 2%
46  ./concatSeries...catSeries.js)   1ms  ▇ 2%
47  ./constant (no.../constant.js)   1ms  ▇ 2%
48  lodash/identit.../identity.js)   1ms  ▇ 2%
49  ./internal/cre...ateTester.js)   1ms  ▇ 2%
50  ./detect (node...nc/detect.js)   2ms  ▇▇ 4%
51  ./doDuring (no.../doDuring.js)   1ms  ▇ 2%
52  ./eachLimit (n...eachLimit.js)   1ms  ▇ 2%
53  ./each (node_m...sync/each.js)   1ms  ▇ 2%
54  ./ensureAsync...sureAsync.js)    1ms  ▇ 2%
55  ./filter (node...nc/filter.js)   1ms  ▇ 2%
56  ./forever (nod...c/forever.js)   1ms  ▇ 2%
57  ./mapValues (n...mapValues.js)   1ms  ▇ 2%
58  ./memoize (nod...c/memoize.js)   1ms  ▇ 2%
59  ./internal/par.../parallel.js)   1ms  ▇ 2%
60  ./parallelLimi...llelLimit.js)   1ms  ▇ 2%
61  ./parallel (no.../parallel.js)   1ms  ▇ 2%
62  ./queue (node_...ync/queue.js)   1ms  ▇ 2%
63  ./priorityQueu...rityQueue.js)   1ms  ▇ 2%
64  ./reduceRight...duceRight.js)    2ms  ▇▇ 4%
65  ./reject (node...nc/reject.js)   1ms  ▇ 2%
66  ./rejectSeries...ectSeries.js)   1ms  ▇ 2%
67  ./retryable (n...retryable.js)   1ms  ▇ 2%
68  ./some (node_m...sync/some.js)   1ms  ▇ 2%
69  ./timeout (nod...c/timeout.js)   1ms  ▇ 2%
70  ./timesSeries...mesSeries.js)    1ms  ▇ 2%
71  ./internal/onl...nal/onlyOnce)   1ms  ▇ 2%
72  ./whilst (node...nc/whilst.js)   1ms  ▇ 2%
73  ./until (node_...ync/until.js)   1ms  ▇ 2%
74  async/index (n...ync/index.js)  44ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 86%
Total require(): 334
Total time: 51ms

P.s. I hope all your browsers support detail tags :)

Summary
Computer
Processor	4x Intel(R) Core(TM) i5-4300M CPU @ 2.60GHz
Memory	11988MB (7240MB used)
Operating System	Ubuntu 14.04.4 LTS

[aearly]

Cool! Thanks for backing up my gut assertions with some actual data. 😄

[melvinmezzy]

i need my information to run as my new device im on

[melvinmezzy]

my information