Thinking about deadlines

[MylesBorins]

During the collaborators summit we discussed that we may need to start thinking about deadlines.

One that was thrown out was trying to ship modules without a flag by Node.js 12 LTS, which would also give us the opportunity to backport to Node 10 LTS before maintenance.

Here are some important dates.

Node.js 11 is cut October 2018
Node.js 12 is cut is April 2019
Node.js 12 LTS is October 2019
Node.js 8 End of Life is December 2019
Node.js 10 LTS Maintenance Starts April 2020

So an example timeline could be:

Ship First Pass at implementation in Node 11 by October 2018
Ship Finalized Implementation in Node 12 by April 2019
Remove flag in Node 12 by October 2019
Remove flag in Node 10 by April 2020

As you can see with this timeline that only give us 4 months to agree on an implementation, 6 months to finalize that implementation, and another 6 months to get it stable. If we manage to do all the things we could be shipping a stable, unflagged implementation of ESM in all maintained version of Node.js by the end of 2019 🎉.

Thoughts?

[GeoffreyBooth]

This seems good, if ambitious. I’m more concerned about getting it right than getting it quickly. I know people have been waiting a long time for native modules support in Node, but this is a case where users have very viable alternatives: keep transpiling like they do now, or use CommonJS. If the implementation that we can produce by the deadline is missing too many of the features that users want, a lot of users will keep using those alternatives; and the all-ESM future will be slower in arriving.

That’s not to say we shouldn’t shoot for these deadlines, though. Let’s see how much we can get done.

[devsnek]

i think 11.0.0 is completely reasonable (personally i've been wanting to finish this up by 10 lts)

[targos]

I think it would be difficult to come up with better deadlines. Let's try this!

[mhdawson]

They are a good target.

[bmeck]

The target seem a good aspiration, but I don't want to cut corners to get to it. We currently have spent several months just agreeing on what potential use cases are and have some effort on relating features to use cases, and have not started relating the work done so far to implementation or specification constraints in meetings. We currently even have some opinions on removing ESM from core for now. I am unsure we will agree on implementation prior to those dates at our current pace. If things are not fully agreed upon by those dates I think punting another cycle is preferable and will hope others also do not rush things. That said, getting to agreements faster would be nice. I also have implementation concerns that I don't see being resolved in that timeline at the pace we are going, particularly around refactoring our existing work and exposing more APIs related to it.

[jkrems]

How far would we want to push something like this? Would it make sense to create a more complete breakdown of the decisions and tasks that are required for each of these milestones?

[giltayar]

I am totally with focusing on a subset of the use cases. I believe we did enough exploratory/brainstorimg thinking, and it's time to focus the group on an issue at a time. I would suggest starting to focus on the main issues (I have my list, but I'll reserve it to myself in the interest of less bikeshedding). And focusing for me is devoting most of the time of our bi-weekly meetings to one issue.

As I said in the meeting yesterday, I would suggest devoting the next meeting mostly to the issue of ESM->CJS interop, e.g.:

1. Can we import x from cjs` in an ESM? (what is called "transparent interop" by some)
2. Can we import {x} from cjs` in an ESM or is it only default exports from CJS?
3. Can we `import.meta.require('cjs') in an ESM? (or any other method therein for require-ing cjs)

And the ramifications therein.

The reason I want to start focusing on this is because it feels to me that this issue is at the heart of most disagreements in the group, and we need to tackle it face on.

I'm a newb, so not sure how to make a formal suggestion.

[benjamingr]

@giltayar just my personal opinion: we need to address two distinct use cases:

• People who want web compatibility.
• People who want code to work out of the box.

Loaders will allow us to address both use cases at once

• allowing people who want web compatibility to use ESM with a loader that does not import named exports from cjs (out of the box) - it also probably won't work with things like native modules. cjs and native modules could still be imported with import.meta.require as an explicit escape hatch here.
• allowing people who want full Node.js ecosystem compatibility to use a loader that supports CJS, probably named exports from cjs, native modules, json and all the goodies to use a separate loader.

The question IMO is just which of those should be the default and which invariant we want to prioritize as a default: web compatibility or ecosystem compatibility.

[giltayar]

@benjamingr - the points you raise are valid, but if we want to reach the deadline, we need to focus. I am all for focusing the whole of next meeting on the issue you raised (instead of ESM/CJS interop), i.e. focus on whether we should use loaders to solve all the use cases, or whether we should have a reasonable default behavior.

My point is—let's decide one one issue we'd like to focus our (rather extensible) firepower on, and decide on that. And it should be a big issue, i.e. one that will remove lots of future discussions due to a decision we took.

I agree that your issue is much more basic (and contentious), and therefore it is possible we should start with it. But let's start with one issue, and dedicate a whole meeting to that issue, up to a resolution, or if not a resolution, a set of two or three options which we can deliberate on in offline discussions.

Can I frame the issue as: "should we use loaders as some kind of mechanism that enables different module resolution and interop methods, and/or should we provide a reasonable default that the ecosystem can use"?

[ljharb]

I don’t think deadline-driven development is going to result in the best solution; we should not be dropping use cases or features by “focusing” on a deadline.

That said, i really do like the approach of figuring out which contentious and foundational questions need a resolution, and spending maximal time on each one, one at a time.

[GeoffreyBooth]

Most discussion about transparent interop so far has been deep into implementation, people suggesting various approaches and others pointing out issues and so on. Which is valuable, for sure, but I don't think there's any decision that needs to be made by the group. People just need to start coding, in several branches, and work through potential implementations. We don't need to decide as a group what approach an implementation should take, since we can create several of them and experiment.

So could the interop discussion perhaps instead be a brainstorm session that happens before the next meeting, with everyone interested, and with the goal being to work out potential approaches that people could then go try to implement? And it could be as long as people want, and it doesn't need quorum because nothing official needs to be decided. We can't really make any truly informed decisions anyway until we have working code.

[ljharb]

What we do need to decide, though, is what features are a requirement for an implementation to make sense at all, and which are nonstarters - implementing isn’t going to be the slow part here, it’s “deciding” that will be (has been) slow.

I disagree that any code whatsoever is needed to make informed decisions - the hard questions deal with the ecosystem, and until we have a final, unflagged implementation, we’re not really going to get much value out of multiple trial implementations imo.

[jkrems]

I don’t think deadline-driven development is going to result in the best solution

Yeah, maybe we should drop the "deadline" terminology from the discussion. But I think that thinking about timelines and trying to roughly plan out remaining work does have value. That doesn't mean everything is locked down and we care more about hitting milestone dates than a good end result.

[GeoffreyBooth]

What we do need to decide, though, is what features are a requirement for an implementation to make sense at all, and which are nonstarters - implementing isn’t going to be the slow part here, it’s “deciding” that will be (has been) slow.

I think we know the answer to this already: we want all the features that people have requested. Some will be in Node core, some will be in loaders, and some ultimately we might not be able to implement at all; but they’re all valid to try to include. In all the discussion so far on all the features, the only disagreements I’ve seen have been people saying that something wouldn’t be achievable without breaking spec, or breaking browser equivalence, or something similar; but those are all implementation concerns. If a particular implementation of a particular feature breaks spec, that can get pointed out and the implementation revised or the feature moved into a loader, or even dropped; but that’s not something we need to decide ahead of time. Any decisions we try to make now will be made predominantly based on what features people think will be achievable or won’t be achievable, or will be achievable without breaking spec or not, and ultimately that’s best determined through attempts at implementation.

I don’t think there’s any reason people can’t start coding. If we want to try to hit these milestones, people should just get going.

[bmeck]

@GeoffreyBooth there have been multiple implementations of ESM (notably the one currently in Node core and npm's) and feature branches trying to achieve various things. The implementation concerns come from a variety of attempts to achieve various things and have been pointed out based upon experience, they are not all theoretical. We shouldn't treat all experience as not existing.

Implementing new things is valuable but don't treat all experience and existing implementations as without relevance please. If you have specific ways of implementing features you can feel free to do so, but note that people who have done work prior to this are trying to alleviate timeline concerns in part by stating the problems before people spend time on things that have already been explored. If someone wishes to explore those areas and hopes to find new solutions, they can but we are all working together and have implementation's with feedback that we can and should use.

[GeoffreyBooth]

@bmeck Of course they’re relevant. Most if not all new implementations will be built on top of the previous ones. My point is that I don’t think we need group decisions here, at least not right now. Discussion is valuable, and should happen, and will inform everyone who’s trying to implement these features; and since no decisions are necessary right now, this discussion (and the subsequent development) can happen at any time. That was what my previous comment suggested, that we schedule a long discussion brainstorm before the next meeting, so that the people interested in these issues can hash out ideas and be pointed in the direction of existing code and start to make progress.

[MylesBorins]

Removing from agenda for now, can be re added later

[MylesBorins]

We should start thinking about timelines for phase 2 and phase 3