Implementation details

[mcollina]

It is probably too early to discuss this, but I think it is important to clarify early on. Where would the content of the standard library be implemented? Would they be part of JS engines, or part of the runtime on top?

[littledan]

So far, specifications that TC39 produces tend to be implemented in JS engines. I was imagining that this layering would likely continue in the future, but as you say, this is an implementation detail that we can't mandate in TC39. Do you see issues with implementing a standard library at the engine level?

However, there has been discussion about whether these libraries could be implemented in JavaScript. This has motivated a number of JavaScript language feature proposals, including #13 and https://github.com/domenic/proposal-function-prototype-tostring-censorship .

[mcollina]

So far, specifications that TC39 produces tend to be implemented in JS engines. I was imagining that this layering would likely continue in the future, but as you say, this is an implementation detail that we can't mandate in TC39. Do you see issues with implementing a standard library at the engine level?

I have two concerns:

• to ensure best compatibility with the Node.js ecosystem, V8 would have to provide a mechanism to load these in a way that can be backward-compatible with commonjs.
• we will need to provide polyfill implementation of those features for bundlers and older Node.js versions, and there will be the need to have reference/pure-js implementations of most of the standard library. This will reduce diversity in implementations because if there is a reference implementation, why ship a different one?

[paul-hammant]

For the love of Turing implement it test-driven, in Rust, on Gihub, and with Microsoft, Google, Apple and Mozilla devs collaborating.

[littledan]

@mcollina These are both legitimate concerns, which I think could be met regardless of whether we put it in the JS engine or not.

• For modules, my understanding was that there was ongoing work to let CJS modules import ESM modules as part of the modules effort, cc @guybedford
• For reuse of a JS implementation: JS engines tend to have some JS in them already. I imagine they will be able to hook themselves up to a JS implementation of a standard library.

[bmeck]

@littledan

For modules, my understanding was that there was ongoing work to let CJS modules import ESM modules as part of the modules effort, cc @guybedford

Yes, using import() but I think the question from @mcollina was about synchronous loading without a wrapper Promise. @mcollina does that sound correct? If so, it would mean ensuring engines are able to put expose these synchronously, but might be a host / engine concern for implementation rather than a TC39 concern. This follows how https://github.com/nodejs/dynamic-modules/ is being driven by Node but intended to upstream to engines.

I like the approach of leaving this specific API as a host operation (could be a concrete op if desired?) since I think this topic will come up again regarding specifiers.

[littledan]

I don't see any particular barriers to figuring out a mechanism that JS's built-in modules could be imported as CJS modules. However, for scripts on the web, import() would be required.

[ljharb]

For the record, I find it an absolute necessity for this proposal to be able to synchronously access any “builtin modules” in both Modules and Scripts, on the web and elsewhere - import() is not sufficient.

[mcollina]

I think a way for loading those synchronously is necessary, yes.

[littledan]

@ljharb Why is synchronous loading from scripts necessary?

[ljharb]

To ensure no other code has the ability to run before polyfilling and environment setup is completed - it requires ensuring that the setup entrypoint runs first, of course. Without synchronous loading from Scripts, my current understanding is that that list of 1 caveat would become longer.

[littledan]

I see, that makes sense. I'm curious about whether this comes up in other virtualization cases, cc @bakkot.

[leobalter]

@ljharb it's weird we are requiring a built in module to be loaded from Script code and the way ES specs sync module loading is with modules. I respectfully disagree as I feel like await import() should be sufficient. Top-level await would not apply here but I'm already fine that we can handle a way to load these built-ins in Script.

It sounds like creating a way to sync load built-ins in Script is for a discussion apart of this proposal.

[zloirock]

@leobalter it should be discussed (and added) before updating of this proposal to the next stage because a proper way of polyfilling modules is one of the main issues which should be resolved before the next stage. I (as polyfills author) know that await import() is not enough.

[ljharb]

@leobalter if a new language feature is not going to be provided by a global, then providing it synchronously in Scripts is not a feature that should be taken away.

In other words, if this proposal wants to proceed as is, that would have to go along with a commitment to continue providing all new language features as globals or syntax - which i suspect isn’t the intention here.

[leobalter]

if a new language feature is not going to be provided by a global, then providing it synchronously in Scripts is not a feature that should be taken away.

@ljharb I really don't see any solution for sync loading in Scripts that is not similar to the static import from Modules. FWIW, built-in modules might even be immediately available and just one (promise) tick away from Script modules.

IMO, the great part of having a built-in modules is to not pollute the global and load more content if necessary. If I really need a built-in in a code that is limited to Script records I imagine my requirement is specific enough to be wrapped in a promise or an async function. I can't see a reason yet to block scripts for sync loading.

Maybe this could be offered directly from Node as some extension for require(). To allow that we could just use the spec in a way it allow built-in modules being loaded from impl. defined forms. Would this work for you? Browsers might already do it using the script tag, setting a way to save the binding names.

---

it should be discussed (and added) before updating of this proposal to the next stage because a proper way of polyfilling modules is one of the main issues which should be resolved before the next stage. I (as polyfills author) know that await import() is not enough.

@zloirock It's hard for me to see it without more examples. You have more experience with polyfills than I do. WRT discussing, I believe this is in the right track. We should discuss it here and during the meetings, regardless of stage advancement.

[zloirock]

@leobalter see this note and comments from #2.

[nicolo-ribaudo]

Why wouldn't this work?

<script>
// polyfill
import("std-lib-array").then(({ default: Array }) => {
  if (!Array.prototype.forEach) {
    Array.prototype.forEach = function (cb) {
      for (const el of this) cb(el);
    };
  }
});
</script>
<script>
// user code
import("std-lib-array").then(({ default: Array }) => {
  new Array(1, 2, 3).forEach(el => console.log(el));
});
</script>

the promises execution order is well-defined, so the polyfill will always run first.

[ljharb]

I don’t agree that “polluting the global” is a problem that this proposal solves. All it does is create a new global namespace. The polyfilling requirements (and import maps) mean that user code is precisely as able to pollute built in specifiers as it is the global object.

[zloirock]

@nicolo-ribaudo at least, because it could depend on some other standard library features and it's more async tasks. Like:

<script>
// polyfill
import("std-lib-array").then(({ default: Array }) => {
  if (!Array.prototype.forEach) {
    import("std-lib-object").then(({ default: Object }) => {
      Object.defineProperty(Array.prototype, 'forEach', { value: function (cb) {
        for (const el of this) cb(el);
      } });
    });
  }
});
</script>
<script>
// user code
import("std-lib-array").then(({ default: Array }) => {
  new Array(1, 2, 3).forEach(el => console.log(el)); // missed
});
</script>

[bmeck]

I don’t agree that “polluting the global” is a problem that this proposal solves. All it does is create a new global namespace. The polyfilling requirements (and import maps) mean that user code is precisely as able to pollute built in specifiers as it is the global object.

A key difference between globals and modules is that modules may differ per callsite importing them such as using import maps or a loader, globals do not have different values for different locations accessing them.

[ljharb]

Not without wrapping and shadowing, that is true.

However, no such capability exists in the language - import maps isn’t a language proposal.

[bmeck]

@ljharb

import maps isn’t a language proposal.

That isn't relevant? The language explains that hosts can do it by any means they see fit.

[ljharb]

@bmeck I think it's relevant in that adding new API in the language must provide a language feature to add/delete/repair/polyfill it - it's not sufficient to just hope that hosts will all provide a mechanism.

[bmeck]

@ljharb that wasn't related to my conversation? I'm pointing out they are very much not the same.

[ljharb]

@bmeck i agree that #41 (comment) is a difference that this proposal provides - but it does not contradict #41 (comment).

[bmeck]

@ljharb it isn't meant to contradict but point out that the idea that polyfilling is prevented by this is false. Hosts are allowed to do all sorts of things including preventing polyfilling on their own. The host has the power to ensure that polyfilling remains, but in this case is able to do so in a way that does not force a shared global namespace be created for modules. Forcing hosts to have polyfilling work is not present even in the current state of the language and can be seen where embedded does freeze the global before any user code can be run. I do not see how the ability to polyfill is harmed by this proposal as the ability to polyfill is not guaranteed to begin with.

[ljharb]

Hosts aren't permitted to make ES globals nonconfigurable or immutable - so in fact the current state of the language does force hosts to support polyfills/shims, as well as various security use cases (deniability).

[bmeck]

@ljharb

In InitializeHostDefinedRealm

"Create any implementation-defined global object properties on globalObj."

Does not prevent replacing properties nor exotic globals from altering things.

In RunJobs

In an implementation-dependent manner, obtain the ECMAScript source texts (see clause 10) and any associated host-defined values for zero or more ECMAScript scripts and/or ECMAScript modules.

Which could run host-defined code prior to user code that does any sort of global mutation.

What do you mean that they cannot make globals non-configurable or immutable?

Deniability can still be achieved with the exotic globals or through usage of other means like creating Realms/Compartments even with all of that in place. In addition, even when talking in the Realms calls the ability to deny access is generally not universal and the idea of a root realm commonly comes up. At some level the host provides all the capabilities including globals and module loading to that root Realm. The ability to virtualize module specifier->namespace mappings is missing from the language but does not mean that the ability is unable to be provided by hosts. Similarly to how hosts can deny globals from users as well, the ability to virtualize everything is not guaranteed.

[ljharb]

It definitely does prevent that - "create" means additive, the host is not permitted to alter mandated characteristics of the ES globals, which includes their property descriptor.

[bmeck]

@ljharb even if that quote is meant to be purely additive (I disagree on term litigation here), SetDefaultGlobalBindings does fire traps that can modify things on the exotic globals provided by hosts.