Any real pitfalls to the WeakMap approach?

[mmiller42]

This might be an inappropriate place to ask this question, but I was looking at the example in the FAQ for using WeakMaps for encapsulation.

The potential pitfall shown seems to me to be easily averted, since there is no reason why makeGreeting would require access to private instance variables, and it would be easily mitigated with .call or .apply. Wouldn't this kind of situation easily be resolved by modifying greet(otherPerson) to do return privates.get(this).makeGreeting.call(this, otherPerson.name);? After all, the expected behavior in a realistic class of this kind is that makeGreeting would have access to the Person instance's public API, not its map of private members.

[bakkot]

Yes, if you know the pitfall is there it can be avoided, with care. I would still consider it an issue, since most people are unlikely to notice. The only other major downsides I'm aware of are that it's fairly opaque as to intent, has unergonomic syntax (especially for chained property access), and is probably harder for engines to optimize.

[boneskull]

The “pitfall” example should be removed. It doesn’t have anything to do with WeakMap per se.

The example could be re-implemented using private fields to show the exact same pitfall; a callback could access an object’s private field if it’s executed with the object’s context from within the class, right?

[rdking]

@boneskull Not "executed" but "created".
If the callback function is not created within the lexical scope of the class definition, then it won't have access to the private fields of the class. If it did, TC39 would have failed miserably on the encapsulation part. That's actually the part they did the best job on.

[erights]

If it did, TC39 would have failed miserably on the encapsulation part. That's actually the part they did the best job on.

Hi @rdking , thanks, I think ;)

[rdking]

As someone who doesn't like this proposal, I've tried everything I could to punch a hole in one of the major features of this proposal. As a result, I can say with impunity that the encapsulation is leak proof, not just leak resistant. The only way private data is getting leaked is if the class was written to do so.

Given how vocal I've been, I'm sure you can understand why I worded it the way I did. 😄

[rdking]

@erights
If you're still looking, I've got a question for you specifically. I want to understand your reasoning.

Why do/did you think it so critical that a private proposal follow WeakMap semantics?

[erights]

A major factor was the subject of our previous extended thread: If private names are reified as so-called private symbols, then (I believed at the time) it would be impossible for membranes to be both transparent and secure. While I was technically wrong about that being impossible, I think this line of thinking led to the right intuitions.

Also:

EcmaScript 3 had one encapsulation mechanism: lexical closure capturing lexical variables. We designed EcmaScript 5's strict mode largely to make this encapsulation perfect.

EcmaScript 6 introduced three more perfect encapsulation mechanisms:

• WeakMaps perfectly encapsulate their values behind their keys.
• Proxys perfectly encapsulate their handler.
• Modules perfectly encapsulate whatever they don't export.

I want to minimize fundamental mechanisms. Each of those above is fundamental, in that none can reasonably be built out of the others. I am glad we stopped at four. So-called private symbols would have been a fifth.

We knew we needed an object abstraction mechanism with encapsulated private state. My first three draft class abstractions were sugar over the objects-as-closures pattern, building on the encapsulation of lexical variables. For various reasons I understand but regret, that didn't fly. Instead, classes became sugar over the common pattern of prototype inheritance. Once WeakMap was introduced, it provided perfect encapsulation for inheritance-based objects, again without any additional fundamental mechanism. By using the WeakMap model of perfect encapsulation, we avoided introducing a fifth fundamental encapsulation mechanism.

Although classes can no longer be perfectly explained as sugar, they can almost be explained as sugar. For many people, this made it much easier to understand how classes relate to the rest of the language.

[rdking]

@erights You said:

For various reasons I understand but regret, that didn't fly.

Can you elaborate?

[hax]

While I was technically wrong about that being impossible, I think this line of thinking led to the right intuitions.

@erights

Well, it's strange that a wrong understandings could lead to a right intuitions...

The four encapsulation mechanisms you mentioned:

1. lexical closure
2. WeakMaps
3. Proxy
4. Modules

As my understanding, only the last one (modules) is designed to be a encapsulation mechanisms. The others are just happened to be able to be used to hide something. For example, Map could also be used to store something related to instances without leak to others, even it will prevent GC. Is it a fifth encapsulation mechanism?

So I don't think number 4 or 5 is relevant here. The most important thing I believe, is how a mechanisms fit for real programming. Currently WeakMap based solution has significant gotcha when use with a common usage of Proxy in the ecosystem (even such usage is not match the original intention of Proxy). On the other side, private symbol do not have such problems and match the mental model of most programmers --- it's just a symbol without reflection functionality, nothing new to learn.

[erights]

As my understanding, only the last one (modules) is designed to be a encapsulation mechanisms. The others are just happened to be able to be used to hide something.

1. Doug Crockford, Allen Wirfs-Brock, Pratap Lackshman, and I, designed EcmaScript 3.1 that became EcmaScript 5. As part of that, we designed strict mode mostly motivated to repair lexical scoping and to make closures true encapsulation mechanisms. See https://www.youtube.com/watch?v=Kq4FpMe6cRs&list=PLKr-mvz8uvUg70w0yKGfytaDqxiIBNo_L
2. and 3. Tom Van Cutsem and I designed WeakMaps and Proxys primarily to support membranes. Membranes must be impenetrable and as transparent as possible. This means they encapsulate each side from the other, and encapsulate the internal mechanisms of the membrane from both. See https://ai.google/research/pubs/pub40736 and https://ai.google/research/pubs/pub37741 and https://ai.google/research/pubs/pub36574

For example, Map could also be used to store something related to instances without leak to others [...]. Is it a fifth encapsulation mechanism?

How so? Can you give an example?

OTOH, I agree with your general point: there are important encapsulation principles we purposely designed into some of the other abstraction mechanisms. Promises separate the right to cause resolution (resolver, rejector) vs the right to obtain the settled resolution (promise via then). This is not fundamental in that many of the user-level promise libraries that preceded it did so with normal object encapsulation: either WeakMaps or lexical closures. Promise does not provide a fundamental new kind of encapsulation. Promise is a built-in, but can be explained as using the existing mechanisms.

I admit this distinction is not crisp. Modules can mostly be emulated as rewrite into closures, where closure encapsulation implements module encapsulation. Does this make it not fundamental? The preceding module proposal, which became CommonJS modules (require/exports) does precisely this by design. The one way in which standard EcmaScript modules are not naturally understandable in terms of equivalence to lexical closures is so-called live bindings, which in retrospect I think was a terrible mistake.

On the other side, private symbol do not have such problems

I understand the arguments pro and con. They both have valid points. I am not interested in re-litigating this in the absence of significant new information.

[rdking]

@erights I'm not interested in re-litigation either. I'm merely seeking understanding. What was it about closures that "didn't fly"?

[erights]

@rdking said:

@erights You said:

For various reasons I understand but regret, that didn't fly.

Can you elaborate?

The objects-as-closure pattern, without novel virtual machine support, costs a closure allocation per method per instance. To express the equivalent of a class that has X methods and Y instances requires allocating X*Y closures. One of the motivations of the early class proposals was not just to provide syntactic sugar for the pattern, but to give implementations the opportunity to implement it without this overhead, and without needing to recognize a pattern buried in normal code.

Implementors argued that they were unwilling to engineer a significant new optimization path that was not simply a substantial reuse of the difficult optimizations already present. I think they over-estimated the implementation burden and under-estimated the benefit. I regret not pushing us to better understand the actual costs and benefits before giving up on these designs. I think we would have been better off than we are with standard classes --- even though I am very proud overall of our standard classes!

[rdking]

@erights

The objects-as-closure pattern, without novel virtual machine support, costs a closure allocation per method per instance.

That doesn't track. It shouldn't be necessary for each class method to have a unique closure allocated to it. In fact, at least in my mind, that's counter-productive. It should only require 1 closure per instance object to support private data encapsulation.

A function is an object. A function created and returned from another function carries the closure of the constructed function with it. If the [[Call]] mechanism is disabled for a function, it is effectively a mere object. Did the idea of carrying the constructor's closure on the instance object never occur to any of you? Doing so would have allowed you to use the [[Call]] mechanism to attach the closure carried by the instance object to the execution context of class methods prior to the method's own execution context. Further, with a single operator, it would've also been possible to access the closure directly. I wrote a proposal around this very concept, but I don't think anyone in TC39 ever took it seriously.

[rdking]

@erights

I regret not pushing us to better understand the actual costs and benefits before giving up on these designs. I think we would have been better off than we are with standard classes...

I regret it as well. For as proud as you may be over what has been achieved (and don't get me wrong; it is definitely a significant achievement), the trade offs that needed to be made are great enough that the viable number of use cases for what's being pushed is less than half when compared to the viable number of use cases for a closure-based solution.

[hax]

@erights I understand that the original designer have the authority of explaining the design goals of the features. What I'm trying to explain is such features were not introduced to programmers by books/manuals for such abstract goals. So there are gaps between you and me (assume I could represent normal programmers better because you designed them and I learned them.)

For example, when introduce WeakMap, most books/articles will tell you it's useful because normal Map will prevent GC. They rarely explain it as a "encapsulation mechanism". Even some article introduce it could be used to implement "private" for js (I myself wrote several such articles), it's more like a trick (because of various reasons like implementation performance, inconvenience of boilerplate code, etc.), compare to some other method, like TS private which do not have such problems and explicitly introduced for "encapsulation mechanism". The old closure based private is also have the similar impression like WeakMap.

How so? Can you give an example?

Just replace WeakMap with normal Map. Obviously it's flawed in all the cases which need to consider GC. Actually I feel GC issue of Map vs WeakMap is a bit like hard private issue of Symbol vs PrivateSymbol.

I am not interested in re-litigating this in the absence of significant new information.

It's not my intention to advocate private symbol again in this thread, what I really want is explaining the mental model of normal programmers to the language designers.

[ljharb]

They’re effectively the same (modulo GC) if the map isn’t exposed - but if you pass it around, the Map allows arbitrary mutation of anything, where the WeakMap only allows it if you already have the key.