Turning "hard private" into "soft private"

[jwalton]

In which we provide a fairly trivial way to bypass "hard private" protections
on third party libraries, in practice.

Is this really "hard private"?

It seems like a lot of decisions in this proposal concerning private properties have been made specifically to support "hard private" properties. There was a long discussion here about whether this was a good idea or not, which didn't really seem to come to a consensus. TC39 is never the less proceeding with "hard private", and I see a decisions being justified using the argument "the alternative wouldn't really be hard private." This is stange to me, because the current proposal isn't hard private (at least in practice), and I suspect creating hard private properties is impossible in JavaScript.

What is "hard-private" anyways?

To quote @ljharb:

If it's accessible, it's public. "soft-private" is no different than underscores, because both are a convention (albeit, the former would be a convention baked into the language).

If a user can do it, they will, and if you break them, you broke them, even if they did something unwise.

(We can have a whole argument here between the "programming by contract" people, and the "language purist" people, about whether that second statement is correct or not - in fact we did in tc39/proposal-private-fields#33, so let's not have it again. 😉)

Obviously, nothing is really hard-private. I mean, the contents of that private variable are somewhere in memory, and I'm sure I can find some interesting clever way to get at them. I can write my own javascript engine, or write a node.js native module that peeks at memory, or I can use some esoteric attack like rowhammer to exfiltrate that data. What we really mean by "hard private" is that, in a "normal" javascript environment (not a debugger, and maybe in a browser), you can't get access to the contents of a private member without doing something extraordinarily.

The motivation for hard private seems mainly (?) to be for library authors; so let's say if a user can access private members in your library "easily", it's not hard private.

So, how "hard private" is this proposal?

PoC||GTFO

Bob writes an npm module:

export default class MyClass {
    #secret = "You can't see me!";
}

Alice is using Bob's npm module, and realizes she wants access to that #secret variable. I mean, it's named with a hashtag, and hashtags are for sharing, right? (And, let's be honest, this is a pretty boring NPM module. What else is Alice going to do with it?)

const inst = new MyClass();
console.log(inst.#secret);  // No worky worky.  Sad tombone.

In order to get around this, Alice adds the following to her .babelrc.js:

module.exports = {

  ...

  overrides: [{
    test: ["./node_modules/bobs-package"],
    plugins: ["private-class-fields-to-public"]
  }],
}

babel-plugin-private-class-fields-to-public is a Babel plugin which transforms Bob's package to look like this:

export default class MyClass {
    #secret = "You can't see me!";
    get _secret() { return this.#secret; }
    set _secret(secret) { this.#secret = secret; }
}

Now Alice can write:

const inst = new MyClass();
console.log(inst._secret);

Mischief managed!

Wait wait. Isn't this cheating? Babel isn't a "normal javascript environment!"

Between @babel/core and babel-core, babel was downloaded around 11 million times in the past week. Not too many javascript programs these days don't go through babel. You could even run babel in a browser, and transpile third party modules on the fly. babel-plugin-private-class-fields-to-public plugin was written in typescript, which for a while was a big babel competitor, but even that plugin was compiled with babel. Seems like a normal javascript environment to me.

Even if you don't buy that argument, look at this from a practical standpoint:

In Python, private members are about as soft-private as you can get. Private members are members that start with an _. To get around this, you need to type the name of the variable, and maybe feel bad for a few moments.

Java, on the other hand, is still soft-private by the "if it's accessible" definition above, but you have to do some work to access a private member. You have to go look up the refletion package and figure out how it works again (since the last time you used it was probably when you last had to get around a pesky visability problem). Then, when it doesn't work, you need to go to stack overflow to find out you forgot to call setAccessible().

With this proposal, as it stands, accessing private members in JavaScript falls somewhere between these two; you need to install a babel plugin, but after that it's basically "type _ and feel slightly guilty". Let's put it somewhere between Python and Java; maybe not as "hard private" as all that.

[ljharb]

Hard private is not “defeated” if you edit the source - either manually or programmatically. The concern is about at runtime - if the code shipped to the engine has encapsulation, then it will be reserved (just like with variables inside a closure).

[jwalton]

Really? I admit that installing a babel plugin might be, as you put it "unwise", but if a user can do it, they will, and here they certainly can. At the end of the day, JavaScript is an interpreted language; if you want to share a module with me, you need to share the source, and I can infer the existence of private variables by reading that source. So can my program. I can even write a loader that reads in your module and does this at execution time.

(Anyways, hopefully I at least got a grin out of you with my hashtag comment. ;)

[ljharb]

If they’ve done that, they’re not actually using the feature. Actually using the feature - shipping the code using private fields that are not intentionally exposed - results in “hard private”, which is what matters.

[jwalton]

Isn't this is like saying "If you type otherObject._something in python, you're not using the features in PEP 8 - actually using the feature, and not typing the _, results in hard private"?

I mean, the "feature" here is a feature for library developers to prevent library users from accessing private members in their library. It's not really a feature for library consumers at all. If there's a simple way for library consumers to get around that protection, I'd argue that it's not "hard private".

[ljharb]

A source rewriter isn’t a simple way, and most importantly, it doesn’t work at runtime.

[jwalton]

People will have the source rewriter installed if they even want to use a module that uses private variables, at least for the next several months.

And why do you think it doesn't work at runtime? Babel is just a javascript program; it's as simple as replacing your import with:

    const moduleSource = await fetch('https://domain.com/bobsPackage.js');
    const transformed = babel.transform(source, {
        plugins: [privateToPublic],
    });
    const bobsPackage = eval(transformed);
    console.log(bobsPackage._secret);

It even works in a browser.

[robpalme]

@jwalton Thanks for the entertaining insight. It's good to highlight this.

My view is that it all comes down to boundaries. If you don't define boundaries, then you can make arguments that any encapsulation is not truly "hard" (not a true Scotsman) unless it's on an air-gapped machine that can only be communicated with via unequally-laden carrier pigeons.

The boundary of ECMAScript is the source text ingestion. So it's fair to say that details are not abstracted away from anyone that can tamper with code prior to ingestion. Therefore any judgement of whether something is truly encapsulated by ECMAScript should be assessed post-ingestion.

A secondary and lesser point: Real-life use of fields will tend to go through minifiers that will mangle private names in ways that (underscore-prefixed) properties cannot safely be mangled. This will act as a deterrent to casual/widespread use of the babel-plugin-private-class-fields-to-public approach because mangled names are harder to deduce and can change frequently.

A final and least important point (because it all just flows back to top-level philosophy): as an electrical OEM who manfactures computer PSUs locked by uncommon star-shaped screws and covered with "danger of death" stickers, changing the internals of the PSU is a reasonable thing to do. If a user gets inside and has a bad experience because they didn't understand the updated internals, we all know who is at fault. If that same PSU just has the stickers but no physical lock, it would be more of a grey area and some (in places with consumer regulations) would argue it's an accident waiting to happen. The Babel plugin is an invasive power-tool akin to a star-shaped screwdriver IMO.

[jwalton]

@robpalme Thanks for the insightful comments. These are all fair points.

And, one deterrent to the use of babel-plugin-private-class-fields-to-public that you missed, at least in the short term, is that most libraries published to npm have already been through babel once, and don't have their original es6 source published; if your private fields have already been turned into a WeakMap, then there's no way for my little babel plugin to find them.

But, in reference to your last point; I would point out that in, say, Java, once you break out the reflection API, you're pretty clearly tearing through "danger of death" stickers. You have very definitely violated the contract put in place by the author of the module you're using, and you know that if bad things happen later on, it's totally your fault. So I would call Java a "hard private" language, if this is where we're drawing the line. (Is Python like one of those cheap "Warranty Void If Removed" stickers that's already started to peel up at the edges when you take the product out of the box?)

But we software types (as a generality, and like all good generalities it's not always true) tend toward the tinkerer type - the sort of people who think up a pretty funny joke about hashtags and private members, and then spend an hour learning how to write a babel plugin just so they can make that joke. Folks like us are going to break out the power tools and star shaped screw drivers from time to time.

Ultimately, I guess, the point of this issue is, if we're going to make design decisions and say "we will make this extra complicated in service of the very important goal of hard privacy," but then we don't actually get the benefits hard privacy, was that complexity worth while? The answer to that question - where and what tradeoffs are "worth it" - is going to be different for everyone, of course, and maybe it is worth it. But hopefully I've given people something to think about over the holidays; once this makes it to stage 4, it's forever.

[littledan]

I'd say that running a Babel transform like this is a way of forking your dependency, which is a reliable way to get at private state. The key is, when forking a dependency, you are taking on the burden of maintaining that fork. By contrast, many large JS projects have had to roll back changes when there were too many compatibility complaints from people accessing "internal" APIs. I'd say this is a qualitative difference.

[jwalton]

@littledan I understand the point you're making, but I think the line you're drawing here is a little murky, and is only going to get less clear as time goes on. Babel added the overrides keyword to their config specifically to make it easy to transpile dependencies.

In our product, we're already transpiling the very popular debug library because they now publish code with es6 features to npm. Our toolchain for an older part of our product has an ancient i18n string extractor, written by a former co-op, which is totally unmaintained and breaks on the const keyword, so we transpile debug all the way down to ES5, so it doesn't crash our i18n preprocessor. We don't even rely on debug directly - our dependencies rely on debug, but that's enough to get the code into our bundle. (Wow, I really need to fix that. That's something I should work on while I'm at my in-laws for the holidays.)

But my point is, there are a lot of advantages to transpiling dependencies, and it's only going to get more popular to do so as time goes on.

And again, to look at this from a purely practical standpoint; if there's a library foo, and foo's users constantly violate it's privacy restrictions, that's a pretty good indicator that the author of foo is perhaps being a bit too restrictive in what they expose. If, in order to be useful, users of foo need to violate privacy restrictions, then they will figure out a way to do so. Life finds a way. If people are violating the privacy restrictions en-masse, and the package maintainer makes a change but it breaks the Internet, Wreck-it-Ralph style, then on that day when we are all huddled in our cold, dark homes, unable to control our IoT thermostats and lights, when Alexa no longer fills our orders for milk and we are unable to feed ourselves, will it really matter if we can say "This is our own fault, because we didn't have hard private variables in JavaScript, and someone posted a popular stack overflow answer that told people to use foo._widget!" Or will we somehow be saved if we can say "We gave the world beautiful perfect hard privacy. It's totally not our fault that someone posted a popular stack overflow answer that told people to install a babel plugin and /then/ to use foo._widget"?

[superamadeus]

Unintentionally depending on a library's "soft-private" api is easier than you'd think. For example:

My software depends on [email protected].

// node_modules/[email protected]

class Library {
  /* ... */

  getBooks() {
    return [ /* ... */ ]
  }
}

// index.js

class MyLibrary extends Library {
  getBooksByAuthor(author) {
    return this._getBooksWithFilter(b => b.author === author);
  }

  _getBooksWithFilter(filterFn) {
    return this.getBooks().filter(filterFn);
  }
}

Ooh, [email protected] was released! I'll upgrade because features I want have been added in a non-breaking way (according to semantic versioning).

// node_modules/[email protected]

class BookFilter {
  applyFilter(books) {
    return books.filter(/* ... */);
  }
}

class Library {
  /* ... */

  getBooks() {
    return [ /* ... */ ]
  }

  _getBooksWithFilter(filter) {
    return filter.applyFilter(this.getBooks());
  }
}

Suddenly my software is not working as expected. How is this possible? It was a minor version change! This library is unpleasant!

[zenparsing]

@superamadeus This is a good use case for symbols.

[superamadeus]

@zenparsing Alternatively, private fields.

Edit:

// node_modules/[email protected]

class BookFilter {
  applyFilter(books) {
    return books.filter(/* ... */);
  }
}

class Library {
  /* ... */

  getBooks() {
    return [ /* ... */ ]
  }

  #getBooksWithFilter = (filter) => {
    return filter.applyFilter(this.getBooks());
  }
}

[ljharb]

Note that the semver spec is a bit ambiguous about whether that change can be semver-minor - are symbols and underscores part of the documented API or not? If by “documented” you mean “what i can see in the repl”, yes - if you mean “what’s written in prose in another place” then probably not. I interpret it in the former way, to protect my users from any unintentional breakage; some authors are less cautious. Private fields make this unambiguous.

[zenparsing]

In my opinion symbols are a better fit for the library use case.

Why?

• They interoperate better with existing language features (like Proxy and Reflect).
• They provide enough friction to discourage most "mucking about with" scenarios.

[superamadeus]

@zenparsing

I appreciate your opinion and see where you're coming from. That said, I think both options (symbols vs. private fields) are viable and would require forethought about the tradeoffs. But neither solution will work for 100% of the use cases.

In my opinion symbols can get verbose:

// private field
class Service {
    #myMethod1 = () => {};
    #myMethod2 = () => {};
    #myMethod3 = () => {};
    #myMethod4 = () => {};
    #myMethod5 = () => {};
    #myMethod6 = () => {};
    #myMethod7 = () => {};
    #myMethod8 = () => {};
    #myMethod9 = () => {};
}

// symbols

const myMethod1 = Symbol();
const myMethod2 = Symbol();
const myMethod3 = Symbol();
const myMethod4 = Symbol();
const myMethod5 = Symbol();
const myMethod6 = Symbol();
const myMethod7 = Symbol();
const myMethod8 = Symbol();
const myMethod9 = Symbol();

class Service {
    [myMethod1] = () => {};
    [myMethod2] = () => {};
    [myMethod3] = () => {};
    [myMethod4] = () => {};
    [myMethod5] = () => {};
    [myMethod6] = () => {};
    [myMethod7] = () => {};
    [myMethod8] = () => {};
    [myMethod9] = () => {};
}

For the record, I'd rather use private methods (#myMethod() {}).

[jwalton]

@superamadeus This is a really helpful example. For this particular example, you could make getBooksWithFilter(self, filter) a top-level function and just not export it. But, I can certainly think of cases where I have a function that I want to be a bound member function, like say a React event handler where I want a per-instance bound function so I can take advantage of PureComponent. And there you are quite likely to have name collisions like this, because people tend to name their functions onSubmit or other such boring things. :)

[jwalton]

(And, of course, data instead of functions. Don't want to be stashing member data in global variables.)

[zenparsing]

@superamadeus That's why we ought to have sugar over the symbol-internal-to-module pattern.

If we had sugar over that pattern, and we assume that that pattern works for the library use case, then what use cases are left for hard private?

• Polyfilling spec things
• ?

[jwalton]

This made me think about this problem in a very different way. It sounds like what we really want here isn't "private" variables, but "class scoped per-instance variables". But we already have keywords for declaring block-scoped variables in JS; const and let:

class MyLibrary extends Library {
  // Declare a per-instance variable that's only accessible within these {}s.
  let getBooksWithFilter = (filterFn) => {
    return this.getBooks().filter(filterFn);
  };

  getBooksByAuthor(author) {
    // Note that `this._getBooksWithFilter` is undefined here.
    return getBooksWithFilter(b => b.author === author);
  }

}

That would make this kind of a unique-to-javascript concept, but the # private thing is also kind of a unique-to-javascript concept. And it de-weirds things considerably - there's no this.#foo !== this['#foo'], for example. The one downside this would have over the # version is that you can't easily access these variables from other instances of the same class.

[superamadeus]

@jwalton I don’t really see what this gains. The syntax with the # sigil is jarring at first because it’s new syntax but it’s very consistent if you don’t try to compare it to typical property access.

In the example you’ve shown, it is hard to tell at call site if you’re using a private member vs a global variable. I personally like knowing when I’m making a private member access.

Edit: but yeah, I think the private field mechanism is very similar to “class scoped per-instance variables”. I don’t see how what you’ve described makes any gains on the current proposed syntax.

[hax]

The one downside this would have over the # version is that you can't easily access these variables from other instances of the same class.

@jwalton This can be solved by introducing a new operator like other::x, or just use non-sugared form other[symbol] if it's symbol-based.

[superamadeus]

@zenparsing @hax It was my (assumed) understanding that the committee had considered a symbol-based approach, and declined it. I can't find any documentation on this though. Anyone have any info on this?

That said, I still fail to see what you gain from the symbol based approach that you don't get in the current private field spec.

I'd love to see some examples of cases where the symbol approach provides benefits over the current private field spec, with respect to hard-private fields. I'm having trouble getting there on my own.

[jwalton]

@superamadeus There's one thing that's always bugged me about the current private property proposal:

class Library {
  #books = ['Snow Crash'];
  listBooks() {
    return this.#books.slice();
  }
}

const library = new Library();

library.addBook = function() {
  this.#books.push('Mistborn'); // Nope!
}

I'm told this doesn't work. Even if you define new functions on Library's prototype, they can't access the private members of Library. That's really odd. I can't think of another language that's like that; private means not only private to the instance, but private to scope as well. The let syntax is maybe functionally nearly identical, but I think semantically it expresses the intent much better.

It's very natural to try to compare # to natural property access, because it looks like a property access, when really it isn't.

[ljharb]

@zenparsing your conclusion rests on soft private actually being sufficient or good for libraries by default - my experience tells me that it is not, even if maintainers initially believe that it is.

[zenparsing]

@superamadeus

I'd love to see some examples of cases where the symbol approach provides benefits over the current private field spec, with respect to hard-private fields. I'm having trouble getting there on my own.

@jwalton points out one benefit: with symbols you don't need to cram everything that references the property into the class body. You can put the functions where they make sense.

Another advantage: with symbols you can trivially wrap an object with a Proxy and it works. With private fields, you have to create a complex membrane around it in order to avoid TypeErrors.

Another one: with symbols you can easily create a generic deep clone by using Reflect methods (like Refect.ownKeys). With private fields, you have to embed cloning logic directly in the class body, for each class.

Or imagine trying to write your own "object inspector" in normal JS. With private fields you can't get to the data values to display them.

[zenparsing]

@ljharb Can you elaborate on why symbols aren't good enough for libraries?

[ljharb]

@zenparsing any time something is accessible externally, it increases the api surface. Symbols are still accessible, and if it’s possible to depend on them, people will (and do).

I have dealt with tons of issues on many libraries i maintain due to people depending on things that weren’t intended to be part of the “public api”; in practice I’ve found that the only way to avoid both user breakage and maintainer support burden is to restrict what is possible, not to just create a false sense of security by “hiding them better”.

[ljharb]

For an object inspector; I’d argue that that private data should never be displayed, just like you can’t display the closed-over variables that a function uses.

[mbrowne]

The second category is relatively rare

Just to clarify what I mean by "rare", I mean the average developer might never encounter a need to do this. It's more likely that they would use a library that would use reflection under the hood. For example, if something like breeze.js wanted to make full use of JS classes in a future version for defining entities, the example use case I mentioned in tc39/proposal-decorators#191 might be important. Or maybe some extension to Apollo client or Relay that allowed you to deserialize your GraphQL data to full-fledged objects with fields and methods defined in JS classes (currently such libraries give us the data as plain objects, but no methods). And obviously while only a relatively small number of developers are involved in maintaining those libraries, their usage is anything but rare.

Others could probably give other example use cases for reflection on private fields; this is just the use case I'm most familiar with.

[superamadeus]

This is an off topic comment about a shorthand syntax for symbols. Please disregard.

Sorry I haven't gotten around to responding much. A lot to take in.

@mbrowne

@rdking The semantics of symbols are certainly sufficient for soft private. I think the two best options for providing it in an ergonomic way are either more concise syntax for symbol-keyed properties, or decorators on classes and/or individual private fields.

I always pictured there would be a shorthand syntax for symbols. And a way to define context-unique symbols. This would be cool:

class Example {
    [:symbolProp] = "hello";
    [::classSymbolProp] = "world"
}

class Test extends Example {
    [:symbolProp] = "yo";
    [::classSymbolProp] = "peeps"
}

// desugared:

class Example {
    [$_1] = "hello";
    [$_2] = "world"
}

const $_1 = Symbol.for("symbolProp");
const $_2 = Symbol.classSymbolFor(Example, "classSymbolProp");

class Test extends Example {
    [$_1] = "yo";
    [$_3] = "peeps"
}

const $_3 = Symbol.classSymbolFor(Test, "classSymbolProp");

console.log($_2 === $_3) // true

Where Symbol.classSymbolFor would resolve common symbols for classes in the same heirarchy.

[rdking]

Ok. So I buy the desire for more ergonomic shorthand around Symbol. Isn't this a problem for later? We already have the ability to do soft-private using Symbol. Creating a shorthand for it should be a proposal all it's own. From what I see, this doesn't have any bearing on what syntax is used for private data.

[superamadeus]

Yes, it was completely off topic. I'm sorry.

[mbrowne]

I think there's a case to be made (and perhaps @zenparsing already proposed this to the committee...in any case I'm sure someone has suggested it) for # to desugar to public symbols and not worry about hard privacy. Obviously that was rejected and is completely unacceptable to some stakeholders. Alternatively, the private symbols proposal could be used to achieve hard privacy for the use cases that require it. (This assumes that some lingering complaints about private symbols could be resolved as @lgmat suggested in other threads. But committee members arguing against private symbols as the solution for hard private weren't only concerned about proxy issues, but the overall semantics; see #178 (comment).)

Regardless of all of this, it seems the majority opinion of the committee is that hard private is a better default for private fields anyhow. Personally I don't have a strong opinion about what the default is for private fields as long as both hard private and soft private are ultimately available with ergonomic syntax.

[Igmat]

Creating a shorthand for it should be a proposal all it's own. From what I see, this doesn't have any bearing on what syntax is used for private data.

@rdking from what I've seen so far - shorthand syntax for all kinds of Symbols (public and private) could become a deal-breaker, which may move committee from current proposal to Symbol.private proposal.

Previously I was focused on solution for classes only, but later I got an idea about shorthand for all Symbols in any place. Further conversation with @zenparsing and his small investigation in this field moved me further in this direction.
In addition there was an answer from one of committee members (@jridgewell) in #183 (comment) that syntax for Symbol.private is like 90% of a requirement.

Since #183 shows that we can preserve most important Membrane invraints with Symbol.private (and I'm going to finish with has checks (that aren't preserved yet) tomorrow), there is only one thing left: ergonomic syntax. And ergonomic syntax for all symbols and not only private ones is one of viable solutions that may replace existing syntax for private fields/properties.

[rdking]

@Igmat If it meant abandoning this excessively problematic proposal which trades away more value than it brings, I'd be all for researching syntax for simplifying Symbol declarations. @zenparsing's "investigation" presented some good ideas. The syntax would have to be different since it seems @ is reserved for decorators, but the semantics are spot-on. If a syntax were contrived that removed the risk of leaking the private symbols, then that proposal would have removed the only real problem I have with it.

[littledan]

OK, I think we've addressed the concerns raised in this thread:

• As I explained in Turning "hard private" into "soft private" #189 (comment), this proposal makes a guarantee against code that's unmodified; it's always possible to fork. Let's stay engaged in educating the developers on the benefits of not forking, and continue providing strong guarantees for unmodified code.
• We discussed above what guarantees this proposal provides with respect to "brand checking". I've tried to clarify brand checking in this FAQ entry. I think this issue comes down to ES6 dynamic class hierarchies, and we can resolve it by education.
• @zenparsing has written up a gist going into the symbols sugar concept. Comments on that gist would be a good way to continue development; maybe it would graduate into its own repository at some point.
• For private symbols, let's continue the discussion in Private fields/properties & the membrane pattern... #158 .

For these reasons, I'm closing this thread.

[mbrowne]

@littledan @bakkot @zenparsing @lifaon74 @rdking @Igmat I would like to follow up on the discussion we were having about hard vs. soft private in #203, and this thread seems like a good place for it. I realize this issue in general has been discussed at great length in many threads over the past few years, but I think it's still worth discussing the consequences of shipping class fields while decorators are still in development, vs. waiting and releasing both together.

IMO it should be fine to go ahead and ship class fields without waiting for decorators. (BTW, let's keep this thread on topic; obviously there are other objections to this proposal moving forward besides the hard private aspect.) As I pointed out in #203, Babel makes it possible to use both class fields and decorators today, and not all users need soft private or protected. In the future, for unmaintained libraries that might need to be patched in a way that requires access to private fields, there's always the option of forking, which is probably a good idea anyhow if the lib is no longer maintained.

I didn't form this opinion lightly, and I do have concerns about defaulting to hard private (especially in the absence of equally or nearly equally concise syntax for soft private, which we may or may not get with decorators and/or a proposal to add sugar for symbols that isn't yet stage 0). I see it as a grand experiment in some ways; I can't think of any other language with classes that doesn't provide some form of reflection (enabled by default) for private members, so regarding classes specifically it seems we're going into uncharted territory. On the other hand, JS already has closures and modules, and there are plenty of libs that already hide implementation details and internal variables. Still, truly inaccessible private object state is something new, and I think it's hard to predict with 100% certainty whether the effect of hard private fields will be more positive or negative for JS users as a whole compared with soft private.

But in other languages like Java that have reflection by default, obtaining the source code of your dependencies and forking them is often not an option. With JS it's obviously different, and hard private also brings a big benefit to maintainers of popular libraries who want to ensure that users can't depend on implementation details—an important ask from library authors for reasons that have been discussed here often. There is of course the risk that some developers (including library developers) authoring classes won't fully think through the consequences of using private fields without providing sufficient hooks for customization, but at least for open-source libraries, I think the open-source process (including the option to fork) will address this.

Getting back to the question of release dates, it's important to consider that the longer it takes for this proposal to be finalized, the longer it will take before class fields can be used in all modern browsers without the use of a transpiler. There are many people who would like that day to come sooner rather than later, even if they have to wait longer for decorators to also be natively available. And unless the feature of hard private by default were dropped entirely, I don't see how waiting until decorators are ready would be helpful to those who want to be able to monkey-patch things either.

[rdking]

@mbrowne I don't know what benefit there is in discussing this further. I've grown quite weary from frustration, not because things didn't go the way I wanted, but rather because the information I desire to help me understand the perplexing decisions that were made isn't forthcoming. That's not to say they didn't try. But that makes it more frustrating, realizing there's a gap somewhere, but not understanding how to bridge it..... But seeing as this is probably a "just for fun and edification" type conversation, I'll join.

---

There are so many misunderstandings in that 3rd paragraph:

I didn't form this opinion lightly, and I do have concerns about defaulting to hard private (especially in the absence of equally or nearly equally concise syntax for soft private, which we may or may not get with decorators and/or a proposal to add sugar for symbols that isn't yet stage 0).

Yes, soft private definitely could use some syntax sugar. There's no 2 ways about that. However, soft private is still so much easier to set up than hard private as to be somewhat ridiculous. Here's the same class, 1 soft, and 2 hard.

//Soft Private
let SP = (() => {
  const data = Symbol("Private:data1");
  const data2 = Symbol("Private:data2");
  return class SP {
    constructor() {
      this[data1] = 42;
      this[data2] = Math.random() * Math.PI;
    }
    print() {
      console.log(`My private data1= ${this[data1]}`);
      console.log(`My private data2 = ${this[data2]}`);
    }
  };
})();

//Hard Private: memory conservative
let HP1 = (() => {
  const pvt = new WeakMap;
  return class HP1 {
    constructor() {
      pvt.set(this, {
        data1: 42,
        data2: Math.random() * Math.PI
      });
    }
    print() {
      let p = pvt.get(this) || {}
      console.log(`My private data1= ${p.data1}`);
      console.log(`My private data2 = ${p.data2}`);
    }
  };
})();

//Hard Private: Babel approach
let HP2 = (() => {
  const data1 = new WeakMap;
  const data2 = new WeakMap;

  function getPrivate(key, map) { return map.get(key); }
  function setPrivate(key, map, value) { map.set(key, value); }

  return class HP2 {
    constructor() {
      setPrivate(this, "data1", 42);
      setPrivate(this, "data2", Math.random() * Math.PI);
    }
    print() {
      console.log(`My private data1= ${getPrivate(this, "data1")}`);
      console.log(`My private data2 = ${getPrivate(this, "data2")}`);
    }
  };
})();

Soft private is already less complex to setup and maintain than hard private, is faster, and costs less memory as well. Something needs to be done to improve hard private. Hence the need for the language-based upgrade.

I can't think of any other language with classes that doesn't provide some form of reflection (enabled by default) for private members, so regarding classes specifically it seems we're going into uncharted territory.

I don't know how long you've been programming, but C++ and Java both existed for several years without that ability, and we're no worse off because of that. What we learned back then was that it's better not to rush through planning your code design, something that modern developers seem to have not been taught. This is old territory, and well mapped.

Still, truly inaccessible private object state is something new, and I think it's hard to predict with 100% certainty whether the effect of hard private fields will be more positive or negative for JS users as a whole compared with soft private.

Run the code above in Chrome debugger and see if you can figure out how to access (or even see) data1 & data2 in HP1 & HP2 instances. Truly inaccessible private object state is as old as factory functions, because that how we used to do it before WeakMap and class. Done well, a new syntax for hard private is a plus, as much or more so than a new syntax for soft private. What's missing is an approach that isn't so riddled with complexities and issues.

Getting back to the question of release dates, it's important to consider that the longer it takes for this proposal to be finalized, the longer it will take before class fields can be used in all modern browsers without the use of a transpiler.

It's important to consider that class fields will still require use of a transpiler to make use of decorators to fix some important and fairly common use case problems even after being released. If you've got a flat tire with 6 fixable holes, do you rush out to go fix 1 hole? No. you wait until you can afford to either fix all 6 or buy a new tire.

And unless the feature of hard private by default were dropped entirely, I don't see how waiting until decorators are ready would be helpful to those who want to be able to monkey-patch things either.

I see it more like this: if private fields were removed from class-fields, then it wouldn't be so bad to release it now. Not good, just not nearly so bad. Sure it has issues, but they are mostly easily mitigated if you're aware of them. So I wouldn't have sufficient reason to want it delayed. The real thing worth having in this proposal is private fields, though. Unfortunately, the formulation is simply no good. While it has no directly observable effects, it has far too many observable side-effects. As such it does not fill the need. That has nothing to do with whether or not it's hard or soft private, though.

[littledan]

As explained in other threads, TC39 doesn't tell anyone when to ship. Class fields advanced to Stage 3 more than a year ago, and I'm trying to get decorators to Stage 3 as soon as possible. (I was hoping to work on decorators all day today, but ended up spending much of the time on this repository.)

[ljharb]

@rdking

Maybe you haven't been programming long enough.

This is an inappropriate and uncharitable comment; please review our Code of Conduct. It would be a wise idea, and a show of good faith, to self-moderate your comment, so that it doesn't need to be hidden. Thanks!

[bakkot]

@rdking, "Maybe you haven't been programming long enough." comes across as really dismissive. If what you meant to say is "I believe you are mistaken on a technical question about the history of other languages", please say that rather than disparaging other people's experience. It is not always the case that people who are inexperienced are mistaken or that people who are experienced will necessarily agree with you, and even when correcting someone you should avoid implying they'. (As a reminder, this repo is governed by TC39's code of conduct.)

[rdking]

@ljharb My intent was not to be uncharitable. I was simply giving @mbrowne benefit of the doubt as I do not know his age or how long he's been a software developer. No slight was intended with that comment. If there was one, I apologize.

[littledan]

There's so many assumptions here, I don't know where to start!

[rdking]

A side note to all about me: I tend to word things very... bluntly. Feel free to call me on it if it's too much. Even after 40+ years, I'm still working out the kinks. In either case, there's no need to "read between the lines" with me because I like putting everything on the line. (Double entendre intended.)

[rdking]

Edited. Does that work better?

[mbrowne]

@rdking

soft private is still so much easier to set up than hard private as to be somewhat ridiculous. 

I am well aware of how hard and soft private can be implemented today. When I said, "I do have concerns about defaulting to hard private", I was talking specifically about the new `#` syntax introduced by this proposal and how it should behave natively, not how you would polyfill it. I'm not sure if you were referring to polyfilling or something else, but in any case you seem to have misunderstood my point.

Truly inaccessible private object state is as old as factory functions

I am aware of this as well (and this is one of the things I was thinking of when I mentioned closures). I didn't mean it so literally...I was saying that private class fields with hard private semantics--with native support for ergonomic syntax--is something new. That's significant, because the verbose syntax probably dissuades a lot of people from using WeakMaps today even though they achieve the same goal. So I think there will be a lot more people using private fields than there are who are currently willing to go to the trouble of using WeakMaps (not to mention performance and bundle size).

…

On January 8, 2019 4:25:31 PM EST, Ranando D Washington-King ***@***.***> wrote: @mbrowne I don't know what benefit there is in discussing this further. I've grown quite weary from frustration, not because things didn't go the way I wanted, but rather because the information I desire to help me understand the perplexing decisions that were made isn't forthcoming. That's not to say they didn't try. But that makes it more frustrating, realizing there's a gap somewhere, but not understanding how to bridge it..... But seeing as this is probably a "just for fun and edification" type conversation, I'll join. --- There are so many misunderstandings in that 3rd paragraph: > I didn't form this opinion lightly, and I do have concerns about defaulting to hard private (especially in the absence of equally or nearly equally concise syntax for soft private, which we may or may not get with decorators and/or a proposal to add sugar for symbols that isn't yet stage 0). Yes, soft private definitely could use some syntax sugar. There's no 2 ways about that. However, soft private is still so much easier to set up than hard private as to be somewhat ridiculous. Here's the same class, 1 soft, and 2 hard. ```js //Soft Private let SP = (() => { const data = Symbol("Private:data1"); const data2 = Symbol("Private:data2"); return class SP { constructor() { this[data1] = 42; this[data2] = Math.random() * Math.PI; } print() { console.log(`My private data1= ${this[data1]}`); console.log(`My private data2 = ${this[data2]}`); } }; })(); //Hard Private: memory conservative let HP1 = (() => { const pvt = new WeakMap; return class HP1 { constructor() { pvt.set(this, { data1: 42, data2: Math.random() * Math.PI }); } print() { let p = pvt.get(this) || {} console.log(`My private data1= ${p.data1}`); console.log(`My private data2 = ${p.data2}`); } }; })(); //Hard Private: Babel approach let HP2 = (() => { const data1 = new WeakMap; const data2 = new WeakMap; function getPrivate(key, map) { return map.get(key); } function setPrivate(key, map, value) { map.set(key, value); } return class HP2 { constructor() { setPrivate(this, "data1", 42); setPrivate(this, "data2", Math.random() * Math.PI); } print() { console.log(`My private data1= ${getPrivate(this, "data1")}`); console.log(`My private data2 = ${getPrivate(this, "data2")}`); } }; })(); ``` Soft private is already less complex to setup and maintain than hard private, is faster, and costs less memory as well. Something needs to be done to improve hard private. Hence the need for the language-based upgrade. > I can't think of any other language with classes that doesn't provide some form of reflection (enabled by default) for private members, so regarding classes specifically it seems we're going into uncharted territory. Maybe you haven't been programming long enough. C++ and Java both existed for several years without that ability, and we're no worse off because of that. What we learned back then was that it's better not to rush through planning your code design, something that modern developers seem to have not been taught. This is old territory, and well mapped. > Still, truly inaccessible private object state is something new, and I think it's hard to predict with 100% certainty whether the effect of hard private fields will be more positive or negative for JS users as a whole compared with soft private. Run the code above in Chrome debugger and see if you can figure out how to access (or even see) `data1` & `data2` in HP1 & HP2 instances. Truly inaccessible private object state is as old as factory functions, because that how we used to do it before WeakMap and `class`. Done well, a new syntax for hard private is a plus, as much or more so than a new syntax for soft private. What's missing is an approach that isn't so riddled with complexities and issues. > Getting back to the question of release dates, it's important to consider that the longer it takes for this proposal to be finalized, the longer it will take before class fields can be used in all modern browsers without the use of a transpiler. It's important to consider that class fields will still require use of a transpiler to make use of decorators to fix some important and fairly common use case problems even after being released. If you've got a flat tire with 6 fixable holes, do you rush out to go fix 1 hole? No. you wait until you can afford to either fix all 6 or buy a new tire. > And unless the feature of hard private by default were dropped entirely, I don't see how waiting until decorators are ready would be helpful to those who want to be able to monkey-patch things either. I see it more like this: if private fields were removed from class-fields, then it wouldn't be so bad to release it now. Not good, just not nearly so bad. Sure it has issues, but they are mostly easily mitigated if you're aware of them. So I wouldn't have sufficient reason to want it delayed. The real thing worth having in this proposal is private fields, though. Unfortunately, the formulation is simply no good. While it has no directly observable effects, it has far too many observable side-effects. As such it does not fill the need. That has nothing to do with whether or not it's hard or soft private, though. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #189 (comment)

-- Sent from my Android device with K-9 Mail.

[rdking]

@mbrowne

I was talking specifically about the new # syntax introduced by this proposal and how it should behave natively, not how you would polyfill it.

I get where you're going, and to an extent I agree with you. I tend to think that when there's a choice like this while designing a language feature, then both paths need to be taken. I.e. there needs to be ergonomic syntax for both hard and soft private. Leave the decision on what to use to the developer. Anything less is a disservice. You can easily guess how that way of thinking applies to the balance of this proposal.

...the verbose syntax probably dissuades a lot of people from using WeakMaps today even though they achieve the same goal.

We definitely agree here. I think that to a somewhat lesser extent, the same is true for Symbol-based privacy. Hence my thought that both should receive ergonomic syntax support.

[trusktr]

Hard privacy at runtime is not just about library authors preventing end devs from using certain features.

Suppose we have a website. Do we know what user scripts a user will be sticking into the page? Think about how many people install plugins without caring about plugin permissions or paste codes into the console. I bet the number is higher than we'd like!

Once our code is loaded at runtime (and we make sure our application ships with runtime-hard-privacy, then we'll be more protected against rogue scripts doing things we don't want them to do.

---

@littledan The behavior that the private proposal in this repo has, has an aspect that works well: the privacy. Maybe if you (and champions of this proposal) can explain how other desires can be fulfilled in future add-on proposals, then people would start to like this repo's proposal more once they get passed the syntax.

For example, how, in the future, can we:

1. use private properties with libs (f.e. _.pick)
2. iterate over them
3. assign them dynamically using strings (f.e. this[#'prop'+'erty']
4. use symbols (of the private variety when they're out) so that when exposing private properties to other libs they don't see the symbol ones
5. have protected
6. use them on object literals
7. use them in ES5-style classes
8. store privates on any object, even if the object is not an instance of the current class, but as long as the private variable is in scope
9. etc

?

Regarding de-sugaring to WeakMaps, if that were true of the current proposal, then at least points 1, 2, 3, 4, 6, 7, and 8 should all be possible, because we can currently do all of those using WeakMap. Regarding 5, protected, even that one is possible with WeakMap, with caveats like having to delete prototype.constructor or other tricks that can hurt public users unless we give them tools to use in their place.

If champions this repo can provide both the current proposal, and ideas for future add-ons that include features like above, then community dislike may lift up. It's just a guess though, and I'm assuming that people can get over having to use #. I like how #206 hints at using the private/protected keywords.

It may just be, that because the proposal doesn't really mention those parts, that people don't see the potential future.

The functionality is more important than the syntax, as long as the syntax isn't terrible (# isn't terrible, though it also isn't verbally expressive like keywords of other JavaScript features).

---

#206 is fairly simple idea, and it makes private work with anything (object literals, ES5-style classes, ES6 classes, etc). The idea is interoperable out of the box (_.pick(this, #y, #x)), and starts to paint a picture for actual private, protected keywords.

[ljharb]

@trusktr you can't iterate WeakMaps, and the key of a WeakMap is object identity, so there's no dynamic way to get at a value.

[lifaon74]

@ljharb You may hack the system by overriding WeakMap (and store the values in some case)

[ljharb]

Only if you're first-run code - otherwise, const OrigWeakMap = WeakMap; etc protects you from that.

[jhpratt]

I know this was rejected, though I don't necessarily agree with the reasoning stated in the FAQ, but I feel it's necessary to point out that the first 5 points @trusktr has presented would be resolved trivially with the proposal of private.x and private(foo).x.

[trusktr]

you can't iterate WeakMaps

I meant I can iterate the properties of the private properties like so:

const _ = privateHelper()

class Foo {
  constructor() {
    this.publicProp = 'foo'
    _(this).privateProp1 = 'bar'
    _(this).privateProp2 = 'baz'
  }

  test() {
    for (const key Object.keys(_(this)))
      console.log(key)
  }
}

const f = new Foo
f.test()
// "privateProp1"
// "privateProp2"

That would be similar to the following if there were some sort of syntax for it:

class Foo {
  publicProp = 'foo'
  #privateProp1 = 'bar'
  #privateProp2 = 'baz'

  test() {
    for (const key Object.keys(this#))
      console.log(key)
  }
}

const f = new Foo
f.test()
// "privateProp1"
// "privateProp2"

or maybe

    for (const key Object.keys(this, #)) // give a function private access somehow

Just throwing ideas out there. It'd be nice for these type of things to be easy.

[mbrowne]

I'd be interested to know more about the findings of TC39's community outreach. I'm sure that something between private and public is something a lot of people are interested in. What else made the top of the list that isn't covered by this proposal? And in addition to decorators, are there any other proposals already in the works to expand on this proposal?