Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Statically tracked shared immutable objects #125

Open
leafpetersen opened this issue Dec 5, 2018 · 47 comments
Open

Statically tracked shared immutable objects #125

leafpetersen opened this issue Dec 5, 2018 · 47 comments
Assignees
Labels
feature Proposed language feature that solves one or more problems

Comments

@leafpetersen
Copy link
Member

leafpetersen commented Dec 5, 2018

This is a proposed solution to the problem of communicating data between isolates.

This proposal adds a static notion of immutability which guarantees that once initialized, any immutable object is the root of a fully immutable object graph, which can therefore be shared between isolates with no copying required.

This proposal also incidentally provides at least a partial solution for the following problems:

Initial draft proposal is here

@leafpetersen leafpetersen added the feature Proposed language feature that solves one or more problems label Dec 5, 2018
@leafpetersen leafpetersen self-assigned this Dec 5, 2018
@leafpetersen
Copy link
Member Author

Draft PR up for comment.

@pulyaevskiy
Copy link

Would this be somewhat similar to Scala case classes or similar immutable structures?

Also how equality and hashCode would work if I have two instances of immutable class that hold identical values? E.g.

Say:

class User immutable {
  final String id;
  final String name;
  User({this.id, this.name});
}

final a = User(id: '123', name: 'John');
final b = User(id: '123', name: 'John');
print (a == b); // ?

If, it's going to be similar it would be nice to have a convenient way to create a copy of such immutable object, similar to Scala's copy method on case classes.

@leafpetersen
Copy link
Member Author

I think it might be a good idea to have equality and hashCode be automatically derived, and have structural semantics.

What is the motivation for having a copy method? I would think that since they are deeply immutable, you should never copy (just share)?

@munificent
Copy link
Member

What is the motivation for having a copy method?

In Scala, the generated copy() method has optional named parameters for each field in the case class, so you can produce a copy while changing one or more of the fields. Sort of like OCaml's functional update syntax.

@pulyaevskiy
Copy link

Yes, copy is sort of a magic method with named parameters for every field.

You can actually see this pattern in Flutter copyWith method.

@munificent
Copy link
Member

When/if this gets further along I'll have a lot more detailed feedback around stuff like syntax and policy. Overall, I think this is a really interesting proposal. I'm not sold on all aspects, but I think it's a good starting point. Here's a couple of high level things:

  • Immutability is a hot topic in general right now. A lot of programmers are realizing that their programs are easier to reason about and maintain when they know which parts of their data is immutable and which isn't. So I expect that if we ship anything around "immutability", users will want to be able to use it for lots and lots of their classes.

    That's a good thing, because it means this feature can provide a lot of value for a lot of users. But it also means we should make sure it's general enough to be a good fit for as many kinds of immutable data as possible. I don't see anything the proposal that gets in the way of that now except that, like Yegor, I think the syntax should be nicer. I think data class Foo { ... } looks pretty good.

  • Related to the above, I think many users will expect these to have value-like semantics, (for some definition of "value"), so implementing hashCode and operator ==()` is probably worth considering.

  • I do think users will need some kind of runtime checking for whether or not an object is immutable, for code like:

    takeList(List stuff) {
      // Defensive copy, only if needed.
      if (stuff is! Immutable) stuff = stuff.toList();
      _field = stuff;
    }

    A marker interface works fine, I think.

  • The flow analysis feels pretty complex for the amount of value it provides.
    For user-defined classes, shallow immutable objects (i.e. all fields final) require every bit of state to be initialized before the constructor body exits, and that doesn't seem to be too painful of a restriction for users.

    Lists and maps do tend to be imperatively built more than classes, so I can see why they might be special. But, even so, I think most intended-to-be-immutable lists and maps probably aren't mutated after their initial literal expression ends. With the stuff we're talking about adding to collection for UI as code (spread, if, for), that goes even farther. The latter in particular means you can do a lot of interesting imperative work right inside the literal before the object is ever observed.

    I'm also worried about brittleness. Hoisting a chunk of initialization code out into a helper function will either break the flow analysis, or require us to add interprocedural analysis. The former feels annoying, and the latter feels like a lot of mechanism to add for what we get in return.

    So, if it's possible, I think it's worth seeing if we can get by without the flow analysis stuff at all.

  • Taking that out would mean there's no way to statically prevent you from calling things like add() on an ImmutableList since it does statically implement List. I wonder if we should consider either something like C#'s explicit interface implementation or have ImmutableList not implement List but support an "implicit conversion" to List that's a no-op at runtime but hides the mutating List methods statically.

I really hope we can make progress on this.

@leafpetersen
Copy link
Member Author

leafpetersen commented Dec 5, 2018

  • With the stuff we're talking about adding to collection for UI as code (spread, if, for), that goes even farther.

I think the UI as code collection features would help a lot. The things they don't cover are:

  • The cases where you need random access initialization (rather than just sequential initialization)
  • The cases where you want to refer to other elements of the array during the initialization process

Maybe these are rare enough that it's ok to do this in two steps: first fill in a mutable list, then copy it to an immutable list?

It would be good to look at some real code to see if it fits these patterns. @aam do you have any concrete examples you have been working from?

@munificent
Copy link
Member

I could be wrong, but my hunch is that both of those cases are rare enough that building a separate mutable list and copying is sufficient. I'd definitely like to see motivating examples too. I don't have a good feel for what kind of code is driving the feature.

@aam
Copy link

aam commented Dec 5, 2018

It would be good to look at some real code to see if it fits these patterns. @aam do you have any concrete examples you have been working from?

I was looking at transferrables, rather than immutables, so the use-case I've been looking at is image loading in https://github.com/flutter/flutter/blob/master/packages/flutter/lib/src/painting/image_provider.dart where if you move image downloading off to a separate isolate(good thing), then you have to copy all downloaded bytes back to main isolate(bad thing). "Simple" solution for this "simple" problem(transfer of unstructured typed data) that I'm using to jump start more vigorous analysis of this area is to introduce another constructor for Uint8List that would create "transferrable" list, a list that can be 0-cost moved (transferred) from one isolate to another.

@yjbanov
Copy link

yjbanov commented Dec 6, 2018

I'd like to link some of the properties of immutable objects that have been mentioned in this thread to specific Flutter issues that they could potentially solve (@Hixie for sanity-checking):

  • Immutability: solves this issue Sealing or watching List objects, or copy-on-write List objects sdk#27755
  • Fast cross-isolate sharing: solves the issue of transferring big chunks of structured data between isolates. One example is parse a big piece of JSON or protobufs in an isolate, then share the data with the main isolate for rendering.
  • operator==/hashCode: this could make UI rebuilds more efficient because the framework relies on operator== to decide whether a widget needs to be rebuilt (source). In practice few users implement hashCode and == so that equality check can only pass when two widgets are identical, which is rarely the case when you call setState. I only works when you cache widget object or use const.
  • Regular classes extending immutable classes: this was discussed in the proposal PR. This property would allow us to make the Widget class immutable as a non-breaking change. Also, not all widgets need to be deeply immutable. Some widgets could carry references to stateful service objects. Instances of such widgets would not be shareable with other isolates. However, having operator== and == would still be useful.
  • Implicit conversion between List and ImmutableList: discussed in the proposal PR. this would allow us to convert final List children to ImmutableList children as a non-breaking change.
  • Canonicalization: discussed in the proposal PR. This would make tree comparison cheap (practically free). For example, if a widget near the top of the widget tree calls setState but only affects one branch in the UI, the framework can push the state change down the tree automatically without developer's involvement. However, this would potentially make allocation of such objects more expensive. This needs to be measured.
  • Non-nullability: this property would produce fewer objects on the heap, particularly non-nullable ints, doubles, bools, Strings.

I have more thoughts on this here. The tl;dr version:

  • Would love to see a lightweight syntax for this, e.g. data Foo to declare an immutable class, <data T> to put a constraint on a generic parameter, and DataList instead of ImmutableList. (value instead of data would also be fine).
  • Consider providing better defaults, such as implicit final and non-nullability by default.
  • Consider providing operator== and hashCode; they would be so so useful in a lot of code.

@leafpetersen
Copy link
Member Author

@yjbanov Thanks for the summary. I'm don't want to promise that we can cover all of those points, but that's a great list to aim for.

Question in light of the example from @aam above. Do you see sharing data between isolates being valuable (as opposed to just being able to transfer data from one isolate to the other). For example, I can imagine an architecture in which one isolate does a functional update on the render tree (that is, treats it like a persistent data structure, allocating new objects for the changed part), and then passes off the pointer to the "updated" tree to another isolate for rendering, and then repeats.

Would you expect to take advantage of something like that? Or is one-time transfer of data (assuming near zero-cost transfer) the sweet spot?

@davidmorgan
Copy link
Contributor

Very excited about where this leads :)

Re: immutable collections. Please note that https://github.com/google/built_collection.dart is already heavily used--e.g. it is referenced in google3 4.4x more than package:collection. (I won't share absolute numbers here, they are easy enough to get).

So I think this proposal needs to contain a plan for built_collection; they should either be the same thing, or they should be compatible, or there should be a very easy migration path. Any disruption to this space risks causing a lot of churn.

I think compatible means at least that there's a way for a built_collection instance to count as Immutable. The provision of hashCode is going to cause issues here, see below.

Thoughts based on what we've seen with built_collection, and some comments on compatibility:

  • ImmutableList implements List is a scary thing as it causes failures at runtime. built_collection chose to skip implementing List and implement just Iterable, and this has worked well. One related thought, if we ever get a way to declare generic covariance/contravariance, at that point having a separate immutable interface will allow us to treat the generics correctly--we can't do that if we reuse List.
  • Immutable collections should forbid null values. It's simpler, it's more efficient, and it's extremely rare that people want them.
  • Comparison+hashing works well. It makes sense to cache the hash code which I think will be the biggest issue for compatibility with this proposal. If ImmutableList manages to solve this efficiently and still remain immutable then we can presumably just delegate to it. Otherwise we're going to hit problems.
  • Switching BuiltList to wrap ImmutableList without making it slower--causing twice as much copying--is not possible with the current proposal. Currently, BuiltList copies when it turns into ListBuilder but does not copy when it turns back into BuiltList; if any further modifications are made to the builder then at that point an additional copy is made. This is what the proposal is trying to make efficient with the part about the instance escaping--but what is currently in the proposal is not sufficient. We would need something more explicit.
  • It might be worth calling out in the proposal if the intention is that they should replace mutable collections whenever collections are passed around, or whether they are for specialist use. To be used everywhere they should be as easy to "update" as possible. If they need to be as convenient as the mutable collections, then the builder pattern works well; if this doesn't make it into the core then hopefully it's possible for a library like built_collection to provide it in a reasonably friction-free way.

Re: immutable value types. Similar story here: https://github.com/google/built_value.dart is already in this space and heavily used. Inside google3 it accounts for just over half of the classes that define hashCode (ignoring the SDK).

Again, we're going to need a plan for what happens to built_value. I think the right outcome is that it's possible to create built_value instances that count as immutable.

The biggest part of this will be making it so built_collection instances can count as immutable.

Beyond that, the only awkward point I see is that built_value has a feature which allows you to mark getters as @memorized, causing them to be cached on first use. We wouldn't be able to support that and be immutable at the same time. This sounds like a feature request for the proposal, or maybe a followup--it makes your immutable types significantly more powerful. It could also be considered a more general form of the request to have collections cache their hash codes.

Finally some general thoughts on immutable classes:

  • +1 for operator== and hashCode. FWIW, built_value does not usually cache these; but it has been requested as a feature, it would be nice to be able to support it.
  • +1 for more focus on non-nullability, although people still want a way to make null be allowed, including primitives; search our code for @nullable, which is how built_value marks this, to see this in action.
  • on updates, again, I like the builder pattern; see use of the rebuild method in our code for examples; and in particular built_value uses nested builders to give a very concise way to write updates to something deeply nested in your object structure; it also makes it efficient (only rebuilds along the path to the root); as far as I can see, though, all this can live happily outside the core language support. I might even argue for not adding something like a copy method ... it's not as good as what built_value provides, but it does cover some of the use cases, so it will reduce built_value use :)

Those are my initial thoughts--thanks for listening :)

Please reach out to me if you need anything specific on how this proposal interacts with existing code, I'll be happy to explore options, write docs, point to examples. I'll anyway keep an eye on the issue and try to volunteer anything that seems relevant.

@davidmorgan
Copy link
Contributor

One more bit of context, in case it's relevant: built_value is gaining popularity as a way to do serialization in flutter apps. Being able to pass them between isolates will fit perfectly with this.

@jmesserly
Copy link

+1 for more focus on non-nullability, although people still want a way to make null be allowed, including primitives; search our code for @nullable, which is how built_value marks this, to see this in action.

I think non-nullability is orthogonal to immutable objects, but the good news is, there's lots of work happening on it: #110. There's some great ideas about how to make the migration tractable (IMO). I'm very excited for that.

@pulyaevskiy
Copy link

I might even argue for not adding something like a copy method ... it's not as good as what built_value provides, but it does cover some of the use cases

If it’s implemented similarly to the Scala version then it would be a great reducer of boilerplate. Actually a default constructor would be great as well.

This is just my personal experience but there are two things that lead me to stop using built_value even though I like the package:

  • required extra build step
  • verbosity of builder pattern, it doesn’t feel like what Dart promises in terms of being concise

Even though Dart build system is now very advanced it has it’s downsides. For something as common as data classes (we saw examples from Angular, Flutter already) it seems strange to require users setup build pipelines and depend on extra packages. It should be in the language.

@leafpetersen
Copy link
Member Author

I landed an initial draft proposal here. I'll try to circle back to this soon and think about the issues raised so far in comments. More discussion welcome!

@kevmoo
Copy link
Member

kevmoo commented Dec 6, 2018

Re ImmutableList (Set/Map) as a subtype or supertype of List – or neither.

We have a 🐔 / 🥚 problem here. We have a LOT of functions in our ecosystem that take List (or Set/Map) which really only need ReadOnlyList (etc).

All of our existing read-only/immutable collection types implement the read/write interface and throw on mutate. (think the views, const, List.unmodifiable, etc)

So any new collection type we add MUST implement the corresponding existing interface – so be a subtype.

We should also, orthogonally/additionally, add read-only List/Set/Map abstract classes which are supertypes of the existing classes. Then we can start migrating the appropriate APIs to the read-only versions.

Then in Dart 3 (maybe) we can go from class ImmutableList implements List to class ImmutableList implements ReadonlyList

  • Edit 1: you can imagine a corresponding lint "it seems like you don't mutate this collection, click here to change your function signature"

Copied from #126 (comment)

CC @yjbanov @matanlurey @jonahwilliams

@kevmoo
Copy link
Member

kevmoo commented Dec 6, 2018

Or...

abstract class ReadOnlyList implements Iterable { }

class List implements ReadOnlyList {}

class ImmutableList implements ReadOnlyList, Immutable {
  List get listView;
}

We get more "purity" short-term, but we make folks use myImmutableList.listView when passing to existing APIs.

THEN we add a lint that tells folks to stop using il.listView when passing into functions that take ReadOnlyList – so we can still (slowly) evolve the ecosystem to a more readonly/immutable future

@Hixie
Copy link

Hixie commented Dec 6, 2018

One thing to be wary of with operator== is that you wouldn't want to apply it to widgets, because you'd end up with O(N²) behavior on updates, which is actually worse than using identical() and just walking the tree (that's only O(N)).

@jonahwilliams
Copy link

Then in Dart 3 (maybe) we can go from class ImmutableList implements List to class ImmutableList implements ReadonlyList

I really doubt we'd be able to make that change

@Hixie
Copy link

Hixie commented Dec 6, 2018

I don't really see how this proposal helps with the Flutter list mutation issues. For that issue we want list literals, and lists generated by the toList() method on various Iterable implementations, to support the semantic that they are being transferred to a new owner and won't be mutated by the original owner anymore.

@kevmoo
Copy link
Member

kevmoo commented Dec 6, 2018

I don't really see how this proposal helps with the Flutter list mutation issues.

Well, an idea.

If we had literal support for (shallow) immutability – ^[a,b,c] as @leafpetersen had in the doc – you could imagine some new annotation in pkg:meta along the lines of @preferImmutable – and the analyzer could offer help for things that are not provably immutable.

Of course, we're mixing things here.

There's a need for deep immutability, which is the core focus of this proposal, to enable sharing in isolates.

You just want shallow immutability, right? I'm guessing it'd be far too cumbersome to require all implementations of Widget to be deeply immutable, but I could be wrong.

It's a bit gnarly to ponder having both concepts for a List – especially in a way you could check statically...

@leafpetersen
Copy link
Member Author

@yjbanov A couple of questions:

  • Regular classes extending immutable classes:

This really changes the feature, since at that point you can't tell from the type whether something is immutable or not (and hence when you are constructing a deeply immutable object, you can't statically tell whether the things you are using to initialize its fields are themselves deeply immutable).

  • This property would allow us to make the Widget class immutable as a non-breaking change. Also, not all widgets need to be deeply immutable.

Can you elaborate on the use case for shallow immutable objects? What don't you get from current classes with final fields? You want the auto-generated equality and hashCode? Is it obvious what those should do if fields are mutable? Presumably at the least you wouldn't want the system to cache the hashCode?

@leafpetersen
Copy link
Member Author

@Hixie

I don't really see how this proposal helps with the Flutter list mutation issues. For that issue we want list literals, and lists generated by the toList() method on various Iterable implementations, to support the semantic that they are being transferred to a new owner and won't be mutated by the original owner anymore.

This isn't primarily aimed at that use case, so if it doesn't fit it, it doesn't fit it. But it could at least partially address it. Under the proposal, lists can be created in a mutable state, and then they become immutable at the point where they are passed to flutter. If the flutter APIs changed to take ImmutableList, then literals in build methods would implicitly become immutable. We could also provide a literal syntax for immutable lists. Calls to .toList() would have to be replaced with something else that produced an immutable list instead.

But the ability to mutate and freeze is limited, by design. So it is possible that this doesn't cover that use case.

@leafpetersen
Copy link
Member Author

@davidmorgan

  • Immutable collections should forbid null values.

Even if we have NNBD separately? That is, you think it should be impossible to create an immutable collection of int?, even explicitly?

  • . It makes sense to cache the hash code which I think will be the biggest issue for compatibility with this proposal.

This could potentially be dealt with by the compiler. It depends a bit on the GC constraints, but as long as hashCode is deterministic, there are no semantic problems with having multiple threads race to write in the hidden hashCode field.

Beyond that, the only awkward point I see is that built_value has a feature which allows you to mark getters as @Memorized, causing them to be cached on first use.

This is a bit trickier. We could possibly do something around this, but I think you'd have to use locking in the getter, which probably cracks the door open a bit to things like deadlocks and non-determinism.

@davidmorgan
Copy link
Contributor

Re ImmutableList (Set/Map) as a subtype or supertype of List – or neither.

We have a / problem here. We have a LOT of functions in our ecosystem that take List (or Set/Map) which really only need ReadOnlyList (etc).

Actually most of them only need 'Iterable', which is already available. This is the route we took with built_collection, and it worked well. We also provide asList (creates a view) and toList (creates a copy on write list), both of which are fast, for when you really do need a List.

@yjbanov
Copy link

yjbanov commented Dec 7, 2018

@Hixie

I don't really see how this proposal helps with the Flutter list mutation issues.

This isn't primarily aimed at that use case, so if it doesn't fit it, it doesn't fit it.

I would be worried about taking this approach. Dart has been a simple language so far and one of the reasons for its simplicity is that any given feature has a lot of applications. For example, look how far a regular class was able to take us. We are just starting to hit its limitations. I think the immutability proposal should try to capture as many use-cases as possible. In the Flutter world this includes both the normal single-isolate usage of widgets as well as data processing on other isolates.

@yjbanov
Copy link

yjbanov commented Dec 7, 2018

@davidmorgan, @leafpetersen

  • Immutable collections should forbid null values.

Even if we have NNBD separately? That is, you think it should be impossible to create an immutable collection of int?, even explicitly?

I would be against forbidding nulls, but NNBD would be great. I understand that built_value uses Optional and that leaves less room for null, but that's not the case for the vast majority of the Dart ecosystem. For most use-cases, int? is as good as Optional, even though it loses composability.

  • . It makes sense to cache the hash code which I think will be the biggest issue for compatibility with this proposal.

This could potentially be dealt with by the compiler. It depends a bit on the GC constraints, but as long as hashCode is deterministic, there are no semantic problems with having multiple threads race to write in the hidden hashCode field.

I think hashCode caching should be transparent. As I mentioned in #125 (comment), depending on whether we have canonicalization caching of the hash code may or may not be redundant.

Beyond that, the only awkward point I see is that built_value has a feature which allows you to mark getters as @Memorized, causing them to be cached on first use.

This is a bit trickier. We could possibly do something around this, but I think you'd have to use locking in the getter, which probably cracks the door open a bit to things like deadlocks and non-determinism.

I think lazy initialization could work such that we initialize upon first access or upon sharing. Although, if we go with the approach of always allocating on the shared heap then lazy initialization is moot.

@jmesserly
Copy link

@yjbanov regarding:

In NNBD you can totally make things nullable, hence the "BD" in the acronym :)
IOW, I would like to stop the trend of us punishing people for wanting the correct default behavior:

We should probably move non-null discussion over to #110? But yeah, that proposal is for "Sound non-nullable types" by default. So bar and baz in your example would be non-null, unless you change them to String? bar and double? baz.

@yjbanov
Copy link

yjbanov commented Dec 8, 2018

@jmesserly Sure, here's my attempt: #110 (comment)

@yjbanov
Copy link

yjbanov commented Dec 8, 2018

@munificent

So, if it's possible, I think it's worth seeing if we can get by without the flow analysis stuff at all.

I think so too. If we get implicit conversion from List to ImmutableList and conditional elements we may not need type system-level mutability tracking for some time. For simple cases where a List never escapes the function scope before it's assigned to ImmutableList a clever compiler could remove list copying even when you do construct the list imperatively. For example:

final list = <int>[1, 42, 3];
list[1] = 2;
list.add(4);
ImmutableList imm = list;  // last usage of `list`, no need to copy, reuse storage
// the rest of code deals only with `imm`

The code where list is assigned to an ImmutableList imm could also be a function call assigning to a parameter of type ImmutableList, a similar constructor call, assignment to field or static variable, etc.

I think it's worth considering mutability tracking more carefully in isolation after shipping immutable types, and consider options beyond automatic flow analysis. For example, if you take Rust's borrow checker and strip all the memory management and lifecycle features (which don't apply to Dart), and only leave mutability tracking you actually get a pretty simple, powerful and easy to learn system. I think it's worth considering it alongside automatic flow analysis.

@davidmorgan
Copy link
Contributor

@davidmorgan, @leafpetersen

  • Immutable collections should forbid null values.

Even if we have NNBD separately? That is, you think it should be impossible to create an immutable collection of int?, even explicitly?

I would be against forbidding nulls, but NNBD would be great. I understand that built_value uses Optional and that leaves less room for null, but that's not the case for the vast majority of the Dart ecosystem. For most use-cases, int? is as good as Optional, even though it loses composability.

Actually built_value uses @nullable; it's intended to map nicely onto NNBD when that arrives.

The suggestion for completely forbidding nulls in immutable collection is separate, and made up of two things: 1) there is strong evidence that people don't want nulls in their immutable collections, from built_collection and Guava, both of which forbid them; and 2) it might be this gives you a better API, e.g. you can now use null to mean absent. If #2 doesn't apply, e.g. if the mutable API is reused exactly as is, then there is less to argue for it, and maybe it should wait for NNBD.

@sigurdm
Copy link
Contributor

sigurdm commented Dec 18, 2018

Just for completeness of discussion i want to mention the work on immutability in Dartino (mainly by @mkustermann)
I'll paraphrase the design as well as I remember it:

  • Instead of tracking immutability in the types it was a dynamic property of an object.
  • If an object was created using a const constructor and all fields was initialized to immutable objects the object would itself be immutable. (one could consider using a different keyword to initiate this:
    new Foo() => immutable Foo())
  • There was a top-level predicate to query if an object was immutable.
  • Immutable objects were allocated on a separate heap and could be sent between threads without copying.
  • I don't remember fully, but I believe List.unmodifiable() would also be allocated as an immutable object.

I feel this design was less invasive on the language while giving cheap communication between threads.

@kevmoo
Copy link
Member

kevmoo commented Dec 18, 2018 via email

@eernstg
Copy link
Member

eernstg commented Jan 24, 2019

For the purpose of communicating with another isolate, deeply immutable entities are required. So it makes sense that this proposal requires immutability to include anything reachable from an immutable object.

But for many other purposes it could also be relevant to support a shallow notion of immutability. For example, it would be possible to offer developers support for safe sharing of standard data structures like lists and maps, in the same isolate, still allowing those data structures to have references to (that is "to contain") arbitrary objects, if the language were to support shallow immutability for those lists and maps.

This means that it would still be possible for multiple agents to access, say, the elements of a shared list, and that could cause all the well-known problems associated with unwanted shared access; but this is no worse than any kind of aliasing, and this is something that Dart developers are dealing with every single day. When using such a shallowly immutable object o, that object itself can be trusted to stay the same (say "nobody else can change this list behind my back"), and for everything reachable from o it would simply be necessary to use the same plethora of techniques that we are already using in order to manage shared access to objects in general.

This should not be hard to do (except, of course, for the fact that the notation used to specify immutability might be a bit more verbose now that we would have two variants, so we'll need additional syntax): We just need to apply a subset of the restrictions specified in the proposal.

With such a generalization, this proposal could be considered as a response to #117 as well.

@icatalud
Copy link

icatalud commented Jan 3, 2020

As a newcomer to the thread it would be beneficial that the proposal comment has a summary section, answers the main viability questions or alternative solutions made in the comments. This makes it easier to catch up with ideas. A shared gdoc could be useful for that, because everyone can comment and suggest directly in the proposal.
There are too many comments and I only took a look to the proposal, please excuse me if I'm overlooking important considerations.

As an alternative to doing T is immutable in a class definition, what if the vars themselves are defined as immutable (I'll name it sealed)?

sealed Foo x = Foo()
var y = x             // invalid, can only be assigned to other sealed vars

sealed z = x.bar  
var zz = x.bar      // invalid, x only returns sealed vars

The reason I name it sealed is that In #758 sealed functions are proposed. In a sealed var, only sealed methods can be invoked.
The idea there allows mutating objects, but a more constrained version of that definition could forbid using setters, so only sealed functions and sealed methods (final var getters are sealed) can be used inside and they always return sealed vars. That definition is quite limiting, it basically only allows viewing things, mutations always require creating a new object. But this fits exactly what is required here. It's also compatible with the Immutable functions proposal, but the name sealed is more appropriate because it's synonym of encapsulation. Immutable is more fitting for the immutable class approach, but it's ambiguous what immutable means for a function (can they only read values? can they call only immutable functions?).

This would not require any special treatment or heap allocation, sealed references can be safely passed between isolates.

Sealed var declarations are easier to approach, because they are defined from any object, they do not require a custom implementation.

It is very constraining to define an immutable (sealed) var, so it should be analyzer suggested against it, unless the vars are shared between isolates.

@kevmoo
Copy link
Member

kevmoo commented May 24, 2022

FYI: https://github.com/tc39/proposal-record-tuple

TC39 looking at syntax for deeply immutable Record (Map) and Tuple (List).

@TekExplorer
Copy link

Isnt this resolved with Records?

@munificent
Copy link
Member

No, records aren't required to be deeply immutable. This is fine:

var list = [1, 2];
var record = (numbers: list);
list.add(3);
print(record.numbers); // Prints "[1, 2, 3]".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Proposed language feature that solves one or more problems
Projects
None yet
Development

No branches or pull requests