-
Notifications
You must be signed in to change notification settings - Fork 205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Statically tracked shared immutable objects #125
Comments
Draft PR up for comment. |
Would this be somewhat similar to Scala case classes or similar immutable structures? Also how equality and hashCode would work if I have two instances of immutable class that hold identical values? E.g. Say: class User immutable {
final String id;
final String name;
User({this.id, this.name});
}
final a = User(id: '123', name: 'John');
final b = User(id: '123', name: 'John');
print (a == b); // ? If, it's going to be similar it would be nice to have a convenient way to create a copy of such immutable object, similar to Scala's |
I think it might be a good idea to have equality and hashCode be automatically derived, and have structural semantics. What is the motivation for having a |
In Scala, the generated |
Yes, copy is sort of a magic method with named parameters for every field. You can actually see this pattern in Flutter copyWith method. |
When/if this gets further along I'll have a lot more detailed feedback around stuff like syntax and policy. Overall, I think this is a really interesting proposal. I'm not sold on all aspects, but I think it's a good starting point. Here's a couple of high level things:
I really hope we can make progress on this. |
I think the UI as code collection features would help a lot. The things they don't cover are:
Maybe these are rare enough that it's ok to do this in two steps: first fill in a mutable list, then copy it to an immutable list? It would be good to look at some real code to see if it fits these patterns. @aam do you have any concrete examples you have been working from? |
I could be wrong, but my hunch is that both of those cases are rare enough that building a separate mutable list and copying is sufficient. I'd definitely like to see motivating examples too. I don't have a good feel for what kind of code is driving the feature. |
I was looking at transferrables, rather than immutables, so the use-case I've been looking at is image loading in https://github.com/flutter/flutter/blob/master/packages/flutter/lib/src/painting/image_provider.dart where if you move image downloading off to a separate isolate(good thing), then you have to copy all downloaded bytes back to main isolate(bad thing). "Simple" solution for this "simple" problem(transfer of unstructured typed data) that I'm using to jump start more vigorous analysis of this area is to introduce another constructor for Uint8List that would create "transferrable" list, a list that can be 0-cost moved (transferred) from one isolate to another. |
I'd like to link some of the properties of immutable objects that have been mentioned in this thread to specific Flutter issues that they could potentially solve (@Hixie for sanity-checking):
I have more thoughts on this here. The tl;dr version:
|
@yjbanov Thanks for the summary. I'm don't want to promise that we can cover all of those points, but that's a great list to aim for. Question in light of the example from @aam above. Do you see sharing data between isolates being valuable (as opposed to just being able to transfer data from one isolate to the other). For example, I can imagine an architecture in which one isolate does a functional update on the render tree (that is, treats it like a persistent data structure, allocating new objects for the changed part), and then passes off the pointer to the "updated" tree to another isolate for rendering, and then repeats. Would you expect to take advantage of something like that? Or is one-time transfer of data (assuming near zero-cost transfer) the sweet spot? |
Very excited about where this leads :) Re: immutable collections. Please note that https://github.com/google/built_collection.dart is already heavily used--e.g. it is referenced in google3 4.4x more than So I think this proposal needs to contain a plan for I think compatible means at least that there's a way for a Thoughts based on what we've seen with
Re: immutable value types. Similar story here: https://github.com/google/built_value.dart is already in this space and heavily used. Inside google3 it accounts for just over half of the classes that define Again, we're going to need a plan for what happens to The biggest part of this will be making it so Beyond that, the only awkward point I see is that Finally some general thoughts on immutable classes:
Those are my initial thoughts--thanks for listening :) Please reach out to me if you need anything specific on how this proposal interacts with existing code, I'll be happy to explore options, write docs, point to examples. I'll anyway keep an eye on the issue and try to volunteer anything that seems relevant. |
One more bit of context, in case it's relevant: |
I think non-nullability is orthogonal to immutable objects, but the good news is, there's lots of work happening on it: #110. There's some great ideas about how to make the migration tractable (IMO). I'm very excited for that. |
If it’s implemented similarly to the Scala version then it would be a great reducer of boilerplate. Actually a default constructor would be great as well. This is just my personal experience but there are two things that lead me to stop using built_value even though I like the package:
Even though Dart build system is now very advanced it has it’s downsides. For something as common as data classes (we saw examples from Angular, Flutter already) it seems strange to require users setup build pipelines and depend on extra packages. It should be in the language. |
I landed an initial draft proposal here. I'll try to circle back to this soon and think about the issues raised so far in comments. More discussion welcome! |
Re We have a 🐔 / 🥚 problem here. We have a LOT of functions in our ecosystem that take All of our existing read-only/immutable collection types implement the read/write interface and throw on mutate. (think the views, const, So any new collection type we add MUST implement the corresponding existing interface – so be a subtype. We should also, orthogonally/additionally, add read-only List/Set/Map abstract classes which are supertypes of the existing classes. Then we can start migrating the appropriate APIs to the read-only versions. Then in Dart 3 (maybe) we can go from
Copied from #126 (comment) |
Or... abstract class ReadOnlyList implements Iterable { }
class List implements ReadOnlyList {}
class ImmutableList implements ReadOnlyList, Immutable {
List get listView;
} We get more "purity" short-term, but we make folks use THEN we add a lint that tells folks to stop using |
One thing to be wary of with operator== is that you wouldn't want to apply it to widgets, because you'd end up with O(N²) behavior on updates, which is actually worse than using identical() and just walking the tree (that's only O(N)). |
I really doubt we'd be able to make that change |
I don't really see how this proposal helps with the Flutter list mutation issues. For that issue we want list literals, and lists generated by the toList() method on various Iterable implementations, to support the semantic that they are being transferred to a new owner and won't be mutated by the original owner anymore. |
Well, an idea. If we had literal support for (shallow) immutability – Of course, we're mixing things here. There's a need for deep immutability, which is the core focus of this proposal, to enable sharing in isolates. You just want shallow immutability, right? I'm guessing it'd be far too cumbersome to require all implementations of It's a bit gnarly to ponder having both concepts for a List – especially in a way you could check statically... |
@yjbanov A couple of questions:
This really changes the feature, since at that point you can't tell from the type whether something is immutable or not (and hence when you are constructing a deeply immutable object, you can't statically tell whether the things you are using to initialize its fields are themselves deeply immutable).
Can you elaborate on the use case for shallow immutable objects? What don't you get from current classes with final fields? You want the auto-generated equality and hashCode? Is it obvious what those should do if fields are mutable? Presumably at the least you wouldn't want the system to cache the hashCode? |
This isn't primarily aimed at that use case, so if it doesn't fit it, it doesn't fit it. But it could at least partially address it. Under the proposal, lists can be created in a mutable state, and then they become immutable at the point where they are passed to flutter. If the flutter APIs changed to take But the ability to mutate and freeze is limited, by design. So it is possible that this doesn't cover that use case. |
Even if we have NNBD separately? That is, you think it should be impossible to create an immutable collection of
This could potentially be dealt with by the compiler. It depends a bit on the GC constraints, but as long as hashCode is deterministic, there are no semantic problems with having multiple threads race to write in the hidden hashCode field.
This is a bit trickier. We could possibly do something around this, but I think you'd have to use locking in the getter, which probably cracks the door open a bit to things like deadlocks and non-determinism. |
Actually most of them only need 'Iterable', which is already available. This is the route we took with |
I would be worried about taking this approach. Dart has been a simple language so far and one of the reasons for its simplicity is that any given feature has a lot of applications. For example, look how far a regular |
I would be against forbidding nulls, but NNBD would be great. I understand that
I think
I think lazy initialization could work such that we initialize upon first access or upon sharing. Although, if we go with the approach of always allocating on the shared heap then lazy initialization is moot. |
@yjbanov regarding:
We should probably move non-null discussion over to #110? But yeah, that proposal is for "Sound non-nullable types" by default. So |
@jmesserly Sure, here's my attempt: #110 (comment) |
I think so too. If we get implicit conversion from final list = <int>[1, 42, 3];
list[1] = 2;
list.add(4);
ImmutableList imm = list; // last usage of `list`, no need to copy, reuse storage
// the rest of code deals only with `imm` The code where I think it's worth considering mutability tracking more carefully in isolation after shipping immutable types, and consider options beyond automatic flow analysis. For example, if you take Rust's borrow checker and strip all the memory management and lifecycle features (which don't apply to Dart), and only leave mutability tracking you actually get a pretty simple, powerful and easy to learn system. I think it's worth considering it alongside automatic flow analysis. |
Actually The suggestion for completely forbidding nulls in immutable collection is separate, and made up of two things: 1) there is strong evidence that people don't want nulls in their immutable collections, from |
Just for completeness of discussion i want to mention the work on immutability in Dartino (mainly by @mkustermann)
I feel this design was less invasive on the language while giving cheap communication between threads. |
I LOVE the idea of concepts introduced at the library level instead of the
language – much more flexible.
(Although I still want my syntactic sugar for immutable collections)
…On Tue, Dec 18, 2018 at 1:15 AM Sigurd Meldgaard ***@***.***> wrote:
Just for completeness of discussion i want to mention the work on
immutability in Dartino (mainly by @mkustermann
<https://github.com/mkustermann>)
I'll paraphrase the design as well as I remember it:
- Instead of tracking immutability in the types it was a dynamic
property of an object.
- If an object was created using a const constructor and all fields
was initialized to immutable objects the object would itself be immutable.
(one could consider using a different keyword to initiate this:
new Foo() => immutable Foo())
- There was a top-level predicate to query if an object was immutable.
- Immutable objects were allocated on a separate heap and could be
sent between thread without copying.
- I don't remember fully, but I believe List.unmodifiable() would also
be allocated as an immutable object.
I feel this design was less invasive on the language while giving cheap
communication between threads.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#125 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AABCinPK1H6xIM5yoGpLyexRnLLITJ62ks5u6LIrgaJpZM4ZBxAy>
.
|
For the purpose of communicating with another isolate, deeply immutable entities are required. So it makes sense that this proposal requires immutability to include anything reachable from an immutable object. But for many other purposes it could also be relevant to support a shallow notion of immutability. For example, it would be possible to offer developers support for safe sharing of standard data structures like lists and maps, in the same isolate, still allowing those data structures to have references to (that is "to contain") arbitrary objects, if the language were to support shallow immutability for those lists and maps. This means that it would still be possible for multiple agents to access, say, the elements of a shared list, and that could cause all the well-known problems associated with unwanted shared access; but this is no worse than any kind of aliasing, and this is something that Dart developers are dealing with every single day. When using such a shallowly immutable object o, that object itself can be trusted to stay the same (say "nobody else can change this list behind my back"), and for everything reachable from o it would simply be necessary to use the same plethora of techniques that we are already using in order to manage shared access to objects in general. This should not be hard to do (except, of course, for the fact that the notation used to specify immutability might be a bit more verbose now that we would have two variants, so we'll need additional syntax): We just need to apply a subset of the restrictions specified in the proposal. With such a generalization, this proposal could be considered as a response to #117 as well. |
As a newcomer to the thread it would be beneficial that the proposal comment has a summary section, answers the main viability questions or alternative solutions made in the comments. This makes it easier to catch up with ideas. A shared gdoc could be useful for that, because everyone can comment and suggest directly in the proposal. As an alternative to doing sealed Foo x = Foo()
var y = x // invalid, can only be assigned to other sealed vars
sealed z = x.bar
var zz = x.bar // invalid, x only returns sealed vars The reason I name it sealed is that In #758 sealed functions are proposed. In a sealed var, only sealed methods can be invoked. This would not require any special treatment or heap allocation, sealed references can be safely passed between isolates. Sealed var declarations are easier to approach, because they are defined from any object, they do not require a custom implementation. It is very constraining to define an immutable (sealed) var, so it should be analyzer suggested against it, unless the vars are shared between isolates. |
FYI: https://github.com/tc39/proposal-record-tuple TC39 looking at syntax for deeply immutable |
Isnt this resolved with |
No, records aren't required to be deeply immutable. This is fine: var list = [1, 2];
var record = (numbers: list);
list.add(3);
print(record.numbers); // Prints "[1, 2, 3]". |
This is a proposed solution to the problem of communicating data between isolates.
This proposal adds a static notion of immutability which guarantees that once initialized, any immutable object is the root of a fully immutable object graph, which can therefore be shared between isolates with no copying required.
This proposal also incidentally provides at least a partial solution for the following problems:
Initial draft proposal is here
The text was updated successfully, but these errors were encountered: