proposal: Go 2: add built-in null for zero value of pointers

[ianlancetaylor]

There is a long-standing confusion in Go between a nil interface value and a non-nil interface value that holds a pointer type with a value of nil. There is a FAQ entry for Why is my nil error value no equal to nil?. Despite the FAQ entry, this has been reported as a bug many times (#43643, #44102, #47551, #53768, #54229, #56930, #57292, #59521 among many others). See also the discussion at https://dave.cheney.net/2017/08/09/typed-nils-in-go-2.

There have been several attempts to address this, including at least #21538, #22729, #24635, #24682, #27890, #30294, #30865, #60786. None of these have worked.

Many of those previous attempts have tried to associate nil with pointers, and to one way or another stop using nil with interfaces. The problem with that is that so much Go code has lines like if err != nil, which is a comparison between an interface type and nil. It's not realistic to expect all of those lines of Go code to change. Any proposal that requires them to all change is a non-starter.

So let's consider this from the other side: change pointers.

I propose adding a new predeclared identifier null. The name null is like nil in that it is an untyped value. It may be assigned to any variable of pointer type, setting that variable to the zero value. It may be compared to any value of pointer type, reporting whether that value is the zero value. In other words, null is exactly like nil, but it may only be used with pointer types, not with interface types nor any other non-pointer type.

This has some similarities to the recent proposal for zero (#61372) in that it introduce yet another name for the zero value, albeit one that can only be used with pointers.

If this proposal is accepted, using pointer types with nil will continue to work. However, people can avoid confusion by consistently using null when they want a null pointer and nil when they want an empty interface. In particular, if people can start to consistently think of null pointers rather than nil pointers, then my hope is that writing err != nil will not suggest that it is a test of whether a pointer value stored in err is null. In other words, distinguishing nil and null should help people understand what happens when you convert a null pointer to interface type.

We can provide a rewriting tool that will change the use of nil with pointer types to use null instead. Over time we can introduce a vet check that will warn about uses of nil with pointer types. And, perhaps, we can eventually entirely disallow the use of nil with pointer types. (This kind of transition is compatible with the Go transition document which permits us to change the language to remove features that used to work, in this case the feature of using nil with pointers).

Although nil is also used with slices, maps, channels, and functions, I don't think it's necessary to change them to use null nor to introduce new names for their zero values, as they rarely seem to cause confusion with interface values.

The assumption here is that nil is used far more often with interfaces than with pointers, and that it is feasible to change pointers to stop using nil whereas it is not feasible to change interfaces. This assumption may be wrong.

To be clear, I doubt this proposal will be adopted. But I wanted to write it down as an idea to point to for people who want to address the nil confusion problem. And perhaps it will suggest some other proposal that might be adopted.

[robpike]

I am not sure about your assumptions, and I am not sure that the proposal would fix the most common confusion, which is the difference between a nil interface and a interface containing a type but no value. If we look at https://go.dev/doc/faq#nil_error, the confusion is not because there are two kinds of nil, but because the interface contains a nil but is not itself nil. Having a choice of null or nil in the language would not change the example there, which would still use nil everywhere except arguably in the unnecessary initialization of myError.

[rsc]

If we did this and wanted it to resolve the confusion, we would have to run a massive conversion campaign to change every instance of nil used as a pointer to say null instead (including in books, tutorials, and so on), and then disallow nil for pointers. Only at that point would there no longer be confusion, since x == nil would be for interfaces and x == null would be for pointers, and (hopefully) no one would think that assigning interface x = (*T)(null) would result in x == nil.

That is a nice plan if we were starting from scratch, but it seems like far too much churn to fix what in practice is an annoying but fairly small problem. The benefit does not seem worth the cost.

We could also flip things around and move to x == zero for interfaces and then disallow x == nil for interfaces. That would probably be less churn, since only interfaces are involved, and it would not involve adding a third name for a zero value, so it's probably preferable to adding null, but it would still be a lot of churn, and probably still more cost than benefit.

[rsc]

I suppose zero doesn't work after all because then the problem would become

var x any = time.Time(zero)
println(x == zero) // false

You need a name that is not shared by any other values.

That also kills the null-for-pointers idea, since you can still have

var x any = []int(nil)
println(x == nil) // false

The text says this is less common than pointers, which is true, but if we're going to cause a whole lot of churn, we might as well make the problem go away entirely. To eliminate every form of this problem, interface zeros need a name that is different from every other type's zero.

The name for an interface zero could be null, but I think it would be very confusing to have both null and nil. And it's still a lot of churn to fix a relatively small problem.

[ianlancetaylor]

Having a choice of null or nil in the language would not change the example there, which would still use nil everywhere except arguably in the unnecessary initialization of myError.

Yes. The hope here is that if people think in terms of nil interface values and null pointers, then they won't be surprised when err != nil returns true for a null pointer. Because nil and null are different.

We could also flip things around and move to x == zero for interfaces and then disallow x == nil for interfaces

But that would break all err != nil conditions, which (I posit) are more common than ptr == nil conditions.

I don't agree that the problem is "relatively small;" it gets reported, incorrectly, a lot (the list of issues above is incomplete and also doesn't count the people who report it on the mailing list). It's a stumbling block for many people learning Go. We don't have an answer, and it's not the end of the world if we never have an answer, but that doesn't mean that it's not a problem.

That said, the point of this proposal is not to actually adopt it. I guess I should have said that more clearly. It's to give us another place to point to for people who make proposals to address this problem, which are fairly frequent.

[ianlancetaylor]

To eliminate every form of this problem, interface zeros need a name that is different from every other type's zero.

For the record, #22729 was one attempt at that. It doesn't work either.

[rsc]

I scrolled a bit but didn't see why it doesn't work. Can you summarize in a sentence or two?

[ianlancetaylor]

It doesn't work because in the long run it requires all err != nil conditions to change to err != nilinterface. I've come to the view that that is simply a non-starter. I mean, probably anything in this area is a non-starter, but something like 1% of all Go code is err != nil and that's too much to change.

[apparentlymart]

I do understand that this situation is a common source of confusion for beginners with Go, but it's also been my experience that once someone learns the reason behind it they tend to understand why the language behaves that way and, as a bonus, leave with a better understanding of what an "interface value" is.

I've also seen Go beginners run into similar confusion with nil slices vs. zero-length slices, and nil maps vs. zero-length maps. The mental model required to understand this similar to the mental model required to understand why it's possible to store a nil pointer inside an interface value, though I will concede that at least in this case it isn't possible to spell a non-nil zero-length slice as nil, so the confusion is limited only to the slice == nil comparison situation. Still, if confusion about nil is the problem to be solved then I'd prefer a solution which solves for all the nil confusion, rather than just one example.

If given the luxury to start again I agree that it might have been helpful to use a different spelling for each type kind's zero value, but I am not convinced that adding a second spelling of the zero value of pointers at this late stage would actually help. Instead, I expect it to just raise additional questions about what the difference is between null and nil -- which have no clear mnemonic to remember which is which.

The only way I could support something like this proposal is if the new name were more obviously type-kind specific -- whether that be nilptr for zero-value pointers or something like nilinterface for zero-value interface types, depending on which one seems more deserving of the special name -- and if there were a clear path to making the undecorated nil actually invalid in that context in some actually-planned future version of Go (as opposed to a "maybe some day" Go 2).

[ianlancetaylor]

Based on the discussion above, this is a likely decline. Leaving open for three weeks for final comments.

[ianlancetaylor]

No further comments.