Proposal: use a value other than nil for uninitialized interfaces

[pcostanza]

It's confusing that nil can be used both as a value to represent an uninitialized interface, and as a value for the pointer that an interface is initialized with. It's also very inconvenient to determine whether a pointer value in an interface is nil without casting it to the pointer type.

Proposal: use a value other than nil for uninitialized interfaces. To avoid adding new pre-defined identifiers, the value could be syntactically represented as {}, for example. This would allow expressing tests like this:

var v interface{}
...
if (v != {}) && (v != nil) {
   ...
}

The test against nil then doesn't need a cast and doesn't need to use the reflect package.

[davecheney]

My counter proposal is remove the "typed nil" scenario I wrote about here https://dave.cheney.net/2017/08/09/typed-nils-in-go-2 by making an nil interface and an interface which contains a type who's value is nil also equal to nil.

…

On Sat, 19 Aug 2017, 23:57 pcostanza ***@***.***> wrote: It's confusing that nil can be used both as a value to represent an uninitialized interface, and as a value for the pointer that an interface is initialized with. It's also very inconvenient to determine whether a pointer value in an interface is nil without casting it to the pointer type. Proposal: use a value other than nil for uninitialized interfaces. To avoid adding new pre-defined identifiers, the value could be syntactically represented as {}, for example. This would allow expressing tests like this: var v interface{} ... if (v != {}) && (v != nil) { ... } The test against nil then doesn't need a cast and doesn't need to use the reflect package. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#21538>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcAzr7lMeaS7v8emB6WoaD9HJHo1Xoks5sZunDgaJpZM4O8Vxm> .

[pcostanza]

i == nil being true both when the interface is uninitialized as well as when it's initialized with a nil pointer would require implicitly compiling the code to two tests rather than one. My suggestion would more directly maintain a correspondence between how many tests are expressed in the source code and how many tests the source code is being compiled to.

Also, your suggestion would require special handling in case you need to distinguish the two cases, through some additional constructs either as part of the language, or through some additional functions in the reflect package, for example. With my approach, you always have to distinguish the two cases, similar to what you already have to do now.

I'm not saying my approach is better than yours, or vice versa, since I don't have an opinion on this. Just trying to understand the differences...

[davecheney]

I'm pretty sure when people compare an interface value against nil, they want to know if the interface contains a valid implentation or not. I think for the case of determining the type and value inside an interface value, reflect is already capable of handling this so we can make the langauge simpler by removing a common gotcha (happens weekly across the support forums I participate in) without adding a new special case identifier.

…

On Sun, 20 Aug 2017, 22:32 pcostanza ***@***.***> wrote: i == nil being true both when the interface is uninitialized as well as when it's initialized with a nil pointer would require implicitly compiling the code to two tests rather than one. My suggestion would more directly maintain a correspondence between how many tests are expressed in the source code and how many tests the source code is being compiled to. Also, your suggestion would require special handling in case you need to distinguish the two cases, through some additional constructs either as part of the language, or through some additional functions in the reflect package, for example. With my approach, you always have to distinguish the two cases, similar to what you already have to do now. I'm not saying my approach is better than yours, or vice versa, since I don't have an opinion on this. Just trying to understand the differences... — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#21538 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA71TxuEYJEuoyeTmEiTEFeUwPuvPks5saCdfgaJpZM4O8Vxm> .

[faiface]

@davecheney

I'm pretty sure when people compare an interface value against nil, they want to know if the interface contains a valid implentation or not.

The thing is, that a nil value (not a nil interface) can be a perfectly valid implementation. And we are not only talking about nil pointers, which are usually not valid implementations, but can be. We are also talking about nil slices, maps and channels, which are valid and used.

[davecheney]

The thing is, that a nil value (not a nil interface) can be a perfectly

valid implementation. Indeed. This is absolutely correct, but I argue _ifff_ nil was a valid implementation, the caller wouldn't be checking to see if the implementation was nil. But this proposal would not remove your ability to do type T struct {} func (t *T) Foo() { ... } type I interface { Foo() } var t *T var i T = t i.Foo()

…

On Mon, Aug 21, 2017 at 9:08 AM, Michal Štrba ***@***.***> wrote: @davecheney <https://github.com/davecheney> I'm pretty sure when people compare an interface value against nil, they want to know if the interface contains a valid implentation or not. The thing is, that a nil value (not a nil interface) can be a perfectly valid implementation. And we are not only talking about nil pointers, which are usually not valid implementations, but can be. We are also talking about nil slices, maps and channels, which are valid and used. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21538 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA7Nu8KwzK0m0w3jQUu1skKQJtWVEks5saLyJgaJpZM4O8Vxm> .

[faiface]

This is absolutely correct, but I argue ifff nil was a valid implementation, the caller wouldn't be checking to see if the implementation was nil.

I don't think so. A function/struct which accepts/stores an interface value has no idea nor control over the concrete implementations that get passed in. Therefore it can't assume that there is no valid nil pointer/slice/map/chan implementation.

[slrz]

Exactly. It's not a reasonable thing to test for. You shouldn't care.

Maybe I'm missing something. Can someone point to some place in the standard library where a check like this is put to good use?

[bcmills]

Another alternative would be to provide some sort of mechanism to declare types which have no zero-value.

var v nonnil[interface{}]  // ERROR: nonnil value initialized to nil.
...
if v != nil {  // ERROR: v is nonnil.
   ...
}
if v != (*someType)(nil) {  // OK
   ...
}

I believe that might also address some of the same concerns as #21737.

I feel like there is probably a concrete proposal somewhere for types without zero-values; anybody have a link handy? (I'm not having any luck finding it in the tracker.)

[jimmyfrasche]

@bcmills non-nil infects.

Say I have a struct like type S struct { f pkg.T } where T is a struct.

If I update pkg and after the update T contains a field with a non-nil type.

Everywhere in my package where I have var s S is now a compiler error and the same for anyone using my package who has var s mypkg.S in their code.

[bcmills]

If I update pkg and after the update T contains a field with a non-nil type.

Yes, that would be a backward-incompatible change: it changes the type from one that has a meaningful zero-value to one that does not, and the compiler would then enforce that the zero-value is not used.

But note that you can already do that today without compiler enforcement: for example, if you add a sync.Cond field to an existing struct type, the zero-value of the struct will have a Cond with a nil L field and panic when used.

(You could imagine a concrete proposal for types without zero-values that allows them to be declared but not used unless they are first assigned on all paths.)

[pcostanza]

Go is unusual in that it's possible to define methods that do something meaningful when invoked on nil values, rather than just panicking like in other languages (Java, C++, ...). The fact that people tend to write methods that panic on nil values in Go nevertheless is probably more an educational problem rather than a technical one. (Accessing fields through nil pointers is still problematic, though.)

My proposal is meant to solve the issue that people can get easily confused by nil having two very different meanings when used with interfaces. Untyped nil is effectively a very different value semantically than typed nil. This is something both programmers need to keep in mind as well as Go compilers. In other words, there is no benefit in representing typed and untyped nil with the same identifier, it's just confusing (literally). What I'm proposing is to just make that distinction explicit by using a different identifier or syntax. (I'm not married to the '{}' syntax, it could also be 'empty' or something else...)

I don't care much about changing anything else around nil. (Allowing a comparison against bare 'nil' to mean a comparison against any typed nil is not very important to me either, it just occurred to me to be a neat opportunity to add that as well. If this is dropped, I'm fine with it. I care a lot more about the syntactic distinction between typed and untyped nil.)

[cznic]

Go is unusual in that it's possible to define methods that do something meaningful when invoked on nil values, rather than just panicking like in other languages (Java, C++, ...).

I agree, but let me add an observation. Without loss of correctness one can s/nil/zero/ in the quotation above and it becomes far less unusual, because methods, after all, are just syntactical sugar and there's nothing unusual in, for example, func foo(receiver bar, arg baz) called with zero bar (think eg. bar is int). IMO, it's the opposite, it's unusual methods are not allowed on zero values in other languages. It's even bad, because it's an implementation detail leaking into the leanguage. The reason is that (some of) those languages prefix the instance in memory with a vtab pointer, so they must dereference to call a method but cannot do that with nil (assuming non static methods).

[rogpeppe]

@bcmills Would it be possible to have channels containing these non-nil types? Maps? If so, what would happen if you received on the closed channel or indexed the map with a non-existent key?

[bcmills]

@rogpeppe

Would it be possible to have channels containing these non-nil types? Maps?

I could see reasonable designs in either direction.

I would expect nonnil T to be assignable to T in the same way a directionless channel is assignable to a directional one, so the biggest difficulty would occur for structs with nonnil fields — but then you could always store or send nillable pointers to those structs rather than values, in much the same way that you would currently store pointers to structs containing a sync.Mutex or other noncopyable type.

If so, what would happen if you received on the closed channel or indexed the map with a non-existent key?

If we were to allow chan T or map[K]T for a T containing nonnil fields, we would presumably want to specify that only the , ok form is allowed for receives and lookups and the variable is treated as unassigned unless ok is true. Determining whether a given statement is only reachable from true paths is a hard problem in the general case, so we would probably also need to specify a control-flow algorithm to ensure consistency. (Something simple like the spec for Terminating statements would likely suffice.)

[jba]

@bcmills How would you create a slice or array of a nonnil type? Other than with literals, that is.

[bcmills]

How would you create a slice or array of a nonnil type?

First thought: you could type-assert a []nonnil T from a corresponding []T. If any of the values is nil, the type-assertion fails.

Making that work for arrays would presumably require the same control-flow algorithm I suggested for channels and maps above (#21538 (comment)).

[jba]

How do you propose to handle cases like the following, where f is a function returning int, and g is a function returning T? I don't see how control-flow analysis will help you.
1.

c := make(chan nonnil T)
a := make([]nonnil T, 2)
...
a[f()], ok <- c
if !ok {
    fmt.Println(a[0])
}

b := make([]T, 1)
b[0] = g()
a := b.([]nonnil T)

[bcmills]

How do you propose to handle cases like the following

Same way you handle any other case a type system cannot express: weaken the types in the code until it can be type-checked, then add checked assertions to strengthen the types as needed for the exported API.

c := make(chan nonnil T)
an := make([]T, 2)
...
an[f()], ok = <- c
if !ok {
    fmt.Println(an[0])
}
a := an.([]nonnil T)  // Dynamic check here.

b := make([]T, 1)
b[0] = g()
a := b.([]nonnil T)  // Dynamic check here.

[jba]

Those dynamic checks have to examine every element of the slice, right? That would make them the only type assertions that are not constant time.

[rogpeppe]

First thought: you could type-assert a []nonnil T from a corresponding []T. If any of the values is nil, the type-assertion fails.

Does that mean that you can't append to a []nonnil T ? (appending creates zero elements beyond the elements appended)

[bcmills]

Does that mean that you can't append to a []nonnil T ? (appending creates zero elements beyond the elements appended)

Hmm, interesting point. In my mind, nonnil would apply only to elements with indices in the range [0, len), so nils in the range [len, cap) don't matter. But that brings us to the problem of covariance: it's not safe to allow a []T to alias a []nonnil T, since a nil-write through the former would be visible through the latter.

Combining that with @jba's observation that the type-assertion approach would be O(N) for what is otherwise an O(1) operation, perhaps a type-assertion is not a good solution after all.

So here's another idea: perhaps []T and []nonnil T are completely independent types, and the way you convert between the two is by copying (or appending). Under that approach, the non-nil elements in a []nonnil T are those at indices [0, len), and elements in [len, cap) may be nil, but are not observable without reslicing, and reslicing a []nonnil T would require nil-checks.

With that approach, @jba's examples become:

c := make(chan nonnil T)
an := make([]T, 2)
...
an[f()], ok = <- c
if !ok {
    fmt.Println(an[0])
}
a := mustappend(make([]nonnil T, 0, len(an)), an...)

b := make([]T, 1)
b[0] = g()
a := []nonnil T{b[0]}

(2) could be written a lot of different ways, depending on how you want to structure the control-flow checks. For example, if you don't want to allow it to peek through index assignments, you could use a loop:

b := make([]T, 1)
b[0] = g()
…
a := make([]nonnil T, 0, len(b))
for _, x := range b {
	if x == nil {
		panic("unexpected nil in b")
	}
	a = append(a, x)
}

[rogpeppe]

Another thought: would you be allowed arrays of nonnil type? If so, how would you fill them? I think it's reasonable to assume that it's not always reasonable to have an array literal of the size of the array, as arrays can be large.

[jba]

Eiffel's addition of nonnil types is well-described in this paper. Their solution combines control flow analysis, type assertion, and a version of make for arrays that takes a fill value.

[ianlancetaylor]

If anybody wants to carry the nonnil discussion forward, please move it to an issue about that. It doesn't seem related to the actual proposal here.

#22729 is an extended version of this proposal, covering other kinds of types as well. Although this issue was earlier, closing this one in favor of the later one.

[jaekwon]

@ianlancetaylor, please re-open this issue. Your proposal is a valid proposal but specifically one that tries to be backwards compatible, with a lot of changes to the language grammar.

This proposal is a must simpler* proposal that I think deserves further discussion. (*simpler in terms of language complexity, not implementation or Go1->Go2 porting complexity).

For this proposal I propose that null be used in place of {}, because null has 2 L's whereas nil only has 1 L.

[jaekwon]

For the record, I keep running into this issue even after programming in Go full time for 5 years. I will personally not recognize nor endorse a Go2 that does not address this issue, regardless of trademark law. ;)

[jaekwon]

Upon reflection (pun unintended), I realized that what I really want is just a fast operator for checking whether an interface value is a type nil.

https://github.com/tendermint/tmlibs/blob/develop/common/nil.go#L10

How about, simply, v == typednil ? It's completely backwards compatible.

[ianlancetaylor]

@jaekwon I don't see a need to reopen this proposal, it can be carried forward on either of the other proposals.

[zigo101]

@jaekwon

For this proposal I propose that null be used in place of {}, because null has 2 L's whereas nil only has 1 L.

I feel none is better than null, for none means nothing.