reflect: add Type.CanSeq/CanSeq2 and Value.Seq/Seq2 methods

[earthboundkid]

Proposal Details

As mentioned in #61897 (comment), it can be tricky to determine if a type is rangeable via reflection. There should be an efficient way to do the check and an efficient way to range over a reflect.Value. The immediate need for this is because of rangefuncs, but this would also be useful for slices, maps, etc.

[rsc]

Retitled to use the more usual CanVerb form than having to invent a new word "rangeable".

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

On the Type side, the new interface method is CanRange() bool.
On the Value, the new method is unclear.
It's unclear if it should support just funcs or all rangeable things.
Let's assume the former, since "all things" is too hard.
At that point what's the signature of Value.FuncRange?
I assume it would have to take a yield function of the right signature,
but at that point you can use Value.Call instead.
Similarly, range over int can be handled by seeing that it is an int and doing the loop separately.
I'm not seeing any good APIs to add. Value.Call seems about as good.

Did you have a specific API in mind?

[earthboundkid]

My idea is that it would work like this:

v := reflect.Value(x)
if !v.Type().CanRange() { panic("bad type") }
for ranged := range v.Range() {
   if len(rangedVals) != 2 { /* e.g. it's an iter.Seq0 or iter.Seq */ panic("bad type") }
   key, val := ranged[0], ranged[1] // ranged is []reflect.Value of len between 0 and 2
   // etc
}

I think ideally it could work with all iterable types and just vary the length of the yielded slice to be as long as needed for slices, maps, iter.Seq[0], etc. This is semi-redundant with the existing map iterator, but I think the new API would be more convenient at the cost of being less performant for set operations.

If you wanted to know if it was specifically a range func you could do Kind == Func and CanRange.

[jimmyfrasche]

why not CanRange{1,2} and Range{1,2}. If you only need to 1-range there's no point in allocating the second value and adding a wrapper slice and if you need to be able 2-range it's not useful for a 1-rangeable to reply true when you query its rangeability

[ianlancetaylor]

A different approach would be

// Seq returns an iter.Seq[reflect.Value] that loops over the elements of v.
// If v's kind is Func, it must be a function that has no results and
// that takes a single argument of type func(T) bool for some type T.
// If v's kind is Pointer, the pointer element type must have kind Array.
// Otherwise v's kind must be Int, Int8, Int16, Int32, Int64, Uint, Uint8, Uint16, Uint32, Uint64, Uintptr,
// Array, Chan, Map, Slice, or String.
func (v Value) Seq() iter.Seq[Value]

// Seq2 is like Seq but for two values.
func (v Value) Seq2() iter.Seq2[Value, Value]

To support either version in the reflect package, when given a function, the reflect package would have to use MakeFunc to build a yield function of the appropriate type. That generated yield function would turn around and call the yield function passed in.

[DeedleFake]

why not CanRange{1,2} and Range{1,2}. If you only need to 1-range there's no point in allocating the second value and adding a wrapper slice and if you need to be able 2-range it's not useful for a 1-rangeable to reply true when you query its rangeability

Or Range() and Range2(). They fit with things like Slice() and Slice3() and Seq and Seq2 that way.

[Merovius]

I would have proposed @ianlancetaylor's signatures and IMO they are also what makes this proposal worth considering over just Value.Call - it handles the construction of the appropriate yield function for you. It could also, from an API perspective, easily work for slices, maps and channels.

As for naming, based on the conventions suggested for iteration APIs, the name should arguably be All. But that reads a bit weird on a generic type that could be anything - and CanAll feels very bad. I dislike Range, as for x := range v.Range() stutters. My favorite so far is Seq, as it would correctly describe what the method (in Ian's suggestion) does and CanSeq isn't any worse than CanInt to me.

Also, there should also be a Seq0/CanSeq0, I believe? At least my Go installation still allows the variant of ranging without values, even though there is no iter.Seq0, I lost track of whether or not that's intended and why.

[earthboundkid]

What does the Seq iterator yield for a slice? Just the indices wrapped as Values, so it matches x, _ := range s? That seems logical but unhelpful. Yielding the value is helpful but a little illogical.

[Merovius]

We could always resolve that ambiguity by only allowing Seq2 for slices (and maps) and Seq for channels.

[jimmyfrasche]

Should there be something like SeqJust2() iter.Seq[Value] that does for _, v := range s while avoiding extraneous allocation of the unneeded key?

[rsc]

It sounds like maybe people are happy with

// Seq returns an iter.Seq[reflect.Value] that loops over the elements of v.
// If v's kind is Func, it must be a function that has no results and
// that takes a single argument of type func(T) bool for some type T.
// If v's kind is Pointer, the pointer element type must have kind Array.
// Otherwise v's kind must be Int, Int8, Int16, Int32, Int64, Uint, Uint8, Uint16, Uint32, Uint64, Uintptr,
// Array, Chan, Map, Slice, or String.
func (v Value) Seq() iter.Seq[Value]

// Seq2 is like Seq but for two values.
func (v Value) Seq2() iter.Seq2[Value, Value]

along with CanSeq and CanSeq2.

[Merovius]

@rsc To clarify: "loops over the elements of v" means that it yields elements, not indices? Also, what does it yield for strings?

[rsc]

@Merovius perhaps the text should be made clearer, but the idea is that Seq and Seq2 would do exactly what range does.

[earthboundkid]

Looks good. The final docs should be explicit that Seq just returns an index Value for most Kinds and that Seq2 doesn't work for channels.

[rsc]

Have all remaining concerns about this proposal been addressed?

The proposal is in #66056 (comment).

[earthboundkid]

On #66631, the question came up of whether to avoid a dependency on the iter package. Should these maybe just be iterators directly instead of returning a named iter.Seq? Having the dependency in this direction could hurt if iter becomes a utility package of some kind. It could always fallback to reflectlite I guess and it’s speculation anyway. I could be convinced either way.

[rsc]

Iter is going to be a very low-level package and will not have significant dependencies added. Using it in reflect is fine.

[rsc]

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

The proposal is in #66056 (comment).

[earthboundkid]

Should I start working on a CL for the naive implementation of this, or does someone else already know how to do a more performant implementation?

[qiulaidongfeng]

Should I start working on a CL for the naive implementation of this, or does someone else already know how to do a more performant implementation?

I have an implementation that I haven't finished yet.

[gopherbot]

Change https://go.dev/cl/578815 mentions this issue: reflect: add iterative related methods

[earthboundkid]

@qiulaidongfeng, the behavior for Seq() on maps is to return the map values, not keys. I think that’s surprising. I think it should be like range and the Seq is just the keys, and the Seq2 is the keys and values. I’m not sure what it should do about slices.

[fzipp]

I’m not sure what it should do about slices.

@earthboundkid Russ' comment was clear:

the idea is that Seq and Seq2 would do exactly what range does

Seq returns the index for slices, and Seq2 index and value.

[qiulaidongfeng]

@qiulaidongfeng, the behavior for Seq() on maps is to return the map values, not keys. I think that’s surprising. I think it should be like range and the Seq is just the keys, and the Seq2 is the keys and values. I’m not sure what it should do about slices.

Thanks for you report. Now CL has solved the unfinished ones. CL can be review but submit need wait https://go-review.googlesource.com/c/go/+/557836.

[rsc]

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

The proposal is in #66056 (comment).

[gopherbot]

Change https://go.dev/cl/584515 mentions this issue: reflect: let documentation for Value.Seq and Value.Seq2 clearer

[qiulaidongfeng]

I noticed this 41d6687084f1837311021a134968479cb40f9021 after CL was committed.
According to the go specification, the type of the iterated variable should be the same as the type of the range integer.

For example, the iteration variable for range int8 should be int8, see https://go.dev/play/p/HYJzYbV3Suc prove.

But the existing reflect implementation, value.seq, iterates over integers and always returns a Value representing int64 or uint64, even for int8-like types.

Should this be done to make Value.Seq iterated integers conform to the go specification?
Make the implementation more complex, such as abstracting the range integer with a generic function and changing the iterated variable to the specified type with type arguments, and calling the generic function with the correct type arguments in Value.Seq.

[gopherbot]

Change https://go.dev/cl/584516 mentions this issue: reflect: let Value.Seq iterate integer conform to the spec