Refactor scatter.cuh and gather.cuh

[ttnghia]

This PR refactors scatter.cuh and gather.cuh, totally moving all the implementations of the templated functions detail::scatter and detail::gather into separates *.cu files with explicit instantiations.

Typically, this is not a trivial task since the APIs scatter and gather take templated iterator parameters (iterators for scatter/gather maps), and the types of those iterators can be very complex templated thrust::transform_iterator. In this PR, in order to explicitly instantiate the scatter and gather APIs, I have to limit the iterators types to be only pointers and thrust::constant_iterator. For thrust::transform_iterator, I create a temporary device_uvector and fill it with the transformed iterator values then pass its begin()/end() iterators to the scatter and gather APIs. Since most cudf APIs are passing scatter/gather maps from a device_uvector iterators, I don't think this would incur much overhead. The files affected by this are here: 37366cf.

After doing this (rough) refactoring, I gained about 5% speedup in compile time (on a 18 cores machine, 64 compile threads):
Before:

cudf: 6:48s
Others: 7:19s
All: 13:13s

After:

cudf: 6:31s
Others: 7:19s
All: 12:55s

It seems that 5% is not much worth do this, but if we merge this (after more cleanup) we will have a very nice scatter and gather APIs implementations: the header files *.cuh contain only declarations, and the implementations will be freed from including into many many others *cu file. Thus, I need more opinions from you guys about this before spending more time working on it.

---

PS: This is not a complete work, just a rough draft and still needs much cleanup.

[jrhemstad]

I am not a fan of requiring materializing any iterator based gather or scatter map into a device_uvector.

[ttnghia]

I am not a fan of requiring materializing any iterator based gather or scatter map into a device_uvector.

Me too. However, most (maybe 90% or so) cudf APIs already did it (use device_uvector) or even use a more expensive solution (materializing cudf::column).

[harrism]

THis should probably be a draft PR.

[harrism]

Yuck.

[harrism]

I don't like the idea of forcing materialization. That kills the benefit of an iterator interface.

cudf::reverse is an example of a case where we don't need to materialize the gather map. #8410

[ttnghia]

THis should probably be a draft PR.

I just tried to convert it to a draft but was told that "subscribed people will be unsubscribed" 😄, so just keep a normal (WIP) PR to let other participants know the activities here.

[ttnghia]

I don't like the idea of forcing materialization. That kills the benefit of an iterator interface.

cudf::reverse is an example of a case where we don't need to materialize the gather map. #8410

Yeap, this is a trade-off.

Based on the opinions, it seems that we should not continue this PR, since explicit instantiation for the APIs accepting any free iterator type is impossible.