[FEA] Convert Binary Operations to use device-side type dispatch rather than JIT

[harrism]

JIT overhead for binary ops is a problem in some situations. @karthikeyann did an experiment a while back to use device-side dispatch rather than JIT. The compile time cost was low, and performance was good.

Compiled binops use a single compiled kernel (per operation) with device-side type dispatch, so the code should be simpler. The operation dispatch can be on the host, but there are only a small number of operations relative to the combinations of types and operations, so this should not be expensive.

[jrhemstad]

For this, I think it'd make sense to make a triple_type_dispatcher, ala

cudf/cpp/include/cudf/utilities/type_dispatcher.hpp

Lines 546 to 570 in 24f3016

	/**
	* @brief Dispatches two type template parameters to a callable.
	*
	* This function expects a callable `f` with an `operator()` template accepting
	* two typename template parameters `T1` and `T2`.
	*
	* @param type1 The `data_type` used to dispatch a type for the first template
	* parameter of the callable `F`
	* @param type2 The `data_type` used to dispatch a type for the second template
	* parameter of the callable `F`
	* @param args Parameter pack forwarded to the `operator()` invocation `F`.
	*/
	#pragma nv_exec_check_disable
	template <template <cudf::type_id> typename IdTypeMap = id_to_type_impl, typename F, typename... Ts>
	CUDA_HOST_DEVICE_CALLABLE constexpr decltype(auto) double_type_dispatcher(cudf::data_type type1,
	cudf::data_type type2,
	F&& f,
	Ts&&... args)
	{
	return type_dispatcher<IdTypeMap>(type1,
	detail::double_type_dispatcher_first_type<IdTypeMap>{},
	type2,
	std::forward<F>(f),
	std::forward<Ts>(args)...);
	}

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[karthikeyann]

Tried triple_dispatcher. Compile time for each triple_dispatcher adds 60 seconds.
So, going with different approach. (common_type type_dispatcher, and double_type_dispatcher).

[cwharris]

JIT overhead for binary ops is a problem in some situations.

Can we get more specific here? How we handle JIT compilation was recently changed with jitify2. Is it possible some of these problems are alleviated?

edit: I missed the link in the original post.

[harrism]

@cwharris one important Spark use case runs on platforms where they can't write a JIT cache so every worker pays the JIT overhead.

[karthikeyann]

integration to sparks, python pending.