Support Windows in CUDA version

[StrikerRUS]

Right now CUDA version can be built only on Linux. The main blocker of making CUDA working on Windows is

Unfortunately the CUDA changes for LightGBM require pthreads, as of now, and Windows doesn't (easily) support pthreads.
#3160 (comment)

Source code references:

LightGBM/src/treelearner/cuda_tree_learner.h

Line 243 in a15a370

pthread_t **cpu_threads_; // pthread, 1 cpu thread / gpu

LightGBM/src/treelearner/cuda_tree_learner.cpp

Lines 207 to 218 in 045ffd9

	for (int device_id = 0; device_id < num_gpu_; ++device_id) {
	if (pthread_create(cpu_threads_[device_id], NULL, launch_cuda_histogram, reinterpret_cast<void *>(&thread_data[device_id]))) {
	Log::Fatal("Error in creating threads.");
	}
	}
	/* Wait for the threads to finish */
	for (int device_id = 0; device_id < num_gpu_; ++device_id) {
	if (pthread_join(*(cpu_threads_[device_id]), NULL)) {
	Log::Fatal("Error in joining threads.");
	}
	}

LightGBM/src/treelearner/cuda_tree_learner.cpp

Lines 490 to 494 in e31244c

	// set cpu threads
	cpu_threads_ = reinterpret_cast<pthread_t **>(_mm_malloc(sizeof(pthread_t *)*num_gpu_, 16));
	for (int device_id = 0; device_id < num_gpu_; ++device_id) {
	cpu_threads_[device_id] = reinterpret_cast<pthread_t *>(_mm_malloc(sizeof(pthread_t), 16));
	}

Code blocks listed above should be rewritten in a cross-platform manner.

Or if it is impossible, pthread's port for Windows can be properly added in source files tree to make code above works with Visual Studio. For example,

• https://github.com/coapp-packages/pthreads,
• https://github.com/GerHobbelt/pthread-win32,
• https://nachtimwald.com/2019/04/05/cross-platform-thread-wrapper/,
• https://demin.ws/blog/russian/2009/01/26/universal-threads-for-windows-and-unix/.

[StrikerRUS]

Closed in favor of being in #2302. We decided to keep all feature requests in one place.

Welcome to contribute this feature! Please re-open this issue (or post a comment if you are not a topic starter) if you are actively working on implementing this feature.

[StrikerRUS]

@shiyu1994
Are there any plans for supporting new CUDA implementation on Windows? #4528

[shiyu1994]

@StrikerRUS Thanks for proposing this, and thanks for the investigation! Definitely, we have plan for CUDA version on Windows, but not necessarily in 4.0.0. If the changes for windows require only minor revision, we may try to include it in 4.0.0. I think by now we'd better focus on make the new CUDA version stable on Linux. And then we can head for Windows. WDYT?

[StrikerRUS]

@shiyu1994 Yeah, totally fine! I just wanted to hear your opinion on this issue and further plans. Thanks!

[jameslamb]

Sorry, this was locked accidentally. Just unlocked it. We'd still love help with this feature!