Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: [CUDA] New CUDA version #4528

Closed
wants to merge 70 commits into from

Conversation

shiyu1994
Copy link
Collaborator

@shiyu1994 shiyu1994 commented Aug 16, 2021

Current GPU and CUDA versions of LightGBM only calculate histograms on GPU. This is a new CUDA version for LightGBM which runs the whole training process on GPU.

@StrikerRUS
Copy link
Collaborator

@shiyu1994

One option would be to keep the old cuda version as cuda_hist, and the new cuda version as cuda in the device type parameter.

Does new version reuse current CUDA histograms building code?

@shiyu1994
Copy link
Collaborator Author

@StrikerRUS No, there's no dependence on the old CUDA histogram code.

@StrikerRUS
Copy link
Collaborator

@shiyu1994 Maybe then we can name new version as cuda_exp (from experimental, or any other better naming) and after some number of releases remove current cuda code and rename cuda_exp into normal cuda?

@shiyu1994
Copy link
Collaborator Author

@StrikerRUS I agree. Let's add the new CUDA version as cuda_exp for some versions, and replace the old CUDA version with it until it stabilizes.

Copy link
Collaborator

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just leaving a "request changes" review to be sure this isn't accidentally merged until we've agreed that enough time has passed since 3.3.0.

Relevant comment: #4633 (comment)

@GeorgePearse
Copy link

How can I build this branch?

@shiyu1994
Copy link
Collaborator Author

@GeorgePearse Thanks for using LightGBM. To build this branch, you may clone from https://github.com/shiyu1994/LightGBM/tree/cuda-new-refactor-2 and follow the build instruction of LightGBM CUDA version here. https://lightgbm.readthedocs.io/en/latest/Installation-Guide.html#build-cuda-version-experimental
Note that this is still a in-progress PR and there may be some bugs remain to be fixed.

@GeorgePearse
Copy link

@shiyu1994 Thanks! Just need to try to find the fastest training model possible. Stability not a worry. Is it useful for me to add any bugs I encounter here? Or is there good enough test coverage that I'll just be restating what's already known.

@shiyu1994
Copy link
Collaborator Author

shiyu1994 commented Nov 5, 2021

@GeorgePearse Yes, if you find any bug you may post it here. There are some features that CUDA version of this branch does not support, including training with categorical features. Other additional features during training, like monotonic constraints, extremely random trees are not supported.

I'm now working with #4630, which only dispatches the tree learner part to CUDA. That branch is relatively more complete. I'll finish #4630 first, and then merge additional contents in this PR.

Thanks for your support.

@jameslamb
Copy link
Collaborator

I'm now working with #4630, which only dispatches the tree learner part to CUDA. That branch is relatively more complete. I'll finish #4630 first, and then merge additional contents in this PR.

@shiyu1994 is it still your plan to merge master into this PR branch? Or should this be closed and new PRs opened for additional CUDA changes.

@shiyu1994
Copy link
Collaborator Author

Or should this be closed and new PRs opened for additional CUDA changes.

@jameslamb I believe we should close this. I'll open additional PRs for the rest CUDA work. Thanks!

Copy link

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 15, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants