Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] fp_quantizer is not correctly built when non-jit installation #5535

Closed
twaka opened this issue May 14, 2024 · 2 comments · Fixed by #5577
Closed

[BUG] fp_quantizer is not correctly built when non-jit installation #5535

twaka opened this issue May 14, 2024 · 2 comments · Fixed by #5577
Assignees
@loadams
Labels
bug Something isn't working inference

Comments

@twaka
Copy link

twaka commented May 14, 2024

Describe the bug
fp_quantizer is not correctly built when non-jit installation.

To Reproduce
Steps to reproduce the behavior:

DS_BUILD_FP_QUANTIZER=1 pip install deepspeed

install will succeed but

from deepspeed.ops.fp_quantizer import FP_Quantize
FP_Quantize()

will raise ImportError: dynamic module does not define module export function (PyInit_fp_quantizer_op)

Expected behavior

Renaming csrc/fp_quantizer/quantize.cu may solve the issue.
This restriction seems to be cause of the bug.

Note that setuptools cannot handle files with the same name but different extensions
https://pytorch.org/tutorials/advanced/cpp_extension.html
@twaka twaka added bug Something isn't working inference labels May 14, 2024
@loadams
Copy link
Contributor

loadams commented May 21, 2024

Thanks for pointing this out @twaka - we will take a look

@loadams loadams self-assigned this May 21, 2024
@loadams
Copy link
Contributor

loadams commented May 28, 2024 •
edited
Loading

@twaka Can you try #5577?

@loadams loadams mentioned this issue May 28, 2024
Merged
loadams added a commit to nelyahu/DeepSpeed that referenced this issue May 28, 2024
@loadams
Rename files in fp_quantize op from quantize.* to fp_quantize.* (micr…
bf66acd 
…osoft#5577)

Fixes microsoft#5535.

Todo: need to test.
sfc-gh-reyazda pushed a commit to Snowflake-Labs/DeepSpeed that referenced this issue Jun 10, 2024
@loadams @sfc-gh-reyazda
Rename files in fp_quantize op from quantize.* to fp_quantize.* (micr…
5e5c8a7 
…osoft#5577)

Fixes microsoft#5535.

Todo: need to test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@loadams loadams

Labels
bug Something isn't working inference
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Rename files in fp_quantize op from quantize.* to fp_quantize.* microsoft/DeepSpeed
2 participants
@twaka @loadams