Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: python backend support for numpy>=2 #384

Merged
merged 1 commit into from
Nov 1, 2024

Conversation

KrishnanPrash
Copy link
Contributor

Updated the PyBind11 version from v2.10.0 to v2.12.0, because v2.12 supports numpy 2 (Release Notes).

Before this update, if numpy>=2.x was installed in the same environment, performing inference requests would lead to error messages, such as:

terminate called after throwing an instance of 'pybind11::error_already_set'
  what():  error: unpack_from requires a buffer of at least 50529031 bytes for unpacking 50529027 bytes at offset 4 (actual buffer size is 7)

At:
  /opt/tritonserver/backends/python/triton_python_backend_utils.py(123): deserialize_bytes_tensor

or

File "/opt/.../py310/lib/python3.10/site-packages/tritonclient/grpc/_infer_result.py", line 93, in as_numpy
    np_array = np_array.reshape(shape)
ValueError: cannot reshape array of size 0 into shape (1,1)

@rmccorm4 rmccorm4 requested review from kthui and krishung5 October 18, 2024 01:28
Copy link
Contributor

@rmccorm4 rmccorm4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but let's get a 2nd opinion from Kris or Jacky before merging. Also if it makes sense, let's enable numpy>=2 install for something like L0_backend_python to confirm it works and catch any edge cases.

Copy link
Contributor

@krishung5 krishung5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes lgtm! +1 on enabling numpy>=2 install L0_backend_python test.

@TheMightyRaider
Copy link

TheMightyRaider commented Oct 22, 2024

@KrishnanPrash , I still faced the issue with numpy==2.1.2 and pybind11==2.13.6. I was able to solve the issue by downgrading numpy to 1.25.1.

But shouldn't numpy=2.1.2 with pybind11=2.13.6 work as you have mentioned?

Let me know if you need more details about my code!

@rmccorm4
Copy link
Contributor

rmccorm4 commented Nov 1, 2024

Testing PR: triton-inference-server/server#7756

@KrishnanPrash KrishnanPrash merged commit 09c3537 into main Nov 1, 2024
3 checks passed
@KrishnanPrash KrishnanPrash deleted the kprashanth-numpy2-support branch November 1, 2024 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants