Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Signed Integer Overflow in TR_RandomSeedSignatureHash #1794

Closed
Tracked by #2055
mgaudet opened this issue Oct 6, 2017 · 2 comments · Fixed by #3851
Closed
Tracked by #2055

Signed Integer Overflow in TR_RandomSeedSignatureHash #1794

mgaudet opened this issue Oct 6, 2017 · 2 comments · Fixed by #3851

Comments

@mgaudet
Copy link

mgaudet commented Oct 6, 2017

Minor issue :)

https://github.com/eclipse/omr/blob/8595d1c3b98b623cf696f9e1cb8a8ae1c94e0925/compiler/compile/OMRCompilation.cpp#L338-L347

../compiler/compile/OMRCompilation.cpp:342:19: runtime error: signed integer overflow: 33 * 1856584571 cannot be represented in type 'int'

Discovered by running Tril's compiler tests with -fsanitized=undefined

@mstoodle
Copy link
Contributor

Would this also fail if hash was uint32_t ?

@mgaudet
Copy link
Author

mgaudet commented Oct 23, 2017

Unsigned overflow is defined behaviour: So yes, it would pass by changing that type.

wbh123456 added a commit to wbh123456/omr that referenced this issue May 14, 2019
The hashing function used signed intger for varaible hash and seed,
which could lead to integer overflow. Fix the issue by changing the
signed integer type to unsigned.

Closes: eclipse-omr#1794
Signed-off-by: Bohao(Aaron) Wang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants