Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Orca datatype convert issue #4476

Closed
hkvision opened this issue Apr 22, 2022 · 0 comments
Closed

Orca datatype convert issue #4476

hkvision opened this issue Apr 22, 2022 · 0 comments
Labels

Comments

@hkvision
Copy link
Contributor

hkvision commented Apr 22, 2022

More details at this PR: #4444

The data type conversion from RDD to numpy is complete. But adding the missing data type conversions will cause failures/modifications in many unit tests.

The main issue is that:

  • If user don't use numpy but simply python for their data; when converting to dataframe, the data types will always be double and long, no float/int.
  • Users may not be aware of the behavior above and if model expects float/int, will throw error when they get double/long as input.
  • For example in many of our unit tests, the dataframe type is wrong/not expected (actually array of double not float); and the data type conversion is also not complete (all convert to ndarray of float); but the two errors result the final input to be correct (torch tensor of float).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant