Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pandas Load Converters #42

Open
ieaves opened this issue Apr 4, 2020 · 1 comment
Open

Pandas Load Converters #42

ieaves opened this issue Apr 4, 2020 · 1 comment
Labels
enhancement New feature or request

Comments

@ieaves
Copy link
Collaborator

ieaves commented Apr 4, 2020

@sbrugman could you describe the problems you ran into with the converters here? I can take a look later
@ieaves ieaves added the enhancement New feature or request label Apr 4, 2020
@sbrugman
Copy link
Contributor

sbrugman commented Apr 6, 2020

A user-friendly of integrating visions in pandas is by using visions' type inference while continuing the pandas API with minimal changes. Pandas' read_csv functionality has the dtype and converters parameters that can be used for this purpose. Loading data directly with visions' type inference has the potential to speed up the complete process.

In pandas, converters and dtype force the usage of the Python engine, which is slower than the C engine, negating the possible performance gain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ieaves @sbrugman