Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: Consider switching from pandas to polars for performance improvements #168

Open
bgrnwd opened this issue Jun 5, 2024 · 2 comments
Assignees

Comments

@bgrnwd
Copy link
Member

bgrnwd commented Jun 5, 2024

What feature would you like to see?

https://docs.pola.rs/user-guide/getting-started/

@bgrnwd bgrnwd self-assigned this Jul 8, 2024
@bgrnwd
Copy link
Member Author

bgrnwd commented Jul 11, 2024

Doing some basic profiling with cProfile. Testing the code from the README

client = AmericanSoccerAnalysis()
asa_players = client.get_players(names="Andre")

Pandas

   240613 function calls (238081 primitive calls) in 21.651 seconds

   Ordered by: cumulative time
   List reduced from 1475 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      2/1    0.000    0.000   21.651   21.651 {built-in method builtins.exec}
        1    0.000    0.000   21.651   21.651 <string>:1(<module>)
        1    0.000    0.000   21.650   21.650 client.py:477(get_players)
        1    0.004    0.004   21.640   21.640 client.py:70(_get_entity)
        6    0.001    0.000   21.588    3.598 client.py:260(_execute_query)
       13    0.009    0.001   21.549    1.658 client.py:290(_single_request)
       13    0.000    0.000   21.117    1.624 sessions.py:593(get)
       13    0.000    0.000   21.116    1.624 sessions.py:502(request)
       13    0.000    0.000   20.950    1.612 sessions.py:673(send)
       13    0.000    0.000   20.679    1.591 adapter.py:50(send)

Polars

@bgrnwd
Copy link
Member Author

bgrnwd commented Sep 6, 2024

Running in to an issue when switching over from pandas to polars.

SchemaError: failed to determine supertype of struct[9] and struct[4]

It seems like polars is running in to an issue reading the JSON structure returned by the API and converting that in to a DataFrame.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

1 participant