Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

basic level typechecking error in polars integration documentation/example #1863

Open
Molier opened this issue Nov 22, 2024 · 1 comment
Open
Labels

Comments

@Molier
Copy link

Molier commented Nov 22, 2024

Location of the documentation

https://pandera.readthedocs.io/en/latest/polars.html

Documentation problem

Issues:

  1. The initial example in the documentation shows validation of a LazyFrame but requires .collect(), which is confusing
  2. The example code has type checking errors with modern type checkers, pylance at "basic"
  3. Return types from validate() don't properly indicate Polars methods

Example of current type errors:
image

and example in my code: image.

This would help new users better understand the intended usage pattern and avoid confusion with type checking.

I am running into a lot of basic level type checking errors using pandera. I'm quite new to type checking, but trying to write more robust code and thus use proper validation on my data when using polars.

Wording

The docs also say which we then use to validate a polars.LazyFrame object. Which is weird since the code beneath needs to 'collect' to validate, thus forgoing the lazy optimizations?

Suggested fix for documentation

I don't really know what the cause of this is or how to improve it, but i keep running into a lot of type errors when using pandera.

Suggested Improvements:

  1. Update the initial example to clearly show the recommended validation pattern
  2. Add proper type hints for Pandera's return types
  3. Move the explanation of LazyFrame vs DataFrame validation earlier in the docs
@Molier Molier added the docs label Nov 22, 2024
@Molier
Copy link
Author

Molier commented Nov 22, 2024

core issue is that
image
at https://pandera.readthedocs.io/en/latest/polars.html#usage is giving type error for .collect()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant