-
Notifications
You must be signed in to change notification settings - Fork 43
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add add pandas_options in load_file (#1466)
# Description closes: #1519 ## What is the current behavior? currently, load_file do not have an option to pass the pandas-related param while reading file ## What is the new behavior? use `load_options` and pass the given values while reading files using the pandas path ## Does this introduce a breaking change? No ### Checklist - [ ] Created tests which fail without the change (if possible) - [ ] Extended the README / documentation, if necessary Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- Loading branch information
1 parent
ad0c4df
commit 6850b44
Showing
17 changed files
with
211 additions
and
15 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
from __future__ import annotations | ||
|
||
from attr import define | ||
from pandas._typing import DtypeArg | ||
|
||
from astro.options import LoadOptions | ||
|
||
|
||
@define | ||
class PandasLoadOptions(LoadOptions): | ||
pass | ||
|
||
|
||
@define | ||
class PandasCsvLoadOptions(PandasLoadOptions): | ||
delimiter: str | None = None | ||
dtype: DtypeArg | None = None | ||
|
||
|
||
@define | ||
class PandasJsonLoadOptions(PandasLoadOptions): | ||
encoding: str | None = None | ||
|
||
|
||
@define | ||
class PandasNdjsonLoadOptions(PandasLoadOptions): | ||
normalize_sep: str = "_" | ||
|
||
|
||
@define | ||
class PandasParquetLoadOptions(PandasLoadOptions): | ||
columns: list[str] | None = None |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.