Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic Transformer V2 #2792

Merged
merged 110 commits into from
Oct 28, 2024
Merged

Pydantic Transformer V2 #2792

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
333a05c
Pydantic Transformer V2
Future-Outlier Oct 8, 2024
9251508
add __init__.py
Future-Outlier Oct 8, 2024
4c46dee
add json schema
Future-Outlier Oct 8, 2024
63ab9fd
convert float to int
Future-Outlier Oct 8, 2024
357ca00
change gitsha in test script mode
Future-Outlier Oct 9, 2024
cdd1b25
change gitsha
Future-Outlier Oct 9, 2024
c89b59a
use strict map=false
Future-Outlier Oct 9, 2024
6b480da
Test flytefile console input + attr access
Future-Outlier Oct 9, 2024
15ce9ad
add conditional branch
Future-Outlier Oct 9, 2024
94ce092
better rx
Future-Outlier Oct 9, 2024
d97bc2a
Add flytedir generic -> flytedir
Future-Outlier Oct 9, 2024
4b0f008
merge async type engine
Future-Outlier Oct 10, 2024
afd4344
support enum
Future-Outlier Oct 10, 2024
773a3b6
update
Future-Outlier Oct 10, 2024
7afbb00
add tests for input from flyte console
Future-Outlier Oct 10, 2024
0258853
Add Tests for dataclass in BaseModel and pydantic.dataclass in BaseModel
Future-Outlier Oct 11, 2024
f0fa0b9
update
Future-Outlier Oct 11, 2024
f4e7581
Merge branch 'master' into pydantic-plugin-v2-with-msgpack
Future-Outlier Oct 11, 2024
dac2da4
update thomas's advice
Future-Outlier Oct 11, 2024
3531201
change tree file structure
Future-Outlier Oct 11, 2024
e34ef69
update Niel's advice
Future-Outlier Oct 11, 2024
2755c14
> to >=
Future-Outlier Oct 11, 2024
69e455d
try monodoc build again
Future-Outlier Oct 11, 2024
93e74ae
add pydantic README.md
Future-Outlier Oct 11, 2024
77ded14
revert -vvv in monodocs
Future-Outlier Oct 11, 2024
8e94aed
use model_validate_json to turn protobuf struct to python val
Future-Outlier Oct 11, 2024
f9e95f6
fix issue
Future-Outlier Oct 11, 2024
a37f8e3
handle flyte types in dict transformer from protobuf struct input (e.…
Future-Outlier Oct 14, 2024
26a848b
Add print
Future-Outlier Oct 14, 2024
029d159
expected python type
Future-Outlier Oct 14, 2024
f175218
switch call function order
Future-Outlier Oct 14, 2024
cbff823
try msgpack to handle protobug struct
Future-Outlier Oct 14, 2024
ae380fd
Better Comment in Dict Transformer
Future-Outlier Oct 14, 2024
28ef345
Propeller -> FlytePropeller
Future-Outlier Oct 14, 2024
55334e9
dict_to_flyte_types
Future-Outlier Oct 14, 2024
169ac0c
remove comments
Future-Outlier Oct 14, 2024
3613e0c
add attr for protobuf struct . dict
Future-Outlier Oct 14, 2024
ffce49d
Add Life Cycle for Flyte Types
Future-Outlier Oct 14, 2024
5c9be13
better comments for derializing flyteschema and sd
Future-Outlier Oct 14, 2024
429ccd5
nit
Future-Outlier Oct 15, 2024
8a8c6ce
add back pv._remote_path = None to flytefile and flytedir
Future-Outlier Oct 15, 2024
81b2169
experiment
Future-Outlier Oct 15, 2024
d72d72d
experiment
Future-Outlier Oct 15, 2024
6d6c112
experiment
Future-Outlier Oct 15, 2024
a860803
Add comments
Future-Outlier Oct 15, 2024
fb82dd5
update Yee's advice
Future-Outlier Oct 15, 2024
8eb45ff
code dc -> bm
Future-Outlier Oct 15, 2024
2edd542
Example dc -> bm and Example all flyte types
Future-Outlier Oct 15, 2024
989e6e0
fix union dataclass, not yet add comments
Future-Outlier Oct 15, 2024
ffd3aa2
solve conflict
Future-Outlier Oct 15, 2024
f699419
add pydantic and dataclass optional test
Future-Outlier Oct 15, 2024
86b34ab
NoneType=type(None)
Future-Outlier Oct 15, 2024
ebe61db
fix union transformer none case with Eduardo
Future-Outlier Oct 15, 2024
ac624d4
add comments for none type transformer + union transformer
Future-Outlier Oct 15, 2024
9982e8c
add TODO
Future-Outlier Oct 15, 2024
42ab3d3
Merge branch 'master' into pydantic-plugin-v2-with-msgpack
Future-Outlier Oct 17, 2024
85839e9
use deserailize = True
Future-Outlier Oct 17, 2024
f188415
add all deserialize
Future-Outlier Oct 17, 2024
90771fd
better comments
Future-Outlier Oct 17, 2024
1d8fe55
better comments
Future-Outlier Oct 17, 2024
e1b3f4f
test
Future-Outlier Oct 17, 2024
729ba27
Fix flyte directory issue by discussion with Kevin
Future-Outlier Oct 17, 2024
5bd3616
merge master
Future-Outlier Oct 18, 2024
9883c7f
add tests for providing conext when doing serialization
Future-Outlier Oct 18, 2024
79c9dc7
Merge branch 'master' into pydantic-plugin-v2-with-msgpack
Future-Outlier Oct 18, 2024
047b6ac
lint
Future-Outlier Oct 18, 2024
26a3745
test
Future-Outlier Oct 19, 2024
f07ec94
move setattr to core
Future-Outlier Oct 19, 2024
a64efa0
remove comments
Future-Outlier Oct 19, 2024
6394175
lint
Future-Outlier Oct 19, 2024
d6707a0
remove
Future-Outlier Oct 19, 2024
6ab30a0
testing
Future-Outlier Oct 19, 2024
3f39657
Merge branch 'master' into pydantic-plugin-v2-with-msgpack
Future-Outlier Oct 19, 2024
f2e165a
nit
Future-Outlier Oct 19, 2024
8e848bf
flytekit/core/type_engine.py
Future-Outlier Oct 19, 2024
7d291e3
add tests for Union
Future-Outlier Oct 21, 2024
5374a14
Trigger CI
Future-Outlier Oct 21, 2024
7d045be
remove nonetype
Future-Outlier Oct 21, 2024
8dd746a
raw=Fasle as default
Future-Outlier Oct 21, 2024
eba2bf0
Merge branch 'master' into pydantic-plugin-v2-with-msgpack
Future-Outlier Oct 22, 2024
753b240
pydantic move to core test
Future-Outlier Oct 22, 2024
a380def
move to core
Future-Outlier Oct 22, 2024
971aa47
log
Future-Outlier Oct 22, 2024
ff2d4a0
update
Future-Outlier Oct 22, 2024
6c3450b
lint
Future-Outlier Oct 22, 2024
5a1d58b
test
Future-Outlier Oct 22, 2024
4cfde94
nit
Future-Outlier Oct 22, 2024
e0d0a76
nit
Future-Outlier Oct 22, 2024
76c1d56
lint
Future-Outlier Oct 22, 2024
7972f95
move to type_engine
Future-Outlier Oct 22, 2024
6de3478
move back to init
Future-Outlier Oct 22, 2024
7c4c009
update kevin's advice
Future-Outlier Oct 23, 2024
4fc0622
wip
Future-Outlier Oct 23, 2024
edfc8ef
use decorator
Future-Outlier Oct 23, 2024
d8e4c6a
Merge branch 'master' into pydantic-plugin-v2-with-msgpack
Future-Outlier Oct 23, 2024
6c5b19f
decorator
Future-Outlier Oct 23, 2024
76ae0ef
fix syntax to support python 3.9
Future-Outlier Oct 24, 2024
dfe8762
add Eduardo's advice
Future-Outlier Oct 25, 2024
7735352
warning
Future-Outlier Oct 25, 2024
959f02b
Merge branch 'master' into pydantic-plugin-v2-with-msgpack
Future-Outlier Oct 25, 2024
85df643
Update Yee's advice
Future-Outlier Oct 28, 2024
f440dab
Show traceback by default (#2862)
pingsutw Oct 25, 2024
4f5f74e
Support Identifier in generate_console_url (#2868)
thomasjpfan Oct 25, 2024
edf995c
Support overriding node metadata for array node (#2865)
pvditt Oct 25, 2024
3dbb336
Fix Jupyter Versioning (#2866)
Mecoli1219 Oct 26, 2024
2bccd1b
improved output handling in notebooks (#2869)
kumare3 Oct 27, 2024
bf283f6
tests by Yee's suggestion
Future-Outlier Oct 28, 2024
6993f09
fix tests
Future-Outlier Oct 28, 2024
a3fef67
Merge branch 'master' into pydantic-plugin-v2-with-msgpack
Future-Outlier Oct 28, 2024
d983c07
format tests
Future-Outlier Oct 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/pythonbuild.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -356,7 +356,6 @@ jobs:
- flytekit-pandera
- flytekit-papermill
- flytekit-polars
- flytekit-pydantic
- flytekit-ray
- flytekit-snowflake
- flytekit-spark
Expand Down
1 change: 1 addition & 0 deletions dev-requirements.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ torch<=1.12.1; python_version<'3.11'
# pytorch 2 supports python 3.11
# pytorch 2 does not support 3.12 yet: https://github.com/pytorch/pytorch/issues/110436
torch; python_version<'3.12'
pydantic

# TODO: Currently, the python-magic library causes build errors on Windows due to its dependency on DLLs for libmagic.
# We have temporarily disabled this feature on Windows and are using python-magic for Mac OS and Linux instead.
Expand Down
Loading
Loading