[Docs] MessagePack IDL, Pydantic Support, and Attribute Access #6022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Future-Outlier <[email protected]>
Future-Outlier
changed the title
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #6022 +/- ##
==========================================
+ Coverage 37.03% 37.07% +0.04%
==========================================
Files 1313 1316 +3
Lines 131622 132191 +569
==========================================
+ Hits 48742 49007 +265
- Misses 78652 78922 +270
- Partials 4228 4262 +34
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
Signed-off-by: Future-Outlier <[email protected]>
|
|
||
| Flytekit version >= v1.14.0 will produce msgpack bytes literal for dataclasses. | ||
|
|
||
| If you're using Flytekit version >= v1.14.0 and you want to produce protobuf struct literal for dataclasses, you can |
There was a problem hiding this comment.
It would be good to mention why would a user want to produce protobuf struct literal instead of msgpack bytes
| .. tags:: Basic | ||
| ``` | ||
|
|
||
| When you've multiple values that you want to send across Flyte entities, and you want them to have, you can use a `pydantic.BaseModel`. |
There was a problem hiding this comment.
Not sure I understand this line
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
|
@Future-Outlier left some suggestions to improve clarity. I hope it helps |
Co-authored-by: David Espejo <[email protected]> Signed-off-by: Han-Ru Chen (Future-Outlier) <[email protected]>
Co-authored-by: David Espejo <[email protected]> Signed-off-by: Han-Ru Chen (Future-Outlier) <[email protected]>
thank you so much David, will finish this today |
Co-authored-by: David Espejo <[email protected]> Signed-off-by: Han-Ru Chen (Future-Outlier) <[email protected]>
Co-authored-by: David Espejo <[email protected]> Signed-off-by: Han-Ru Chen (Future-Outlier) <[email protected]>
Co-authored-by: David Espejo <[email protected]> Signed-off-by: Han-Ru Chen (Future-Outlier) <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
| With the 1.14 release, `flytekit` adopted `MessagePack` as the | ||
| serialization format for dataclasses, overcoming a major limitation of serialization into a JSON string within a Protobuf `struct` datatype, like the previous versions do: | ||
|
|
||
| to store `int` types, Protobuf's `struct` converts them to `float`, forcing users to write boilerplate code to work around this issue. |
There was a problem hiding this comment.
no need to insert a new line
There was a problem hiding this comment.
I think insert a new line is more readable
| By default, `flytekit >= 1.14` will produce `msgpack` bytes literals when serializing dataclasses. | ||
|
|
||
| :::{important} | ||
|
|
There was a problem hiding this comment.
Suggested change
| By default, `flytekit >= 1.14` will produce `msgpack` bytes literals when serializing dataclasses. |
| :::{important} | ||
|
|
||
| If you're serializing dataclasses using `flytekit` version >= v1.14.0, and you want to produce Protobuf `struct | ||
| literal` instead, you can set environment variable `FLYTE_USE_OLD_DC_FORMAT` to `true`. |
There was a problem hiding this comment.
no need to insert a new line, I think it breaks the code format
Comment on lines
31
to
35
| Flytekit version < v1.14.0 will produce protobuf struct literal for dataclasses. | ||
|
|
||
| Flytekit version >= v1.14.0 will produce msgpack bytes literal for dataclasses. | ||
|
|
||
| If you're using Flytekit version >= v1.14.0 and you want to produce protobuf struct literal for dataclasses, you can |
There was a problem hiding this comment.
Suggested change
| Flytekit version < v1.14.0 will produce protobuf struct literal for dataclasses. | |
| Flytekit version >= v1.14.0 will produce msgpack bytes literal for dataclasses. | |
| If you're using Flytekit version >= v1.14.0 and you want to produce protobuf struct literal for dataclasses, you can |
| Flytekit version >= v1.14.0 will produce msgpack bytes literal for dataclasses. | ||
|
|
||
| If you're using Flytekit version >= v1.14.0 and you want to produce protobuf struct literal for dataclasses, you can | ||
| set environment variable `FLYTE_USE_OLD_DC_FORMAT` to `true`. |
There was a problem hiding this comment.
Suggested change
| set environment variable `FLYTE_USE_OLD_DC_FORMAT` to `true`. |
This was already mentioned above
Also in the readthedocs build, you can see there are two important blocks nested
| ``` | ||
|
|
||
| `flytekit` version >=1.14 supports natively the `JSON` format that Pydantic `BaseModel` produces, enhancing the | ||
| interoperability of Pydantic BaseModels with the Flyte type system. |
There was a problem hiding this comment.
no need to insert new line
|
|
||
| With the 1.14 release, `flytekit` adopted `MessagePack` as the serialization format for Pydantic `BaseModel`, | ||
| overcoming a major limitation of serialization into a JSON string within a Protobuf `struct` datatype like the previous versions do: | ||
|
|
| With the 1.14 release, `flytekit` adopted `MessagePack` as the serialization format for Pydantic `BaseModel`, | ||
| overcoming a major limitation of serialization into a JSON string within a Protobuf `struct` datatype like the previous versions do: | ||
|
|
||
| to store `int` types, Protobuf's `struct` converts them to `float`, forcing users to write boilerplate code to work around this issue. |
There was a problem hiding this comment.
I think new line after a colon (unless a list) looks strange
|
@Future-Outlier thank you for your patience. A few more suggestions, plus please make sure there are no nested |
Co-authored-by: David Espejo <[email protected]> Signed-off-by: Han-Ru Chen (Future-Outlier) <[email protected]>
Co-authored-by: David Espejo <[email protected]> Signed-off-by: Han-Ru Chen (Future-Outlier) <[email protected]>
Co-authored-by: David Espejo <[email protected]> Signed-off-by: Han-Ru Chen (Future-Outlier) <[email protected]>
Co-authored-by: David Espejo <[email protected]> Signed-off-by: Han-Ru Chen (Future-Outlier) <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Docs link
Tracking issue
#5318