Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show traceback by default #2862

Merged
merged 7 commits into from
Oct 25, 2024
Merged

Show traceback by default #2862

merged 7 commits into from
Oct 25, 2024

Conversation

pingsutw
Copy link
Member

@pingsutw pingsutw commented Oct 24, 2024

Tracking issue

NA

Why are the changes needed?

The error message is not helpful in some cases when verbosity is 0.

Screenshot 2024-10-24 at 3 42 12 PM

What changes were proposed in this pull request?

print the traceback by default

How was this patch tested?

from dataclasses import dataclass

from flytekit import task, workflow, ImageSpec
from flytekit.configuration import Config
from flytekit.remote import FlyteRemote

remote = FlyteRemote(Config.auto(), default_project="flytesnacks", default_domain="development", interactive_mode_enabled=True)
new_flytekit = "git+https://github.com/flyteorg/flytekit.git@8017ae825b7c4cc3913971fab5308b172370d335"
img = ImageSpec(registry="pingsutw", packages=[new_flytekit], apt_packages=["git"])


@dataclass
class MyDatclass:
    id: int
    name: str
    age: int


@task(container_image=img)
def hello(dc: MyDatclass) -> str:
    return f"Hello {dc.name}"


@workflow
def wf(dc: MyDatclass = MyDatclass(id=1, name="foo", age=18)) -> str:
    return hello(name=dc.name)

Setup process

pyflyte run --remote flyte-example/improve_error/dataclass_mismatch.py wf

Screenshots

Screenshot 2024-10-24 at 4 01 40 PM

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

NA

Docs link

NA

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
@pingsutw pingsutw merged commit c5cc389 into master Oct 25, 2024
104 checks passed
Future-Outlier pushed a commit that referenced this pull request Oct 28, 2024
Future-Outlier added a commit that referenced this pull request Oct 28, 2024
* Pydantic Transformer V2

Signed-off-by: Future-Outlier <[email protected]>

* add __init__.py

Signed-off-by: Future-Outlier <[email protected]>

* add json schema

Signed-off-by: Future-Outlier <[email protected]>

* convert float to int

Signed-off-by: Future-Outlier <[email protected]>

* change gitsha in test script mode

Signed-off-by: Future-Outlier <[email protected]>

* change gitsha

Signed-off-by: Future-Outlier <[email protected]>

* use strict map=false

Signed-off-by: Future-Outlier <[email protected]>

* Test flytefile console input + attr access

Signed-off-by: Future-Outlier <[email protected]>

* add conditional branch

Signed-off-by: Future-Outlier <[email protected]>

* better rx

Signed-off-by: Future-Outlier <[email protected]>

* Add flytedir generic -> flytedir

Signed-off-by: Future-Outlier <[email protected]>

* support enum

Signed-off-by: Future-Outlier <[email protected]>

* update

Signed-off-by: Future-Outlier <[email protected]>

* add tests for input from flyte console

Signed-off-by: Future-Outlier <[email protected]>

* Add Tests for dataclass in BaseModel and pydantic.dataclass in BaseModel

Signed-off-by: Future-Outlier <[email protected]>

* update

Signed-off-by: Future-Outlier <[email protected]>

* update thomas's advice

Signed-off-by: Future-Outlier <[email protected]>

* change tree file structure

Signed-off-by: Future-Outlier <[email protected]>

* update Niel's advice

Signed-off-by: Future-Outlier <[email protected]>

* > to >=

Signed-off-by: Future-Outlier <[email protected]>

* try monodoc build again

Signed-off-by: Future-Outlier <[email protected]>

* add pydantic README.md

Signed-off-by: Future-Outlier <[email protected]>

* revert -vvv in monodocs

Signed-off-by: Future-Outlier <[email protected]>

* use model_validate_json to turn protobuf struct to python val

Signed-off-by: Future-Outlier <[email protected]>

* fix issue

Signed-off-by: Future-Outlier <[email protected]>

* handle flyte types in dict transformer from protobuf struct input (e.g FlyteConsole)

Signed-off-by: Future-Outlier <[email protected]>

* Add print

Signed-off-by: Future-Outlier <[email protected]>

* expected python type

Signed-off-by: Future-Outlier <[email protected]>

* switch call function order

Signed-off-by: Future-Outlier <[email protected]>

* try msgpack to handle protobug struct

Signed-off-by: Future-Outlier <[email protected]>

* Better Comment in Dict Transformer

Signed-off-by: Future-Outlier <[email protected]>

* Propeller -> FlytePropeller

Signed-off-by: Future-Outlier <[email protected]>

* dict_to_flyte_types

Signed-off-by: Future-Outlier <[email protected]>

* remove comments

Signed-off-by: Future-Outlier <[email protected]>

* add attr for protobuf struct . dict

Signed-off-by: Future-Outlier <[email protected]>

* Add Life Cycle for Flyte Types

Signed-off-by: Future-Outlier <[email protected]>

* better comments for derializing flyteschema and sd

Signed-off-by: Future-Outlier <[email protected]>

* nit

Signed-off-by: Future-Outlier <[email protected]>

* add back pv._remote_path = None to flytefile and flytedir

Signed-off-by: Future-Outlier <[email protected]>

* experiment

Signed-off-by: Future-Outlier <[email protected]>

* experiment

Signed-off-by: Future-Outlier <[email protected]>

* experiment

Signed-off-by: Future-Outlier <[email protected]>

* Add comments

Signed-off-by: Future-Outlier <[email protected]>

* update Yee's advice

Signed-off-by: Future-Outlier <[email protected]>

* code dc -> bm

Signed-off-by: Future-Outlier <[email protected]>

* Example dc -> bm and Example all flyte types

Signed-off-by: Future-Outlier <[email protected]>

* fix union dataclass, not yet add comments

Signed-off-by: Future-Outlier <[email protected]>

* add pydantic and dataclass optional test

Signed-off-by: Future-Outlier <[email protected]>

* NoneType=type(None)

Signed-off-by: Future-Outlier <[email protected]>

* fix union transformer none case with Eduardo

Signed-off-by: Future-Outlier <[email protected]>

* add comments for none type transformer + union transformer

Signed-off-by: Future-Outlier <[email protected]>

* add TODO

Signed-off-by: Future-Outlier <[email protected]>

* use deserailize = True

Signed-off-by: Future-Outlier <[email protected]>

* add all deserialize

Signed-off-by: Future-Outlier <[email protected]>

* better comments

Signed-off-by: Future-Outlier <[email protected]>

* better comments

Signed-off-by: Future-Outlier <[email protected]>

* test

Signed-off-by: Future-Outlier <[email protected]>

* Fix flyte directory issue by discussion with Kevin

Signed-off-by: Future-Outlier <[email protected]>
Co-authored-by: pingsutw  <[email protected]>

* add tests for providing conext when doing serialization

Signed-off-by: Future-Outlier <[email protected]>

* lint

Signed-off-by: Future-Outlier <[email protected]>

* test

Signed-off-by: Future-Outlier <[email protected]>

* move setattr to core

Signed-off-by: Future-Outlier <[email protected]>

* remove comments

Signed-off-by: Future-Outlier <[email protected]>

* lint

Signed-off-by: Future-Outlier <[email protected]>

* remove

Signed-off-by: Future-Outlier <[email protected]>

* testing

Signed-off-by: Future-Outlier <[email protected]>

* nit

Signed-off-by: Future-Outlier <[email protected]>

* flytekit/core/type_engine.py

Signed-off-by: Future-Outlier <[email protected]>

* add tests for Union

Signed-off-by: Future-Outlier <[email protected]>

* Trigger CI

Signed-off-by: Future-Outlier <[email protected]>

* remove nonetype

Signed-off-by: Future-Outlier <[email protected]>

* raw=Fasle as default

Signed-off-by: Future-Outlier <[email protected]>

* pydantic move to core test

Signed-off-by: Future-Outlier <[email protected]>

* move to core

Signed-off-by: Future-Outlier <[email protected]>

* log

Signed-off-by: Future-Outlier <[email protected]>

* update

Signed-off-by: Future-Outlier <[email protected]>

* lint

Signed-off-by: Future-Outlier <[email protected]>

* test

Signed-off-by: Future-Outlier <[email protected]>

* nit

Signed-off-by: Future-Outlier <[email protected]>

* nit

Signed-off-by: Future-Outlier <[email protected]>

* lint

Signed-off-by: Future-Outlier <[email protected]>

* move to type_engine

Signed-off-by: Future-Outlier <[email protected]>

* move back to init

Signed-off-by: Future-Outlier <[email protected]>

* update kevin's advice

Signed-off-by: Future-Outlier <[email protected]>
Co-authored-by: pingsutw  <[email protected]>

* wip

Signed-off-by: Future-Outlier <[email protected]>

* use decorator

Signed-off-by: Future-Outlier <[email protected]>

* decorator

Signed-off-by: Future-Outlier <[email protected]>

* fix syntax to support python 3.9

Signed-off-by: Future-Outlier <[email protected]>

* add Eduardo's advice

Signed-off-by: Future-Outlier <[email protected]>

* warning

Signed-off-by: Future-Outlier <[email protected]>

* Update Yee's advice

Signed-off-by: Future-Outlier <[email protected]>
Co-authored-by: Yee Hing Tong  <[email protected]>

* Show traceback by default (#2862)

Signed-off-by: Kevin Su <[email protected]>

* Support Identifier in generate_console_url (#2868)

Signed-off-by: Thomas J. Fan <[email protected]>

* Support overriding node metadata for array node (#2865)

* set parent node metadata on arraynode subnode

Signed-off-by: Paul Dittamo <[email protected]>

* clean up setting passed in metadata for array node

Signed-off-by: Paul Dittamo <[email protected]>

* add comment

Signed-off-by: Paul Dittamo <[email protected]>

* test

Signed-off-by: Paul Dittamo <[email protected]>

---------

Signed-off-by: Paul Dittamo <[email protected]>

* Fix Jupyter Versioning  (#2866)

* Update versioning

Signed-off-by: Mecoli1219 <[email protected]>

* Fix unit test

Signed-off-by: Mecoli1219 <[email protected]>

* Update path if jupyter file actually exist

Signed-off-by: Mecoli1219 <[email protected]>

* Fix pickled twice problem

Signed-off-by: Mecoli1219 <[email protected]>

---------

Signed-off-by: Mecoli1219 <[email protected]>

* improved output handling in notebooks (#2869)

---------

Signed-off-by: Future-Outlier <[email protected]>
Co-authored-by: pingsutw <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>
kumare3 pushed a commit that referenced this pull request Nov 8, 2024
kumare3 pushed a commit that referenced this pull request Nov 8, 2024
* Pydantic Transformer V2

Signed-off-by: Future-Outlier <[email protected]>

* add __init__.py

Signed-off-by: Future-Outlier <[email protected]>

* add json schema

Signed-off-by: Future-Outlier <[email protected]>

* convert float to int

Signed-off-by: Future-Outlier <[email protected]>

* change gitsha in test script mode

Signed-off-by: Future-Outlier <[email protected]>

* change gitsha

Signed-off-by: Future-Outlier <[email protected]>

* use strict map=false

Signed-off-by: Future-Outlier <[email protected]>

* Test flytefile console input + attr access

Signed-off-by: Future-Outlier <[email protected]>

* add conditional branch

Signed-off-by: Future-Outlier <[email protected]>

* better rx

Signed-off-by: Future-Outlier <[email protected]>

* Add flytedir generic -> flytedir

Signed-off-by: Future-Outlier <[email protected]>

* support enum

Signed-off-by: Future-Outlier <[email protected]>

* update

Signed-off-by: Future-Outlier <[email protected]>

* add tests for input from flyte console

Signed-off-by: Future-Outlier <[email protected]>

* Add Tests for dataclass in BaseModel and pydantic.dataclass in BaseModel

Signed-off-by: Future-Outlier <[email protected]>

* update

Signed-off-by: Future-Outlier <[email protected]>

* update thomas's advice

Signed-off-by: Future-Outlier <[email protected]>

* change tree file structure

Signed-off-by: Future-Outlier <[email protected]>

* update Niel's advice

Signed-off-by: Future-Outlier <[email protected]>

* > to >=

Signed-off-by: Future-Outlier <[email protected]>

* try monodoc build again

Signed-off-by: Future-Outlier <[email protected]>

* add pydantic README.md

Signed-off-by: Future-Outlier <[email protected]>

* revert -vvv in monodocs

Signed-off-by: Future-Outlier <[email protected]>

* use model_validate_json to turn protobuf struct to python val

Signed-off-by: Future-Outlier <[email protected]>

* fix issue

Signed-off-by: Future-Outlier <[email protected]>

* handle flyte types in dict transformer from protobuf struct input (e.g FlyteConsole)

Signed-off-by: Future-Outlier <[email protected]>

* Add print

Signed-off-by: Future-Outlier <[email protected]>

* expected python type

Signed-off-by: Future-Outlier <[email protected]>

* switch call function order

Signed-off-by: Future-Outlier <[email protected]>

* try msgpack to handle protobug struct

Signed-off-by: Future-Outlier <[email protected]>

* Better Comment in Dict Transformer

Signed-off-by: Future-Outlier <[email protected]>

* Propeller -> FlytePropeller

Signed-off-by: Future-Outlier <[email protected]>

* dict_to_flyte_types

Signed-off-by: Future-Outlier <[email protected]>

* remove comments

Signed-off-by: Future-Outlier <[email protected]>

* add attr for protobuf struct . dict

Signed-off-by: Future-Outlier <[email protected]>

* Add Life Cycle for Flyte Types

Signed-off-by: Future-Outlier <[email protected]>

* better comments for derializing flyteschema and sd

Signed-off-by: Future-Outlier <[email protected]>

* nit

Signed-off-by: Future-Outlier <[email protected]>

* add back pv._remote_path = None to flytefile and flytedir

Signed-off-by: Future-Outlier <[email protected]>

* experiment

Signed-off-by: Future-Outlier <[email protected]>

* experiment

Signed-off-by: Future-Outlier <[email protected]>

* experiment

Signed-off-by: Future-Outlier <[email protected]>

* Add comments

Signed-off-by: Future-Outlier <[email protected]>

* update Yee's advice

Signed-off-by: Future-Outlier <[email protected]>

* code dc -> bm

Signed-off-by: Future-Outlier <[email protected]>

* Example dc -> bm and Example all flyte types

Signed-off-by: Future-Outlier <[email protected]>

* fix union dataclass, not yet add comments

Signed-off-by: Future-Outlier <[email protected]>

* add pydantic and dataclass optional test

Signed-off-by: Future-Outlier <[email protected]>

* NoneType=type(None)

Signed-off-by: Future-Outlier <[email protected]>

* fix union transformer none case with Eduardo

Signed-off-by: Future-Outlier <[email protected]>

* add comments for none type transformer + union transformer

Signed-off-by: Future-Outlier <[email protected]>

* add TODO

Signed-off-by: Future-Outlier <[email protected]>

* use deserailize = True

Signed-off-by: Future-Outlier <[email protected]>

* add all deserialize

Signed-off-by: Future-Outlier <[email protected]>

* better comments

Signed-off-by: Future-Outlier <[email protected]>

* better comments

Signed-off-by: Future-Outlier <[email protected]>

* test

Signed-off-by: Future-Outlier <[email protected]>

* Fix flyte directory issue by discussion with Kevin

Signed-off-by: Future-Outlier <[email protected]>
Co-authored-by: pingsutw  <[email protected]>

* add tests for providing conext when doing serialization

Signed-off-by: Future-Outlier <[email protected]>

* lint

Signed-off-by: Future-Outlier <[email protected]>

* test

Signed-off-by: Future-Outlier <[email protected]>

* move setattr to core

Signed-off-by: Future-Outlier <[email protected]>

* remove comments

Signed-off-by: Future-Outlier <[email protected]>

* lint

Signed-off-by: Future-Outlier <[email protected]>

* remove

Signed-off-by: Future-Outlier <[email protected]>

* testing

Signed-off-by: Future-Outlier <[email protected]>

* nit

Signed-off-by: Future-Outlier <[email protected]>

* flytekit/core/type_engine.py

Signed-off-by: Future-Outlier <[email protected]>

* add tests for Union

Signed-off-by: Future-Outlier <[email protected]>

* Trigger CI

Signed-off-by: Future-Outlier <[email protected]>

* remove nonetype

Signed-off-by: Future-Outlier <[email protected]>

* raw=Fasle as default

Signed-off-by: Future-Outlier <[email protected]>

* pydantic move to core test

Signed-off-by: Future-Outlier <[email protected]>

* move to core

Signed-off-by: Future-Outlier <[email protected]>

* log

Signed-off-by: Future-Outlier <[email protected]>

* update

Signed-off-by: Future-Outlier <[email protected]>

* lint

Signed-off-by: Future-Outlier <[email protected]>

* test

Signed-off-by: Future-Outlier <[email protected]>

* nit

Signed-off-by: Future-Outlier <[email protected]>

* nit

Signed-off-by: Future-Outlier <[email protected]>

* lint

Signed-off-by: Future-Outlier <[email protected]>

* move to type_engine

Signed-off-by: Future-Outlier <[email protected]>

* move back to init

Signed-off-by: Future-Outlier <[email protected]>

* update kevin's advice

Signed-off-by: Future-Outlier <[email protected]>
Co-authored-by: pingsutw  <[email protected]>

* wip

Signed-off-by: Future-Outlier <[email protected]>

* use decorator

Signed-off-by: Future-Outlier <[email protected]>

* decorator

Signed-off-by: Future-Outlier <[email protected]>

* fix syntax to support python 3.9

Signed-off-by: Future-Outlier <[email protected]>

* add Eduardo's advice

Signed-off-by: Future-Outlier <[email protected]>

* warning

Signed-off-by: Future-Outlier <[email protected]>

* Update Yee's advice

Signed-off-by: Future-Outlier <[email protected]>
Co-authored-by: Yee Hing Tong  <[email protected]>

* Show traceback by default (#2862)

Signed-off-by: Kevin Su <[email protected]>

* Support Identifier in generate_console_url (#2868)

Signed-off-by: Thomas J. Fan <[email protected]>

* Support overriding node metadata for array node (#2865)

* set parent node metadata on arraynode subnode

Signed-off-by: Paul Dittamo <[email protected]>

* clean up setting passed in metadata for array node

Signed-off-by: Paul Dittamo <[email protected]>

* add comment

Signed-off-by: Paul Dittamo <[email protected]>

* test

Signed-off-by: Paul Dittamo <[email protected]>

---------

Signed-off-by: Paul Dittamo <[email protected]>

* Fix Jupyter Versioning  (#2866)

* Update versioning

Signed-off-by: Mecoli1219 <[email protected]>

* Fix unit test

Signed-off-by: Mecoli1219 <[email protected]>

* Update path if jupyter file actually exist

Signed-off-by: Mecoli1219 <[email protected]>

* Fix pickled twice problem

Signed-off-by: Mecoli1219 <[email protected]>

---------

Signed-off-by: Mecoli1219 <[email protected]>

* improved output handling in notebooks (#2869)

---------

Signed-off-by: Future-Outlier <[email protected]>
Co-authored-by: pingsutw <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants