Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Handle Optional FlyteFile in dataclasses #3187

Closed
2 tasks done
eapolinario opened this issue Dec 23, 2022 · 0 comments
Closed
2 tasks done

[BUG] Handle Optional FlyteFile in dataclasses #3187

eapolinario opened this issue Dec 23, 2022 · 0 comments
Assignees
Labels
bug Something isn't working flytekit FlyteKit Python related issue
Milestone

Comments

@eapolinario
Copy link
Contributor

eapolinario commented Dec 23, 2022

Describe the bug

The dataclass type transformer does not handle fields of type Optional[FlyteFile] correctly.

For example, this workflow fails:

from dataclasses import dataclass
from typing import Optional

from dataclasses_json import dataclass_json
from flytekit import workflow, task
from flytekit.types.file import FlyteFile


@dataclass_json
@dataclass
class TaskOutput:
    file1: FlyteFile
    file2: Optional[FlyteFile]

@task
def task1() -> TaskOutput:
    with open("file21.txt", "w+") as f:
        f.write("b")
    with open("file22.txt", "w+") as f:
        f.write("c")

    return TaskOutput(
        FlyteFile("file21.txt", remote_path="s3://my-s3-bucket/tmp/file21.txt"),
        FlyteFile("file22.txt", remote_path="s3://my-s3-bucket/tmp/file22.txt")
    )

@task
def assert_values_in_dataclass(to: TaskOutput):
    assert to.file1.remote_source == "s3://my-s3-bucket/tmp/file21.txt"
    assert to.file2.remote_source == "s3://my-s3-bucket/tmp/file22.txt"


@workflow
def wf():
    to = task1()
    assert_values_in_dataclass(to=to)

Expected behavior

Optional[FlyteFile] should be allowed in Dataclasses.

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@eapolinario eapolinario added bug Something isn't working untriaged This issues has not yet been looked at by the Maintainers flytekit FlyteKit Python related issue labels Dec 23, 2022
@eapolinario eapolinario added this to the 1.3.0 milestone Dec 23, 2022
@eapolinario eapolinario self-assigned this Dec 23, 2022
@eapolinario eapolinario removed the untriaged This issues has not yet been looked at by the Maintainers label Dec 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working flytekit FlyteKit Python related issue
Projects
None yet
Development

No branches or pull requests

2 participants