Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "Revert "MetadataValue subclasses to DagsterModel (#21324)" (#… #21543

Merged
merged 1 commit into from
May 1, 2024

Conversation

sryza
Copy link
Contributor

@sryza sryza commented Apr 30, 2024

…21462)"

This reverts commit 7b103db.

Summary & Motivation

This moves MetadataValue subclasses to also be subclasses of DagsterModel, instead of NamedTuples.

This was originally reverted because it caused a test failure. This PR fixed the test failure: #21463. This PR further reduced the risk: #21531.

Original PR: #21324

How I Tested These Changes

@sryza sryza requested review from alangenfeld and benpankow April 30, 2024 21:59
Copy link
Member

@alangenfeld alangenfeld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outside the __init__s that coerce inputs like [1] most of them like [2] seem like they could be omitted. Am I missing something?

I think pretty much every object is either a [1] (coerces input but property type does not reflect) or [2] (maybe not needed __init__)

return super(TextMetadataValue, cls).__new__(
cls, check.opt_str_param(text, "text", default="")
)
text_inner: Optional[str] = Field(..., alias="text")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[1] this object property (and @propertys) could be str since we coerce None to "" in the __init__ that takes Optional[str]

Comment on lines +627 to +628
def __init__(self, text: Optional[str]):
super().__init__(text=text or "")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ or we could just drop this custom __init__ and let the None flow through since thats how everything is typed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this is weird - this was just me trying to minimize changes to the existing behavior.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it - consider dropping the unnecessary Optional in a follow up PR then I guess. Hard to argue that its that impactful of an improvement

Comment on lines +678 to +679
def __init__(self, path: Optional[Union[str, os.PathLike]]):
super().__init__(path=check.opt_path_param(path, "path", default=""))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[1]

Comment on lines +777 to +778
def __init__(self, module: str, name: str):
super().__init__(module=module, name=name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[2]

@sryza
Copy link
Contributor Author

sryza commented May 1, 2024

Outside the __init__s that coerce inputs like [1] most of them like [2] seem like they could be omitted. Am I missing something?

They're needed to support positional arguments (for backcompat). The Pydantic-generated constructors only support keyword arguments.

Copy link
Member

@alangenfeld alangenfeld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

support positional arguments (for backcompat)

Ah yes. Not something easy to intuit on first glance, but not sure its worth dropping # support positional args for backcompat comments on all of these

Comment on lines +627 to +628
def __init__(self, text: Optional[str]):
super().__init__(text=text or "")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it - consider dropping the unnecessary Optional in a follow up PR then I guess. Hard to argue that its that impactful of an improvement

@sryza
Copy link
Contributor Author

sryza commented May 1, 2024

Some more discussion on the positional stuff here btw: #20461

@sryza sryza merged commit b62864c into master May 1, 2024
1 check passed
@sryza sryza deleted the metadata-value-dagster-model branch May 1, 2024 18:37
alangenfeld added a commit that referenced this pull request May 8, 2024
fix-up side effects from the bug correction in
#21543 to prevent non vanilla
json objects from being part of JsonMetadatValue by adding special
processing for the `data` field

## How I Tested These Changes

added test
sbquinlan pushed a commit that referenced this pull request May 8, 2024
fix-up side effects from the bug correction in
#21543 to prevent non vanilla
json objects from being part of JsonMetadatValue by adding special
processing for the `data` field

## How I Tested These Changes

added test
danielgafni pushed a commit to danielgafni/dagster that referenced this pull request Jun 18, 2024
fix-up side effects from the bug correction in
dagster-io#21543 to prevent non vanilla
json objects from being part of JsonMetadatValue by adding special
processing for the `data` field

## How I Tested These Changes

added test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants