Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added test_set_empty_params #6484

Closed
wants to merge 11 commits into from
Closed

Added test_set_empty_params #6484

wants to merge 11 commits into from

Conversation

daavoo
Copy link
Contributor

@daavoo daavoo commented Aug 24, 2021

@daavoo daavoo requested a review from a team as a code owner August 24, 2021 10:44
@daavoo daavoo requested a review from efiop August 24, 2021 10:44
@daavoo daavoo requested a review from skshetry August 24, 2021 10:45
@daavoo daavoo self-assigned this Aug 24, 2021
@@ -50,6 +51,8 @@ def _get_yaml():
# tell Dumper to represent OrderedDict as normal dict
yaml_repr_cls = yaml.Representer
yaml_repr_cls.add_representer(OrderedDict, yaml_repr_cls.represent_dict)
yaml_repr_cls.add_representer(benedict, yaml_repr_cls.represent_dict)
Copy link
Member

@skshetry skshetry Aug 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

benedict should never escape when we merge params from --set-params. This may result in loss of comments.
A fix might be to merge the params in the file recursively from the --set-params benedict dictionary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I'm fully understanding, what do you mean by escape ?

Copy link
Member

@skshetry skshetry Aug 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of #6423, I'd like not to use or import benedict elsewhere than the following location, until it is resolved:

def merge_params(src: Dict, to_update: Dict) -> Dict:

So, the merge_params should not return any benedict object, it should return the data of same type as src, which is usually dict/CommentedMap (which preserves comments, coming from ruamel.yaml).

EDIT: updates on src should happen in-place, as required by modify_data contextmanager.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed that and replaced with just a regression test currently checking that a exception is being raised. I guess that if we go with #6423 (comment) we should make that test pass

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#6476 is more urgent than #6423, though we could do it together too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure but maybe we could add this tests for now and discuss in #6521 how we want to handle #5477 (as it would make #6476 no longer a bug but a different exception)

@daavoo daavoo changed the title Added benedict representer to yaml Added test_set_empty_params Aug 30, 2021
dvc/utils/serialize/_yaml.py Outdated Show resolved Hide resolved
@skshetry
Copy link
Member

Sorry for the late review. I have incorporated this test in #6706 (comment), as it gets automatically fixed after updating benedict to the latest version. This bug happened because we were handling an empty dictionary in a different way because of a bug in benedict which we don't need to handle now at all.

@daavoo daavoo closed this Sep 29, 2021
@skshetry skshetry deleted the 6476 branch September 29, 2021 07:51
skshetry added a commit that referenced this pull request Oct 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants