Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DVC does fails when the md5 hash contains only numbers #3331

Closed
charlesbaynham opened this issue Feb 14, 2020 · 5 comments · Fixed by #3333
Closed

DVC does fails when the md5 hash contains only numbers #3331

charlesbaynham opened this issue Feb 14, 2020 · 5 comments · Fixed by #3333
Assignees
@efiop
Labels
p0-critical Critical issue. Needs to be fixed ASAP.

Comments

@charlesbaynham
Copy link
Contributor

The description is in the title. When an md5 hash randomly contains only numbers DVC fails, reporting

ERROR: failed to update 'step_X.dvc'. - DVC-file 'step_X.dvc' format error: expected str for dictionary value @ data['md5']

This should happen on average once for every 3 million hashes. For a big project with a lot of files that get changed, that's not so implausible.
@triage-new-issues triage-new-issues bot added the triage Needs to be triaged label Feb 14, 2020
@efiop
Copy link
Contributor

efiop commented Feb 14, 2020

Hi @charlesbaynham ! Could you please show dvc version output?

@efiop
Copy link
Contributor

efiop commented Feb 14, 2020

For the record: can reproduce this myself as well, by modifying md5 in a dvc file by-hand. We've had a similar issue with pyyaml earlier, need to check this.

@efiop efiop added the p0-critical Critical issue. Needs to be fixed ASAP. label Feb 14, 2020
@triage-new-issues triage-new-issues bot removed the triage Needs to be triaged label Feb 14, 2020
@efiop efiop self-assigned this Feb 14, 2020
@efiop
Copy link
Contributor

efiop commented Feb 14, 2020

Ok, so both pyyaml and ruamel are affected, so we need to tweak our SCHEMA instead. Working on a fix...

@efiop efiop mentioned this issue Feb 14, 2020
Merged
3 tasks
efiop added a commit to efiop/dvc that referenced this issue Feb 14, 2020
@efiop
schema: use a more flexible hash schema
aad20c5 
Fixes iterative#3331
efiop added a commit that referenced this issue Feb 14, 2020
@efiop
schema: use a more flexible hash schema (#3333)
8a2699d 
Fixes #3331
@efiop
Copy link
Contributor

efiop commented Feb 14, 2020

@charlesbaynham We are releasing 0.85.0 right now. Please upgrade, give it a try and let us know if that fixed the issue for you or not. Thanks for the feedback! 🙂

@charlesbaynham
Copy link
Contributor Author

Wow I think that's the fastest response I've ever seen on github! Thanks a lot, I'll give it a try

@weekly-digest weekly-digest bot mentioned this issue Feb 16, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@efiop efiop

Labels
p0-critical Critical issue. Needs to be fixed ASAP.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

schema: use a more flexible hash schema efiop/dvc
2 participants
@charlesbaynham @efiop