Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problems with dvc push #4234

Closed
LarryThermo opened this issue Jul 17, 2020 · 11 comments
Closed

Problems with dvc push #4234

LarryThermo opened this issue Jul 17, 2020 · 11 comments
Labels
awaiting response we are waiting for your reply, please respond! :)

Comments

@LarryThermo
Copy link

LarryThermo commented Jul 17, 2020 •
edited
Loading

Bug Report

Please provide information about your setup

Output of dvc version:

> dvc version
DVC version: 1.1.10
Python version: 3.7.5
Platform: Windows-10-10.0.17763-SP0
Binary: True
Package: exe
Supported remotes: azure, gdrive, gs, hdfs, http, https, s3, ssh, oss
Cache: reflink - not supported, hardlink - supported, symlink - not supported
Filesystem type (cache directory): ('NTFS', 'C:\\')
Repo: dvc, git
Filesystem type (workspace): ('NTFS', 'C:\\')

Additional Information (if any):
I am having problems with dvc push no longer working after I upgraded to the latest dvc.

My steps in a Windows Powershell window (if it appears a little bit disjointed its because I copied and pasted the output verbatim):

dvc init

You can now commit the changes to git.

+---------------------------------------------------------------------+
| |
| DVC has enabled anonymous aggregate usage analytics. |
| Read the analytics documentation (and how to opt-out) here: |
| https://dvc.org/doc/user-guide/analytics |
| |
+---------------------------------------------------------------------+

What's next?

dvc remote add -d myremote s3://analysis-training-sets/masu-autofocus-models2
Setting 'myremote' as a default remote.
dvc add "model-all-00038-0.70911.pt"
100% Add|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|1/1 [00:02, 2.99s/file]

To track the changes with git, run:

    git add model-all-00038-0.70911.ptj.dvc

dvc push
ERROR: unexpected error - unable to open database file

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
@triage-new-issues triage-new-issues bot added the triage Needs to be triaged label Jul 17, 2020
@efiop
Copy link
Contributor

efiop commented Jul 18, 2020

Hi @LarryThermo . Could you please show full dvc push -v log?

@efiop efiop added the awaiting response we are waiting for your reply, please respond! :) label Jul 18, 2020
@triage-new-issues triage-new-issues bot removed the triage Needs to be triaged label Jul 18, 2020
@weekly-digest weekly-digest bot mentioned this issue Jul 19, 2020
Closed
@LarryThermo
Copy link
Author

LarryThermo commented Jul 20, 2020 via email •
edited by efiop
Loading

@efiop
Copy link
Contributor

efiop commented Jul 20, 2020

@LarryThermo Thanks. Looks very strange, could you please show the contents of model-all-00038-0.70911.ptj.dvc ?

@LarryThermo
Copy link
Author

LarryThermo commented Jul 20, 2020 via email

@efiop
Copy link
Contributor

efiop commented Jul 20, 2020

@LarryThermo Thanks!

@pmrowla Looks like something is up with the sqlite index db (though the very first log is different and confusing), maybe you have some ideas about it?

@pmrowla
Copy link
Contributor

pmrowla commented Jul 21, 2020

@LarryThermo can you try deleting the .dvc/tmp/index folder from your repo, and then re-run dvc push -v?

@LarryThermo
Copy link
Author

LarryThermo commented Jul 21, 2020 via email

@LarryThermo
Copy link
Author

LarryThermo commented Jul 21, 2020 via email

@pmrowla
Copy link
Contributor

pmrowla commented Jul 23, 2020

I don't think there's much we can do about the path length limitation on Windows, but we could at least check for this and give a better error message if we fail to open the index DB.

@LarryThermo
Copy link
Author

LarryThermo commented Jul 23, 2020 via email

@efiop
Copy link
Contributor

efiop commented Jul 23, 2020

Hm, btw, we use an absolute path when opening the db, so we probably could've avoided this issue if we were to use a relative path instead. This is something that we are constantly talking about in many places in dvc, and is prone to hitting MAX_PATH on other platforms as well. So the best solution will be if we make that switch, but that won't happen in the nearest future :(

As to the error(or warning), a meaningful one will require some sysconf magic (MAX_PATH might not be 260, it might be more or might even be ignored by the system) and will bring us questionable results. Shame sqlite doesn't raise a meaningful exception for this 🙁 So considering this, I think we should just close this issue for now, since there is an easy workaround. If we get more reports - we'll reopen and will probably try to use relpaths at least when dealing with sqlite, but that will likely make the issue arise in a different place 🙁

If anyone else is running into this issue, please leave a comment so we are aware. Thank you.

@efiop efiop closed this as completed Jul 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
awaiting response we are waiting for your reply, please respond! :)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pmrowla @efiop @LarryThermo