Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected error - Adding files #2831

Closed
luchoPipe87 opened this issue Nov 21, 2019 · 69 comments · Fixed by #2918
Closed

Unexpected error - Adding files #2831

luchoPipe87 opened this issue Nov 21, 2019 · 69 comments · Fixed by #2918
Assignees
Labels
bug Did we break something? p0-critical Critical issue. Needs to be fixed ASAP.

Comments

@luchoPipe87
Copy link

Everytime that im trying to add some individuals files or complete directories the same unexpected error appears:

> dvc add -v -R model
DEBUG: Trying to spawn '['c:\\users\\luisfelipe_melo_mora\\appdata\\local\\programs\\python\\python37-32\\python.exe', 'C:\\Users\\luisfelipe_melo_mora\\AppData\\Local\\Programs\\Python\\Python37-32\\Scripts\\dvc', 'daemon', '-q', 'updater']'
DEBUG: Spawned '['c:\\users\\luisfelipe_melo_mora\\appdata\\local\\programs\\python\\python37-32\\python.exe', 'C:\\Users\\luisfelipe_melo_mora\\AppData\\Local\\Programs\\Python\\Python37-32\\Scripts\\dvc', 'daemon', '-q',
'updater']'
ERROR: unexpected error - Already unlocked
------------------------------------------------------------
Traceback (most recent call last):
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\main.py", line 48, in main
    cmd = args.func(args)
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\command\base.py", line 48, in __init__
    updater.check()
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\updater.py", line 54, in check
    self._with_lock(self._check, "checking")
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\updater.py", line 45, in _with_lock
    func()
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 338, in __exit__
    self.unlock()
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 287, in unlock
    raise NotLockedError('Already unlocked')
flufl.lock._lockfile.NotLockedError: Already unlocked
------------------------------------------------------------


Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

I have a remote configuration by SSH:

['remote "myssh"']
url = ssh://domain:/path
user = myuser
port = 22
ask_password = true
[core]
remote = myssh

And here the version of dvc that im using:

> dvc version
DVC version: 0.69.0
Python version: 3.7.4
Platform: Windows-10-10.0.17134-SP0
Binary: False
Package: pip
Cache: reflink - False, hardlink - True, symlink - False

Thanks for your help!

@triage-new-issues triage-new-issues bot added the triage Needs to be triaged label Nov 21, 2019
@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

Hi @luchoPipe87 !

That looks pretty weird. Btw, why did you remove fs info from dvc version output? Are you working inside some mounted partition?

@luchoPipe87
Copy link
Author

Hi @efiop!
I didn't remove any information from the dvc version
I dont even know what fs (file system?) means. I'm not working in any particular partition.

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 You are not working on a NAS partition or anything like that? Just regular hard drive(ssd) on your machine? Btw, did you run dvc version inside of the dvc repo?

@casperdcl
Copy link
Contributor

I get the same error when doing:

dvc init
dvc destroy -f

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@casperdcl I am able to reproduce from that too! Thank you! Looks like it is our daemon getting confused by a deleted lock. Not quite sure what is up with dvc add in the original issue 🤔

@luchoPipe87
Copy link
Author

I'm doing dvc init -f -v to try to restart the config but nothing seems to help :(

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 Is there any other dvc process running in the background?

@efiop efiop added the bug Did we break something? label Nov 21, 2019
@triage-new-issues triage-new-issues bot removed the triage Needs to be triaged label Nov 21, 2019
@efiop efiop added the p0-critical Critical issue. Needs to be fixed ASAP. label Nov 21, 2019
@luchoPipe87
Copy link
Author

I only have one project. So i would say no.
i was looking into the task manager and it doesnt seem any dvc process...
image

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 Got it. And does running simple dvc status show the same error?

@luchoPipe87
Copy link
Author

luchoPipe87 commented Nov 21, 2019

If i do dvc status after the "add" error i got that:

> dvc status
ERROR: failed to obtain data status - Already unlocked


Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

And i received this one after a dvc init -f -v

> dvc status
ERROR: unexpected error - Already unlocked


Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 And could you try manually deleting .dvc/updater.lock and running dvc status once again, please?

@luchoPipe87
Copy link
Author

every time that i deleted .dvc/updater.lock and i ran dvc status i receive the same error:

 dvc status
ERROR: unexpected error - Already unlocked


Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 Interesting. And what flufl.lock version do you have installed? Could you show pip freeze | grep flufl.lock?

@luchoPipe87
Copy link
Author

Sure! I cant do grep btw, im in Windows machine ...
image

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 Right, sorry, I thought that you are using gitbash 🙂 Not sure what is the alternative command for cmd/powershell.

Ok, so flufl lock version looks alright. Ok, let's try to narrow this down. Could you please try running this script in your repo root:

from dvc.lock import Lock

with Lock("mylock"):
    pass

@luchoPipe87
Copy link
Author

No worries! :)
Here the output after running the script:

> python test.py
Traceback (most recent call last):
  File "test.py", line 4, in <module>
    pass
  File "C:\Users\luisfelipe_melo_mora\AppData\Local\Programs\Python\Python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 338, in __exit__
    self.unlock()
  File "C:\Users\luisfelipe_melo_mora\AppData\Local\Programs\Python\Python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 287, in unlock
    raise NotLockedError('Already unlocked')
flufl.lock._lockfile.NotLockedError: Already unlocked

@casperdcl
Copy link
Contributor

findstr instead of grep on windows

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 Thanks! Ok, looks like it might be flufl.lock issue. Could you try this too, please:

from dvc.lock import Lock

with Lock("mylock") as lock:
    print("is locked: " + str(lock.is_locked))

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 Ok, this might be the issue https://gitlab.com/warsaw/flufl.lock/blob/master/flufl/lock/_lockfile.py#L490 , because stat on windows is pretty useless IIRC.

Could you try running:

import os
from dvc.lock import Lock

with Lock("mylock") as lock:
    print("is locked: " + str(lock.is_locked))
    print("stat " + str(os.stat(lock._lockfile)))

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

As one could see https://github.com/iterative/dvc/blob/master/dvc/system.py#L280 we actually don't use os.stat to count the number of hardlinks. flufl.lock should do the same, but the project is pretty much dead. 'Guess we should consider forking it into something like flufl.lock2 and fix up these kind of things. But we should definitely start from simply patching it up in our Lock class.

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 I'll prepare a patch in a minute and will ask you to try it out, if you don't mind 🙂

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 Ok, please install my version of dvc with

pip install git+https://github.com/efiop/dvc.git@2831

and try running those commands again.

@luchoPipe87
Copy link
Author

Here the output of the last command :)

> python test.py
Traceback (most recent call last):
  File "C:\Users\luisfelipe_melo_mora\AppData\Local\Programs\Python\Python37-32\lib\site-packages\dvc\lock.py", line 78, in lock
    super(Lock, self).lock(timedelta(seconds=DEFAULT_TIMEOUT))
  File "C:\Users\luisfelipe_melo_mora\AppData\Local\Programs\Python\Python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 259, in lock
    raise TimeOutError('Could not acquire the lock')
flufl.lock._lockfile.TimeOutError: Could not acquire the lock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 4, in <module>
    with Lock("mylock") as lock:
  File "C:\Users\luisfelipe_melo_mora\AppData\Local\Programs\Python\Python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 334, in __enter__
    self.lock()
  File "C:\Users\luisfelipe_melo_mora\AppData\Local\Programs\Python\Python37-32\lib\site-packages\dvc\lock.py", line 80, in lock
    raise LockError(FAILED_TO_LOCK_MESSAGE)
dvc.lock.LockError: cannot perform the command because another DVC process seems to be running on this project. If that is not the case, manually remove `.dvc/lock` and try again.

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 You mean that test script with the patched dvc? Could you run dvc status with that patched dvc, please?

@luchoPipe87
Copy link
Author

I installed your dvc version and its getting better (Is another error :P )

> dvc init -f -v
DEBUG: Removing '.dvc'
DEBUG: Adding '.dvc\config.local' to '.dvc\.gitignore'.
DEBUG: Adding '.dvc\updater' to '.dvc\.gitignore'.
DEBUG: Adding '.dvc\state-journal' to '.dvc\.gitignore'.
DEBUG: Adding '.dvc\state-wal' to '.dvc\.gitignore'.
DEBUG: Adding '.dvc\state' to '.dvc\.gitignore'.
DEBUG: Adding '.dvc\lock' to '.dvc\.gitignore'.
DEBUG: Adding '.dvc\tmp' to '.dvc\.gitignore'.
DEBUG: Adding '.dvc\updater.lock' to '.dvc\.gitignore'.
DEBUG: Adding '.dvc\cache' to '.dvc\.gitignore'.
Exception ignored in: <function Lock.__del__ at 0x03E86738>
Traceback (most recent call last):
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 99, in __del__
    self.finalize()
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 326, in finalize
    self.unlock(unconditionally=True)
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 285, in unlock
    is_locked = self.is_locked
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 319, in is_locked
    if self._linkcount != 2:
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 111, in
_linkcount
    return System._getdirinfo(self._lockfile).nNumberOfLinks
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\system.py", line 171, in _getdirinfo
    path, 0, FILE_SHARE_READ, None, OPEN_EXISTING, flags, None
pywintypes.error: (2, 'CreateFileW', 'The system cannot find the file specified.')

You can now commit the changes to git.

DEBUG: Analytics is enabled.
+---------------------------------------------------------------------+
|                                                                     |
|        DVC has enabled anonymous aggregate usage analytics.         |
|     Read the analytics documentation (and how to opt-out) here:     |
|              https://dvc.org/doc/user-guide/analytics               |
|                                                                     |
+---------------------------------------------------------------------+

What's next?
------------
- Check out the documentation: https://dvc.org/doc
- Get help and share ideas: https://dvc.org/chat
- Star us on GitHub: https://github.com/iterative/dvc
Exception ignored in: <function Lock.__del__ at 0x03E86738>
Traceback (most recent call last):
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 99, in __del__
    self.finalize()
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 326, in finalize
    self.unlock(unconditionally=True)
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 285, in unlock
    is_locked = self.is_locked
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 319, in is_locked
    if self._linkcount != 2:
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 111, in
_linkcount
    return System._getdirinfo(self._lockfile).nNumberOfLinks
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\system.py", line 171, in _getdirinfo
    path, 0, FILE_SHARE_READ, None, OPEN_EXISTING, flags, None
pywintypes.error: (2, 'CreateFileW', 'The system cannot find the file specified.')
Exception ignored in: <function Lock.__del__ at 0x03E86738>
Traceback (most recent call last):
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 99, in __del__
    self.finalize()
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 326, in finalize
    self.unlock(unconditionally=True)
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 285, in unlock
    is_locked = self.is_locked
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 319, in is_locked
    if self._linkcount != 2:
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 111, in
_linkcount
    return System._getdirinfo(self._lockfile).nNumberOfLinks
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\system.py", line 171, in _getdirinfo
    path, 0, FILE_SHARE_READ, None, OPEN_EXISTING, flags, None
pywintypes.error: (2, 'CreateFileW', 'The system cannot find the file specified.')
Exception ignored in: <function Lock.__del__ at 0x03E86738>
Traceback (most recent call last):
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 99, in __del__
    self.finalize()
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 326, in finalize
    self.unlock(unconditionally=True)
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 285, in unlock
    is_locked = self.is_locked
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 319, in is_locked
    if self._linkcount != 2:
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 111, in
_linkcount
    return System._getdirinfo(self._lockfile).nNumberOfLinks
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\system.py", line 171, in _getdirinfo
    path, 0, FILE_SHARE_READ, None, OPEN_EXISTING, flags, None
pywintypes.error: (2, 'CreateFileW', 'The system cannot find the file specified.')
Exception ignored in: <function Lock.__del__ at 0x03E86738>
Traceback (most recent call last):
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 99, in __del__
    self.finalize()
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 326, in finalize
    self.unlock(unconditionally=True)
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 285, in unlock
    is_locked = self.is_locked
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 319, in is_locked
    if self._linkcount != 2:
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 111, in
_linkcount
    return System._getdirinfo(self._lockfile).nNumberOfLinks
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\system.py", line 171, in _getdirinfo
    path, 0, FILE_SHARE_READ, None, OPEN_EXISTING, flags, None
pywintypes.error: (2, 'CreateFileW', 'The system cannot find the file specified.')
Exception ignored in: <function Lock.__del__ at 0x03E86738>
Traceback (most recent call last):
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 99, in __del__
    self.finalize()
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 326, in finalize
    self.unlock(unconditionally=True)
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 285, in unlock
    is_locked = self.is_locked
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\flufl\lock\_lockfile.py", line 319, in is_locked
    if self._linkcount != 2:
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\lock.py", line 111, in
_linkcount
    return System._getdirinfo(self._lockfile).nNumberOfLinks
  File "c:\users\luisfelipe_melo_mora\appdata\local\programs\python\python37-32\lib\site-packages\dvc\system.py", line 171, in _getdirinfo
    path, 0, FILE_SHARE_READ, None, OPEN_EXISTING, flags, None
pywintypes.error: (2, 'CreateFileW', 'The system cannot find the file specified.')

@efiop
Copy link
Contributor

efiop commented Nov 21, 2019

@luchoPipe87 I've updated my patch, please reinstall with

pip install --force-reinstall git+https://github.com/efiop/dvc.git@2831

and try dvc status again.

@efiop
Copy link
Contributor

efiop commented Nov 27, 2019

In my case:

PS C:\WINDOWS\system32> fsutil fsinfo ntfsinfo C:
NTFS Volume Serial Number :        0xa0a82441a82417f4
NTFS Version      :                3.1
LFS Version       :                2.0
Total Sectors     :                304,953,408  (145.4 GB)
Total Clusters    :                 38,119,176  (145.4 GB)
Free Clusters     :                  3,058,114  ( 11.7 GB)
Total Reserved Clusters :                5,425  ( 21.2 MB)
Reserved For Storage Reserve :               0  (  0.0 KB)
Bytes Per Sector  :                512
Bytes Per Physical Sector :        512
Bytes Per Cluster :                4096
Bytes Per FileRecord Segment    :  1024
Clusters Per FileRecord Segment :  0
Mft Valid Data Length :            809.75 MB
Mft Start Lcn  :                   0x00000000000c0000
Mft2 Start Lcn :                   0x0000000000000002
Mft Zone Start :                   0x0000000001df47c0
Mft Zone End   :                   0x0000000001e00fe0
MFT Zone Size  :                   200.13 MB
Max Device Trim Extent Count :     64
Max Device Trim Byte Count :       0x7fff8000
Max Volume Trim Extent Count :     62
Max Volume Trim Byte Count :       0x40000000
Resource Manager Identifier :      EBDB7152-118A-11E8-B3E9-EF47FED40743

@efiop
Copy link
Contributor

efiop commented Nov 27, 2019

@MrOutis It was windows 10 :)

@casperdcl
Copy link
Contributor

casperdcl commented Nov 27, 2019

Which is just a skin over XP, right?

Win 7 and XP both have 40M lines of code (Vista in between had 50). I doubt things are much different now :)

@ghost
Copy link

ghost commented Nov 27, 2019

@efiop, but there are different versions of Windows 10, right? I haven't read the release log, but might be a driver thingy? https://docs.microsoft.com/en-us/windows/release-information/

@efiop
Copy link
Contributor

efiop commented Nov 27, 2019

@MrOutis It was provided in the dvc version output: @luchoPipe87 has Windows-10-10.0.17134-SP0 and I have Windows-10-10.0.18362-SP0.

@efiop
Copy link
Contributor

efiop commented Nov 27, 2019

@luchoPipe87 Hm, btw, do you have a RAID or something setup on your machine? Maybe some special encryption stuff? I just wonder if the link we are creating get's altered by something right away, maybe some weird encryption stuff or backups... Considering that it is work machine, i suppose it has both raid/backups and some special encryption. Or maybe antivirus O_o?

@efiop
Copy link
Contributor

efiop commented Dec 5, 2019

@luchoPipe87 ping 🙂

@luchoPipe87
Copy link
Author

hi @efiop !
Any news about the bug? I'm very sorry i was a bit busy these last days ;)

@efiop
Copy link
Contributor

efiop commented Dec 9, 2019

@luchoPipe87 No worries 🙂 I still don't have a certain explanation for what is going on your machine, but the workaround that I've provided you with should still work and it will soon become the default behavior (see #2918).

efiop added a commit to efiop/dvc that referenced this issue Dec 13, 2019
As it turned out (see issue numbers down below), we can't really take
hardlinks for granted, so `flufl.lock` is not a panacea for all
filesystems. Considering that the vast majority of filesystems that our
users use support `zc.lockfile`(flock-based) and it has benefits like
more reliable mechanism, auto-delete when process exits, more sturdy
implementation, etc, it makes more sense to bring it back and use by
default again. For filesystems that don't support `flock()`, users will
be able to manually enable `flufl.lock` use through the config option.
It would be ideal if we could auto-detect that flock is not supported,
but in the real world, it turned out to be non-trivial, as it might hang
forever in a kernel context, which makes the implementation way too
complex for our purposes. So what we're doing instead is showing a
message before locking with `zc.lockfile` that, under normal
circumstances will disappear once the lock is taken or failed, otherwise
it will point users to the related documentation where they can learn
about how to opt-in for `flufl.lock`.

Fixes iterative#2831
Fixes iterative#2897
Related iterative#2860
@efiop
Copy link
Contributor

efiop commented Jan 8, 2020

Hi @luchoPipe87 !

Another user is having a similar strange hardlink behavior and we've created a patch that might solve it. If you have time, could you please run commands described in #3080 (comment) and show us the output of the last one, please?

Thanks!

@luchoPipe87
Copy link
Author

Hi!
Here the output of dvc version after installing dvc@3080:

dvc version
WARNING: Unable to detect supported link types, as cache directory '.dvc\cache' doesn't exist. It is usually auto-created by commands such as dvc add/fetch/pull/run/import, but you could create it manually to enable this check.
DVC version: 0.80.0+7839cc
Python version: 3.7.4
Platform: Windows-10-10.0.17134-SP0
Binary: False
Package: None
Filesystem type (workspace): ('NTFS', 'C:\')

@efiop
Copy link
Contributor

efiop commented Jan 9, 2020

Thanks @luchoPipe87 ! 🙏 Could you please create .dvc\cache directory and run it again?

@luchoPipe87
Copy link
Author

hi,
I have created the .dvc\cache manually.
Then i ran the dvc version, here the output:

$dvc version
ERROR: unexpected error

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

@efiop
Copy link
Contributor

efiop commented Jan 9, 2020

@luchoPipe87 Thanks! Ok, so it looks like your case is indeed a bit different. Hardlinks are created, but there is no way for us to count them to verify it. Very interesting. Still not sure what we could do there to fix it, but at least it works for you now with flock() based locks. Btw, did you ask your sys admin about it by any chance? Also, how is your dvc experience so far? 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Did we break something? p0-critical Critical issue. Needs to be fixed ASAP.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants