get: Failed to clone repo (local)

[jorgeorpinel]

Originally posted by @Toekan in #3590 (comment)

...I'm trying to use dvc get path/to/local_dvc_project name_of_file_to_download and I'm getting:

ERROR: failed to get 'name_of_file_to_download' from 'local_dvc_project' - Failed to clone repo 'local_dvc_project'.

Has there been work done on allowing to use dvc get without git? I saw people wondering about use cases: I want to use dvc to download some files during runtime of a docker container, at which point I have no git credentials. Ideally I would just use boto3, but at the moment I'm not sure how to reconstruct the path to the file I want to download.

[jorgeorpinel]

Failed to clone repo 'local_dvc_project'.

Sounds like path/to/local_dvc_project is not a git repo. Is it a subdirectory of a repo? If so you can try something like

dvc get path/to_repo subdir/file_to_download

[jorgeorpinel]

From #3590 (comment) is seems like list at least should open local repos directly but I'm not sure about get. For now can you post the output of the same command but adding the -v flag, @Toekan? Thanks

[yukipyng]

Hello, I have run into the same issue but with dvc import.
For my situation, In our team we have a large git monorepo where there are 2 dvc repos which have initialized with dvc init --subdir. Those .pt files are generated from the stage and I want to import model_B_1.pt into dvc_project_A for further development.

. （git repo）
├── others scripts without dvc
├── dvc_project_A 
│   └── output
│       ├── model_A_2.pt
│       └── model_A_1.pt
└── dvc_project_B
    └── output
        └── model_B_1.pt

When I'm trying to use dvc import ../dvc_project_B output/model_B_1.pt --out output, I'm also getting:

ERROR: failed to import 'outputs/model_B_1.pt' from '../dvc_project_B/'. - Failed to clone repo '../dvc_project_B/' to '/tmp/tmpzyo_6ri4dvc-clone': Cmd('git') failed due to: exit code(128)
  cmdline: git clone -v --no-single-branch --progress ../dvc_project_B/ /tmp/tmpzyo_6ri4dvc-clone

After adding -v flag, my error message is as follows:

2022-01-19 18:00:50,652 DEBUG: Lockfile 'dvc.lock' needs to be updated.
2022-01-19 18:00:50,694 DEBUG: Removing output 'output/model_B_1.pt' of stage: 'output/model_B_1.pt.dvc'.
2022-01-19 18:00:50,694 DEBUG: Removing 'output/model_B_1.pt'
Importing 'output/model_B_1.pt (../dvc_project_B/)' -> 'output/model_B_1.pt'
2022-01-19 18:00:50,695 DEBUG: Computed stage: 'output/model_B_1.pt.dvc' md5: '5ab7adf883c617c909b7ff79b13249e6'
2022-01-19 18:00:50,695 DEBUG: 'md5' of stage: 'output/model_B_1.pt.dvc' changed.
2022-01-19 18:00:50,695 DEBUG: Creating external repo ../dvc_project_B/@None
2022-01-19 18:00:50,695 DEBUG: erepo: git clone '../dvc_project_B/' to a temporary dir
2022-01-19 18:00:50,725 ERROR: failed to import 'outputs/model_B_1.pt' from '../dvc_project_B/'. - Failed to clone repo '../dvc_project_B/' to '/tmp/tmpt2oiatz_dvc-clone': Cmd('git') failed due to: exit code(128)
  cmdline: git clone -v --no-single-branch --progress ../dvc_project_B/ /tmp/tmpt2oiatz_dvc-clone
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/scm/git/backend/gitpython.py", line 181, in clone
    tmp_repo = clone_from()
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/git/repo/base.py", line 1111, in clone_from
    return cls._clone(git, url, to_path, GitCmdObjectDB, progress, multi_options, **kwargs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/git/repo/base.py", line 1041, in _clone
    handle_process_output(proc, None, to_progress_instance(progress).new_message_handler(),
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/git/cmd.py", line 141, in handle_process_output
    return finalizer(process)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/git/util.py", line 370, in finalize_process
    proc.wait(**kwargs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/git/cmd.py", line 447, in wait
    raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git clone -v --no-single-branch --progress ../dvc_project_B/ /tmp/tmpt2oiatz_dvc-clone

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/command/imp.py", line 14, in run
    self.repo.imp(
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/repo/imp.py", line 6, in imp
    return self.imp_url(
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/repo/__init__.py", line 50, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/repo/scm_context.py", line 14, in run
    return method(repo, *args, **kw)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/repo/imp_url.py", line 83, in imp_url
    stage.run(jobs=jobs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/funcy/decorators.py", line 45, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/stage/decorators.py", line 36, in rwlocked
    return call()
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/funcy/decorators.py", line 66, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/stage/__init__.py", line 532, in run
    self._sync_import(dry, force, jobs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/funcy/decorators.py", line 45, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/stage/decorators.py", line 36, in rwlocked
    return call()
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/funcy/decorators.py", line 66, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/stage/__init__.py", line 556, in _sync_import
    sync_import(self, dry, force, jobs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/stage/imports.py", line 47, in sync_import
    stage.deps[0].download(stage.outs[0], jobs=jobs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/dependency/repo.py", line 68, in download
    for odb, objs in self.get_used_objs().items():
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/dependency/repo.py", line 96, in get_used_objs
    used, _ = self._get_used_and_obj(**kwargs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/dependency/repo.py", line 109, in _get_used_and_obj
    with self._make_repo(
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/external_repo.py", line 32, in external_repo
    path = _cached_clone(url, rev, for_write=for_write)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/external_repo.py", line 152, in _cached_clone
    clone_path, shallow = _clone_default_branch(url, rev, for_write=for_write)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/funcy/decorators.py", line 45, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/funcy/flow.py", line 274, in wrap_with
    return call()
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/funcy/decorators.py", line 66, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/external_repo.py", line 216, in _clone_default_branch
    git = Git.clone(url, clone_path)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/scm/git/__init__.py", line 117, in clone
    backend.clone(url, to_path, **kwargs)
  File "/home/support/miniconda3/envs/model_training_pytorch/lib/python3.8/site-packages/dvc/scm/git/backend/gitpython.py", line 186, in clone
    raise CloneError(url, to_path) from exc
dvc.scm.base.CloneError: Failed to clone repo '../dvc_project_B/' to '/tmp/tmpt2oiatz_dvc-clone'
------------------------------------------------------------
2022-01-19 18:00:50,727 DEBUG: Analytics is enabled.
2022-01-19 18:00:50,758 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmp2qroqvtn']'
2022-01-19 18:00:50,759 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmp2qroqvtn']'

I'm still quite new to dvc, so I'm not sure if it is a bug or any error from our side. Thanks in advance.

[pared]

From:
#3590 (comment)

@Tokean
instead of
dvc get path/to/local_dvc_project name_of_file_to_download
you could use dvc get-url path/to/local_dvc_project/name_of_file_to_download
Does it make sense in your use case?

[pared]

@yukipyng
could you try specyfying git repo as a target repo? In this case:

dvc import .. dvc_project_B/output/model_B_1.pt --out output

instead of

dvc import ../dvc_project_B output/model_B_1.pt --out output

[yukipyng]

@yukipyng could you try specyfying git repo as a target repo? In this case:

dvc import .. dvc_project_B/output/model_B_1.pt --out output

instead of

dvc import ../dvc_project_B output/model_B_1.pt --out output

That worked! Thanks a lot!