checkout fails using index

[dberenbaum]

Bug Report

Description

Checkout fails since introducing index checkout

Reproduce

On https://github.com/iterative/iterative.ai/, in a clean workspace, checkout fails:

$ dvc data status
No changes.
(there are changes not tracked by dvc, use "git status" to see)
$ git status
On branch dvc-3.0
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   content/blog/2023-06-14-dvc-3.0.md

no changes added to commit (use "git add" and/or "git commit -a")
$ dvc checkout -v
2023-06-08 20:19:21,642 DEBUG: v3.0.0a3.dev3+ged664143b, CPython 3.10.10 on macOS-13.3.1-arm64-arm-64bit
2023-06-08 20:19:21,643 DEBUG: command: /Users/dave/miniforge3/envs/dvc/bin/dvc checkout -v
2023-06-08 20:19:22,136 ERROR: unexpected error - [Errno 66] Directory not empty: '/Users/dave/Code/iterative.ai/static'
Traceback (most recent call last):
  File "/Users/dave/Code/dvc/dvc/cli/__init__.py", line 209, in main
    ret = cmd.do_run()
  File "/Users/dave/Code/dvc/dvc/cli/command.py", line 26, in do_run
    return self.run()
  File "/Users/dave/Code/dvc/dvc/commands/checkout.py", line 34, in run
    stats = self.repo.checkout(
  File "/Users/dave/Code/dvc/dvc/repo/__init__.py", line 64, in wrapper
    return f(repo, *args, **kwargs)
  File "/Users/dave/Code/dvc/dvc/repo/checkout.py", line 130, in checkout
    changes = icheckout(
  File "/Users/dave/miniforge3/envs/dvc/lib/python3.10/site-packages/dvc_data/index/checkout.py", line 335, in checkout
    _delete_dirs(dirs_delete, path, fs)
  File "/Users/dave/miniforge3/envs/dvc/lib/python3.10/site-packages/dvc_data/index/checkout.py", line 197, in _delete_dirs
    fs.rmdir(fs.path.join(path, *entry.key))
  File "/Users/dave/miniforge3/envs/dvc/lib/python3.10/site-packages/dvc_objects/fs/base.py", line 459, in rmdir
    self.fs.rmdir(path)
  File "/Users/dave/miniforge3/envs/dvc/lib/python3.10/site-packages/dvc_objects/fs/local.py", line 110, in rmdir
    self.fs.rmdir(path)
  File "/Users/dave/miniforge3/envs/dvc/lib/python3.10/site-packages/fsspec/implementations/local.py", line 58, in rmdir
    os.rmdir(path)
OSError: [Errno 66] Directory not empty: '/Users/dave/Code/iterative.ai/static'

2023-06-08 20:19:22,158 DEBUG: Removing '/Users/dave/Code/.M2bysRCShJaqaZZPJsQDaX.tmp'
2023-06-08 20:19:22,158 DEBUG: Removing '/Users/dave/Code/.M2bysRCShJaqaZZPJsQDaX.tmp'
2023-06-08 20:19:22,158 DEBUG: Removing '/Users/dave/Code/.M2bysRCShJaqaZZPJsQDaX.tmp'
2023-06-08 20:19:22,158 DEBUG: Removing '/Users/dave/Code/iterative.ai/.dvc/cache/.Zp7LLJBpP7X4F3Vk6JKfhy.tmp'
2023-06-08 20:19:22,160 DEBUG: Version info for developers:
DVC version: 3.0.0a3.dev3+ged664143b
------------------------------------
Platform: Python 3.10.10 on macOS-13.3.1-arm64-arm-64bit
Subprojects:
        dvc_data = 0.55.0
        dvc_objects = 0.22.0
        dvc_render = 0.5.3
        dvc_task = 0.2.1
        scmrepo = 1.0.3
Supports:
        azure (adlfs = 2023.4.0, knack = 0.10.1, azure-identity = 1.12.0),
        gdrive (pydrive2 = 1.15.3),
        gs (gcsfs = 2023.5.0),
        hdfs (fsspec = 2023.5.0, pyarrow = 11.0.0),
        http (aiohttp = 3.7.4.post0, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.7.4.post0, aiohttp-retry = 2.8.3),
        oss (ossfs = 2021.8.0),
        s3 (s3fs = 2023.5.0, boto3 = 1.26.76),
        ssh (sshfs = 2023.4.1),
        webdav (webdav4 = 0.9.8),
        webdavs (webdav4 = 0.9.8),
        webhdfs (fsspec = 2023.5.0)
Config:
        Global: /Users/dave/Library/Application Support/dvc
        System: /Library/Application Support/dvc
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk3s1s1
Caches: local
Remotes: s3
Workspace directory: apfs on /dev/disk3s1s1
Repo: dvc, git
Repo.site_cache_dir: /Library/Caches/dvc/repo/e4247a13e500df28823f6bc74fd46080

Environment information

Output of dvc doctor:

$ dvc doctor
DVC version: 3.0.0a3.dev3+ged664143b
------------------------------------
Platform: Python 3.10.10 on macOS-13.3.1-arm64-arm-64bit
Subprojects:
        dvc_data = 0.55.0
        dvc_objects = 0.22.0
        dvc_render = 0.5.3
        dvc_task = 0.2.1
        scmrepo = 1.0.3
Supports:
        azure (adlfs = 2023.4.0, knack = 0.10.1, azure-identity = 1.12.0),
        gdrive (pydrive2 = 1.15.3),
        gs (gcsfs = 2023.5.0),
        hdfs (fsspec = 2023.5.0, pyarrow = 11.0.0),
        http (aiohttp = 3.7.4.post0, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.7.4.post0, aiohttp-retry = 2.8.3),
        oss (ossfs = 2021.8.0),
        s3 (s3fs = 2023.5.0, boto3 = 1.26.76),
        ssh (sshfs = 2023.4.1),
        webdav (webdav4 = 0.9.8),
        webdavs (webdav4 = 0.9.8),
        webhdfs (fsspec = 2023.5.0)
Config:
        Global: /Users/dave/Library/Application Support/dvc
        System: /Library/Application Support/dvc
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk3s1s1
Caches: local
Remotes: s3
Workspace directory: apfs on /dev/disk3s1s1
Repo: dvc, git
Repo.site_cache_dir: /Library/Caches/dvc/repo/e4247a13e500df28823f6bc74fd46080

Additional Information (if any):

Regression introduced in 7863771

[efiop]

@dberenbaum That's obviously on me, I'll take a look.

[dberenbaum]

If needed, we could consider rolling back some changes like this until after the release.

[efiop]

@dberenbaum Sure, but I'm sure this is easier to just fix. Taking a look and will let you know.

[efiop]

@dberenbaum If you have time, could you give it a try with the upstream dvc? Just to double confirm that it works for you as well.