If a gif image enters the tmp dir, it will cause Permission Error and NotADirectory error

[LittleApple-fp16]

i got [WinError 32] and [WinError 267] on Windows10

[LittleApple-fp16]

It's testing on some earlier commits, I found the latest code fixes at web.py(30): TemporaryDirectory(ignore_cleanup_errors=True).
I tried this param before, but it didn't work, and I asked some Windows users who are on latest commit, seems they don't have NotADirError now

[LittleApple-fp16]

In any case, I don't think it's a good idea just ignore it. I don't know how temp directories work in Windows, I guess these large gifs may continue to take up some space?

[narugo1992]

can u provide a minimum example to reproduce this issue?

we have been failed to reproduce this.

[ALT99999]

In any case, I don't think it's a good idea just ignore it. I don't know how temp directories work in Windows, I guess these large gifs may continue to take up some space?

你得给明白：
报错详情
运行环境
具体涉及到的本地代码
以及最好有多次实验来缩小范围

[LittleApple-fp16]

报错详情

对不起😓，我给的信息有点少！这是报错详情

首先我看到一个权限错误 表示一个gif图像正被其他进程锁定
之后是一个NotADirectory错误
我暂时没法复现 也没有截图 但是我记得在一个cleanup()方法里 这些代码段终止了

我的猜测：temp模块正在试图删除一个gif图像的缓存 但可能由于操作系统的未知原因失败了 而且下方的代码又把它当成了一个文件夹 但清理还是失败了

我的猜测：错误发生在 source/web.py 的 with TemporaryDirectory() as td: 位置
在测试中只有gif格式的图像会造成 另一个程序正在使用此文件，进程无法访问

我曾经试过 d73c198 中30行添加的参数 ignore_cleanup_errors=True 当时错误仍然复现了
我不知道 d73c198 是不是针对此类问题进行的修复 所以有点困惑 希望你们开发组可以稍微解释下

运行环境

在两台Windows11系统和Windows10系统的PC Python3.10.6
能确定的是错误发生在commit d73c198 之前

后续问题

总之在拉取最新版代码后没有出现这个错误 以及我询问了更多使用新版代码的人 他们也没有
我仍然怀疑如果只是用这个忽略错误的参数避免程序崩溃（我没看它的具体运作方式 我不太懂Python）可能让缓存文件夹有这些gif图的冗余，具体就是 C:/Users/<Your PC>/AppData/Local/Temp/tmps0meth1ng/<sourcename>_1234567.gif 会留下来，造成一点空间占用上的问题，但这仍然是猜测，我的设备条件还不支持全天候的测试，有一些tag会爬取到gif图，有一些则不会，甚至过一段时间又会爬取到新的gif图。
所以我拉了最新版代码，试着加了一个扩展名的判断，不让这些gif图进入缓存 #12
真的抱歉让你们去测试😥，我只有一个人，没有太多时间

[narugo1992]

Hi @LittleApple-fp16 .

We've tried all the environments available to us, including Python 3.8-3.11, and various operating systems such as Ubuntu, Windows, and macOS. Additionally, all the downloaded content is in GIF format, but we still couldn't reproduce the issue. Therefore, if this problem needs to be resolved, we'll need you to provide the minimal reproducible code on your end so that we can investigate the issue.

The code we've attempted to use for reproduction is available here, but unfortunately, it hasn't reproduced successfully in any of the runtime environments.

[LittleApple-fp16]

Thanks for your test @narugo1992.

I did some tests with history code and the problem recurred.
on python 3.10.6 Windows10.
about 120it you can see NotADirectoryError and process will shutdown.

recurred even after adding this parameter ignore_cleanup_errors=True
using waifuc@ff7c32f74bd2ba4cb5fe980814fa611b674e9df3 no other dependencies

[narugo1992]

@LittleApple-fp16 This is our try on your code with Github Action: https://github.com/narugo1992/gif_pil_test/actions/runs/7110049756/job/19355939844 .

About 130+ images crawled, nothing happened, is that the same as yours?

[LittleApple-fp16]

@LittleApple-fp16 This is our try on your code with Github Action: https://github.com/narugo1992/gif_pil_test/actions/runs/7110049756/job/19355939844 .

About 130+ images crawled, nothing happened, is that the same as yours?

Weird, this is mine:
https://github.com/LittleApple-fp16/gif_test_windows/actions/runs/7123284370/job/19395660689
NotADirectoryError appears at line 359

[narugo1992]

A surprising revelation: I forked the code repository you provided, ran the code personally, and encountered the same error! However, after multiple runs in my repository, I couldn't reproduce the issue.

I initially suspected subtle differences in the executed code. Still, I compared your test.py and my gif_150.py through diff, and the contents were identical.

The GitHub Action configuration file you used for running, too, matched mine.

Hence, I am inclined to believe that the only possibility is the direct inclusion of waifuc source code in the repository causing this anomaly. While this sounds unbelievable, all other options have been ruled out. I will double-check to confirm shortly.

[narugo1992]

@LittleApple-fp16 Confirmed!

Deleting the local waifuc source code and adding dependencies to requirements.txt can make the error disappear. My experiment, available here, shows that the error has vanished. You can refer to the specific changes made at: LittleApple-fp16/gif_test_windows@main...narugo1992:gif_test_windows:test/install.

[LittleApple-fp16]

Thanks again! @narugo1992

The problem doesn't seem to be with the package, I know it may have been fixed in follow commits, but I just trying to figure out which fixed it.

About package, in this test I update the waifuc source code to latest and it passed. So I decided it's the version differences, and that was the difference between 87cc9f6 & newer commits.

Initially I thought that the weird bug was fixed by changes in web.py and it may cause might some cache in temp folder, however, in later tests I found that web.py history in the commit of waifuc does not affect this bug, the latest waifuc source passed the temp folder test directly.

Now we can close this issue, it's confirmed that was fix in the history commit by unknown way.

but I still want to know how it be fixed

[LittleApple-fp16]

@narugo1992 lucky!! this problem appeared in the cyberharem again! and i finally found the way to solved it. in debugging the breakpoints of cyberharem, i found that using imgutils to open images in a loop is fatal.

Usually, an opened image in current loop will be locked by python. when it throws out and enters the next loop, until the next be open, the image in last loop is still locked, which leads to the last image in a processing stage always being locked.
At when it happend, if other code segment try to operate it, the error occurs.

There is my job, still using which commit occurs NotADirectory Error: (https://github.com/LittleApple-fp16/gif_test_windows/actions/runs/7264197580/job/19791190755)
closed that image object after yield, and error disappered

[codexq123]

@LittleApple-fp16
同样的错误，请问如何修改代码，感谢

[LittleApple-fp16]

@LittleApple-fp16 同样的错误，请问如何修改代码，感谢

其实造成这种报错的可能性非常多样 因为耗子大量使用临时目录，任何位置和原因的一个异常都可能最后抛出这个错误 所以我还是建议你单独开一个提问 尽可能详细描述你的操作
或者在产生问题的行前后使用断点调试 比较有效，希望能帮到你！