You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
imports3fss3=s3fs.S3FileSystem(client_kwargs={"endpoint_url": "http://my-minio-server.url"})
# test_bug is a folder with a few files in it. s3.put("/path/to/test_bug", "s3://bug", recursive=True)
print("Done")
This gives
Output:
Traceback (most recent call last):
File "s3fs_bug.py", line 6, in <module>
s3.put("/path/to/test_bug", "s3://bug", recursive=True)
File "/snip/lib/python3.8/site-packages/fsspec/asyn.py", line 72, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/snip/lib/python3.8/site-packages/fsspec/asyn.py", line 53, in sync
raise result[0]
File "/snip/lib/python3.8/site-packages/fsspec/asyn.py", line 20, in _runner
result[0] = await coro
File "/snip/lib/python3.8/site-packages/fsspec/asyn.py", line 283, in _put
return await _throttled_gather(
File "/snip/lib/python3.8/site-packages/fsspec/asyn.py", line 144, in _throttled_gather
results.extend(await asyncio.gather(*chunk, **gather_kwargs))
File "/snip/lib/python3.8/site-packages/s3fs/core.py", line 851, in _put_file
with open(lpath, "rb") as f0:
IsADirectoryError: [Errno 21] Is a directory: '/path/to/test_bug'
I noticed the following while investigating this:
This issue is the same regardless of whether s3://bug exists.
This issue is the same with any combination of trailing slashes on lpath and rpath
The problem does not occur when copying to a subfolder within the bucket: if you replace s3://bug with s3://bug/foo then the problem goes away.
This didn't happen on an older version (0.4.2 on python 3.6)
Environment:
Dask version: s3fs 2021.5.0, the latest available in conda
Python version: 3.8.10
Operating System: Ubuntu
Install method (conda, pip, source): conda
The text was updated successfully, but these errors were encountered:
This gives
I noticed the following while investigating this:
s3://bug
exists.s3://bug
withs3://bug/foo
then the problem goes away.Environment:
The text was updated successfully, but these errors were encountered: