Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accessing files in JSON.GZIP format causes UnicodeDecodeError: 'utf-8' #11

Open
iqbalali opened this issue May 5, 2024 · 0 comments
Open

Comments

@iqbalali
Copy link

iqbalali commented May 5, 2024

I'm trying to access files in my S3 bucket. These files JSON.GZIP files. My example code:

conn = st.connection("s3", type=FilesConnection)
json = conn.read(f"s3://static.my_bucket.com/my_file.json.gz", input_format='json')

I get the following error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

I think the issue is that gzip is not supported? Is it possible to add this support, or is there a workaround I can use (that doesn't involve unzipping the gzip files).

Thanks for the module. This is going to be super useful!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant