Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-761256 Fix finish_download in gcs_storage_client #1488

Merged
merged 3 commits into from
Mar 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions DESCRIPTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ Source code is also available at: https://github.com/snowflakedb/snowflake-conne

# Release Notes

- v3.0.3(TBD)

- Fixed a bug that prints error in logs for GET command on GCS

- v3.0.2(March 23, 2023)

- Fixed a memory leak in the logging module of the Cython extension.
Expand Down
2 changes: 1 addition & 1 deletion src/snowflake/connector/gcs_storage_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ def finish_download(self) -> None:
# Sadly, we can only determine the src file size after we've
# downloaded it, unlike the other cloud providers where the
# metadata can be read beforehand.
self.meta.src_file_size = os.path.getsize(self.intermediate_dst_path)
self.meta.src_file_size = os.path.getsize(self.full_dst_file_name)

def _update_presigned_url(self) -> None:
"""Updates the file metas with presigned urls if any.
Expand Down
8 changes: 6 additions & 2 deletions test/integ/test_put_get.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from __future__ import annotations

import filecmp
import logging
import os
import pathlib
from getpass import getuser
Expand Down Expand Up @@ -675,7 +676,7 @@ def test_get_empty_file(tmp_path, conn_cnx):


@pytest.mark.skipolddriver
def test_get_file_permission(tmp_path, conn_cnx):
def test_get_file_permission(tmp_path, conn_cnx, caplog):
test_file = tmp_path / "data.csv"
test_file.write_text("1,2,3\n")
stage_name = random_string(5, "test_get_empty_file_")
Expand All @@ -687,7 +688,10 @@ def test_get_file_permission(tmp_path, conn_cnx):
f"PUT 'file://{filename_in_put}' @{stage_name}",
)

cur.execute(f"GET @{stage_name}/data.csv file://{tmp_path}")
with caplog.at_level(logging.ERROR):
cur.execute(f"GET @{stage_name}/data.csv file://{tmp_path}")
assert "FileNotFoundError" not in caplog.text

# get the default mask, usually it is 0o022
default_mask = os.umask(0)
os.umask(default_mask)
Expand Down