Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Merge _should_test_azure and _get_azure_url inside Azure test helper #3055

Merged
merged 2 commits into from
Jan 5, 2020
Merged

test: Merge _should_test_azure and _get_azure_url inside Azure test helper #3055

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c4783f4
test: merge _should_test_azure and _get_azure_url inside Azure class
skshetry Jan 4, 2020
60e5121
Merge branch 'master' into refactor-remote-azure
efiop Jan 5, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions tests/func/test_data_cloud.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,20 +32,19 @@

from tests.remotes import (
_should_test_aws,
_should_test_azure,
_should_test_gcp,
_should_test_gdrive,
_should_test_hdfs,
_should_test_oss,
_should_test_ssh,
Azure,
TEST_CONFIG,
TEST_SECTION,
TEST_GCP_CREDS_FILE,
TEST_GDRIVE_CLIENT_ID,
TEST_GDRIVE_CLIENT_SECRET,
TEST_REMOTE,
get_aws_url,
get_azure_url,
get_gcp_url,
get_gdrive_url,
get_hdfs_url,
Expand Down Expand Up @@ -264,10 +263,10 @@ def _get_cloud_class(self):

class TestRemoteAZURE(TestDataCloudBase):
def _should_test(self):
return _should_test_azure()
return Azure.should_test()

def _get_url(self):
return get_azure_url()
return Azure.get_url()

def _get_cloud_class(self):
return RemoteAZURE
Expand Down Expand Up @@ -535,10 +534,10 @@ def _test(self):

class TestRemoteAZURECLI(TestDataCloudCLIBase):
def _should_test(self):
return _should_test_azure()
return Azure.should_test()

def _test(self):
url = get_azure_url()
url = Azure.get_url()

self.main(["remote", "add", TEST_REMOTE, url])

Expand Down
33 changes: 15 additions & 18 deletions tests/remotes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,16 +84,6 @@ def _should_test_gcp():
return True


def _should_test_azure():
do_test = env2bool("DVC_TEST_AZURE", undefined=None)
if do_test is not None:
return do_test

return os.getenv("AZURE_STORAGE_CONTAINER_NAME") and os.getenv(
"AZURE_STORAGE_CONNECTION_STRING"
)


def _should_test_oss():
do_test = env2bool("DVC_TEST_OSS", undefined=None)
if do_test is not None:
Expand Down Expand Up @@ -205,12 +195,6 @@ def get_gcp_url():
return "gs://" + get_gcp_storagepath()


def get_azure_url():
container_name = os.getenv("AZURE_STORAGE_CONTAINER_NAME")
assert container_name is not None
return "azure://{}/{}".format(container_name, str(uuid.uuid4()))


def get_oss_storagepath():
return "{}/{}".format(TEST_OSS_REPO_BUCKET, (uuid.uuid4()))

Expand Down Expand Up @@ -271,8 +255,21 @@ def put_objects(remote, objects):


class Azure:
should_test = staticmethod(_should_test_azure)
get_url = staticmethod(get_azure_url)
@staticmethod
def should_test():
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic is different from other should_test, for example:

class GDrive:
    @staticmethod
    def should_test():
        return os.getenv("GDRIVE_USER_CREDENTIALS_DATA") is not None

At least as credentials are set, it will run the tests.

But as I'm reading this, if DVC_TEST_AZURE is defined, it is assumed that AZURE_STORAGE_{CONTAINER_NAME,CONNECTION_STRING} are set.

Why not going for testing only the credentials?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MrOutis Gdrive is the only one that doesn't respect the previous convention 🙂 This change does. DVC_TEST* are useful when you want to ensure that your tests will run even if the env vars are not defined, for example when you have some other way of providing credentials already. For example, should_test for s3 tests for AWS_ env vars, but I don't define them locally, since I have the default ones that work as is.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@efiop , got it 👍

do_test = env2bool("DVC_TEST_AZURE", undefined=None)
if do_test is not None:
return do_test

return os.getenv("AZURE_STORAGE_CONTAINER_NAME") and os.getenv(
"AZURE_STORAGE_CONNECTION_STRING"
)

@staticmethod
def get_url():
container_name = os.getenv("AZURE_STORAGE_CONTAINER_NAME")
assert container_name is not None
return "azure://{}/{}".format(container_name, str(uuid.uuid4()))


class OSS:
Expand Down