Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Instantiate default client lazily #428

Closed
iamhatesz opened this issue Apr 23, 2024 · 2 comments · Fixed by #432
Closed

Instantiate default client lazily #428

iamhatesz opened this issue Apr 23, 2024 · 2 comments · Fixed by #432

Comments

@iamhatesz
Copy link

Hi!

I've refactored my code and moved from s3path to cloudpathlib to support S3 and GCS backends. Everything works great except that clients are created in constructors, possibly when credentials are not yet available.

Here's an example:

from cloudpathlib import S3Path
from dotenv import load_dotenv

UPLOADS_PATH = S3Path("s3://my-bucket/uploads")


def main():
    load_dotenv()
    UPLOADS_PATH.exists()  # <--- botocore.exceptions.NoCredentialsError: Unable to locate credentials


if __name__ == "__main__":
    main()

If I call load_dotenv before defining UPLOADS_PATH it works, but that's not a good practice.

I worked around this by making them functions:

@cache
def UPLOADS_PATH() -> S3Path:
    return S3Path("s3://my-bucket/uploads")

but it is an extra boilerplate and also requires changes to the code.

Would it be possible to delay the instantiation of the client class, e.g. to the moment of the first call that actually requires it? I think this is the way it works in s3path since it worked for me for a very long time.

@pjbull
Copy link
Member

pjbull commented Apr 23, 2024

Seems reasonable, thanks for the issue.

Happy to take a PR or tackle this sometime in the near future.

Implementation is probably pretty straightforward and looks like:

  • Adds a property CloudPath.client
  • That property checks if CloudPath._client is None; if so, it instantiates default client and sets ._client; returns self._client.
  • In the __init__, if client kwarg is None, set self._client to None; else, set it with the passed client.

Here's the relevant block in __init__:

# setup client
if client is None:
if isinstance(cloud_path, CloudPath):
client = cloud_path.client
else:
client = self._cloud_meta.client_class.get_default_client()
if not isinstance(client, self._cloud_meta.client_class):
raise ClientMismatchError(
f"Client of type [{client.__class__}] is not valid for cloud path of type "
f"[{self.__class__}]; must be instance of [{self._cloud_meta.client_class}], or "
f"None to use default client for this cloud path class."
)
self.client: Client = client

@pjbull pjbull changed the title Instantiate clients lazily Instantiate default client lazily Apr 23, 2024
@pjbull
Copy link
Member

pjbull commented Aug 29, 2024

@iamhatesz This is realeased now in v0.19.0 if you want to upgrade and test your use case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants