Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add cross_sync #999

Merged
merged 270 commits into from
Nov 8, 2024
Merged

chore: add cross_sync #999

merged 270 commits into from
Nov 8, 2024

Conversation

daniel-sanche
Copy link
Contributor

@daniel-sanche daniel-sanche commented Jul 18, 2024

PR 1/x for adding the sync surface for the new data client

This class adds ast transformers for converting existing async classes into sync copies.

The conversion works through the following phases:

  • generate.py is called with a path, and finds all Python files in all subdirectories
  • CrossSyncClassDecoratorHandler is called on each file, looking for any classes decorated with @CrossSync.export_sync
  • For all classes found, they are passed through:
    • CrossSyncMethodDecoratorHandler to look for any methods annotated with CrossSync decorators (CrossSunc.convert, CrossSync.drop_method, CrossSync.pytest, etc).
    • RmAioFunctions to strip out any asyncio keywords annotated with CrossSync.rmaio()
    • SymbolReplacer to replace CrossSync with CrossSync._Sync_Impl in the codebase, and other user-specified symbol transformatioons

Next PR will add annotations to existing code to guide transformations

.cross_sync/generate.py Outdated Show resolved Hide resolved
.cross_sync/transformers.py Outdated Show resolved Hide resolved
.cross_sync/transformers.py Outdated Show resolved Hide resolved
.cross_sync/transformers.py Show resolved Hide resolved
.cross_sync/transformers.py Show resolved Hide resolved
.cross_sync/transformers.py Outdated Show resolved Hide resolved
self.tree = ast_tree
self.header = header or ""

def render(self, with_black=True, save_to_disk: bool = False) -> str:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we default save_to_disk to true?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I said no, because we also use this function as part of tests later, to compare to make sure the current outputs are up to date. In that case, we'd want to see the output without writing it. But I'm fine either way

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a user of the library will want to set this to true, and we can override it for testing, let's make it default to true?

if new_mapping:
CrossSync.add_mapping(new_mapping, cls)
if self.async_docstring_format_vars:
cls.__doc__ = cls.__doc__.format(**self.async_docstring_format_vars)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this do? why is this only formatting async_docstring? How about sync_docstring?

Copy link
Contributor Author

@daniel-sanche daniel-sanche Oct 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CrossSync allows you to add variables into docstrings, and then it will replace them with a different string for sync vs async. This came up because some async methods raise exceptions when run outside of an event loop context, but that doesn't apply to sync versions. So the docstrings are mostly the same, but with an extra little note for async

On the sync side, we can replace the variables when generating the sync file. On the async side, we update the __doc__ attribute of the class as part of the class decoration process. The code itself will have template variables when you read it directly, but docs generators will read from class.__doc__ when generating content, so it will render as expected

@daniel-sanche daniel-sanche added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 5, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 5, 2024
@daniel-sanche daniel-sanche added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 8, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 8, 2024
@daniel-sanche daniel-sanche merged commit 2057c20 into main Nov 8, 2024
28 of 32 checks passed
@daniel-sanche daniel-sanche deleted the cross_sync2_pr1_architecture branch November 8, 2024 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the googleapis/python-bigtable API. size: xl Pull request size is extra large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants