Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix code inspection notebook #832

Merged
merged 6 commits into from
Feb 5, 2024
Merged

Fix code inspection notebook #832

merged 6 commits into from
Feb 5, 2024

Conversation

PhilippeMoussalli
Copy link
Contributor

@PhilippeMoussalli PhilippeMoussalli commented Jan 31, 2024

When running the lightweight components in a notebook, the component class code inspection fails with error source code is not available.
Full error log here log.txt

The error is related to the fact that inspect relies on the class located in a source file which is not the case in a notebook where everything runs interactively. This is described in details here

Implemented the workaround suggested here

@PhilippeMoussalli PhilippeMoussalli force-pushed the fix-code-inspection-notebook branch from c8d543d to f0326c0 Compare January 31, 2024 16:20
Copy link
Member

@RobbeSneyders RobbeSneyders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @PhilippeMoussalli

Would be nice to include this in our test suite (eg. using https://stackoverflow.com/questions/40172281/unit-tests-for-functions-in-a-jupyter-notebook). Can you either include it in this PR or open a ticket to add it later instead?

from IPython import get_ipython

shell = get_ipython().__class__.__name__
return shell == "ZMQInteractiveShell"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about running in an IPython terminal instead of a notebook? (if shell == "TerminalInteractiveShell")

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this fix work for the other shells as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested on my IDE python consol shell (PyDevTerminalInteractiveShell), jupter notebook (ZMQInteractiveShell) and colab (Shell). I presume those are the main 3 options that will be used for compilation (even IDE console seems a bit unlikely)

src/fondant/pipeline/lightweight_component.py Outdated Show resolved Hide resolved
Comment on lines 325 to 327
component_source = extract_symbols(component_source, component_cls.__name__)[0][
0
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Y u do this Black 😢

Suggested change
component_source = extract_symbols(component_source, component_cls.__name__)[0][
0
]
component_source = extract_symbols(
component_source,
component_cls.__name__,
)[0][0]

Can you try changing it to this and see if Black keeps it this way? 😛

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha it does this

  component_source = extract_symbols(
      component_source,
      component_cls.__name__,
  )[
      0
  ][0]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'll break it into two lines

@PhilippeMoussalli
Copy link
Contributor Author

Thanks @PhilippeMoussalli

Would be nice to include this in our test suite (eg. using https://stackoverflow.com/questions/40172281/unit-tests-for-functions-in-a-jupyter-notebook). Can you either include it in this PR or open a ticket to add it later instead?

Best to tackle it in a separate PR since it seems a bit tricky, opened an issue

@mrchtr mrchtr merged commit 48cd7e0 into main Feb 5, 2024
11 checks passed
@mrchtr mrchtr deleted the fix-code-inspection-notebook branch February 5, 2024 11:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants