Add pre- and post-build script to work around Poetry bug #868
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
RobbeSneyders
force-pushed
the
bugfix/install-from-git
branch
from
694ea8e to
3a93905
Compare
RobbeSneyders
force-pushed
the
bugfix/install-from-git
branch
from
3a93905 to
fff4071
Compare
GeorgesLorre
approved these changes
Feb 20, 2024
mrchtr
approved these changes
Feb 20, 2024
| @@ -75,6 +75,9 @@ vertex = ["docker", "kfp", "google-cloud-aiplatform"] | |||
| sagemaker = ["sagemaker", "boto3"] | |||
| docker = ["docker"] | |||
|
|
|||
| all = ["dask", "dask-cuda", "s3fs", "adlfs", "gcsfs", "docker", "kfp", "google-cloud-aiplatform", | |||
There was a problem hiding this comment.
Guess this is useful for local development in case you want to install all dependencies. Should we adjust the readme?
There was a problem hiding this comment.
For local development, this can still be done with poetry install --all-extras. I just needed this to install all extras when installing from the wheel, which I think is only needed in our CI/CD.
Based on the response of the Poetry issue, I'm not sure if we will be able to :) I would follow up on that one instead of having a blocked issue on our repo. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The new packaging strategy introduced in #849 doesn't work correctly for wheels due to a bug in poetry (python-poetry/poetry#8994), which breaks installing both from Github and PyPi. This PR introduces a workaround.
Since the bug is related to the
srcdirectory, this workaround adds apre-build/shscript which unpacks thesrcdirectory before building, and apost-build.shscript which moves the package back into thesrcdirectory afterwards.This leads to the following result:
fondant_component.yamlfiles for each componentIn the ideal case, we get the same result for all installation methods, but this already at least leads to a working result for each method.