You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
user finds ImportErrors all over the place, because the relative imports break
We see similar issues with locally registered flows that are stored as pickles.
Ideally, at least for file-based storage, we can infer a "smart" place to run the flow from. In the case of GitHub Storage, we'd want to clone the repo and then use the directory that the flow-defining file lives in as the PWD for execution. This can be inferred entirely off the configuration on the storage object.
This location could also be overridable by the user if they choose.
The text was updated successfully, but these errors were encountered:
jcrist
changed the title
Have agents set configurable PWD on the runs they submit
Have agents set configurable working directory on the runs they submit
Jun 24, 2020
After getting a working version of this up and running I don't see a clear / clean pattern for achieving this. My initial implementation uses a recursive pattern for downloading each file in a set working directory of the repo however this adds a clump of latency and we definitely do not want to be downloading a large amount of files for each run. With GitHub's current API implementation there isn't a clean way to "clone" a repo so it would require some mix of using GitHub's API with native git bindings and that doesn't feel right / would definitely get weird when trying to work with private repositories.
Discussed this with @cicdw offline it seems like the issue this is trying to solve would be better suited with some sort of git sync that agents could use to mount files on flow runs. That way agent's could be "warmed up" with a repository's contents for use. The issue is still valid but I feel like this could be rolled into something else.
This is motivated by the following scenario:
ImportError
s all over the place, because the relative imports breakWe see similar issues with locally registered flows that are stored as pickles.
Ideally, at least for file-based storage, we can infer a "smart" place to run the flow from. In the case of GitHub Storage, we'd want to clone the repo and then use the directory that the flow-defining file lives in as the PWD for execution. This can be inferred entirely off the configuration on the storage object.
This location could also be overridable by the user if they choose.
The text was updated successfully, but these errors were encountered: