Add ability to wait for an environment #1957
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* This allows us not to read the config for the app threads until absolutely necessary.
* Delay loading the cog yaml until it is read from in the code.
* Allow environment variables to control whether we wait for a file to appear before further processing * This allows the python interpreter to boot up while we wait for other files to become available
* Allow waiting for a general environment by importing select python packages while the system boots up around cog.
* I shouldn’t need to do this but want to see if it relieves the errors on linux.
* A small test to make sense of what the code stripper is doing.
* Create a class for accessing cog config * Only access variables from the config by properties * Gate those properties with environment variable function decorators to allow fetching the config from the environment rather than a file. * This allows the environment to begin running without needing the /src
* While waiting for the environment to boot, load The designated imports to speed up interpreter Time.
* In strip model source code we use an AST function that isn’t available prior to 3.9
* We aren’t waiting for these imports, we are loading them eagerly while we wait.
* We have so many that we need to increase this.
* Fix an issue with env_property where it could not handle Optional or Union
* Create these functions because they don’t exist in python 3.7
* Do not wait for a signal, just use a while loop with a 10 ms interval to check for presence of file.
* Some tests previously checked the debug logging * Make sure we conform to the same debug logging
* Confirm that this behaviour is consistent with functions.
* Allows better debugging of stderr et al
* Currently predict assumes that PredictionResponse is the serialisation target. * This isn’t the case if we are calling _predict from a training endpoint * Allow response_type to be fed into the _predict method to inform it of what kind of response it should expect.
* Use the proper endpoints to call training functions. * Log the command properly to the user.
Signed-off-by: Will Sackfield <[email protected]>
Signed-off-by: Will Sackfield <[email protected]>
Merged
Signed-off-by: Will Sackfield <[email protected]>
8W9aG
force-pushed
the
add-waiting-env
branch
2 times, most recently
from
5cf3ace
to
2840fc7
Compare
Signed-off-by: Will Sackfield <[email protected]>
Merged
Signed-off-by: Will Sackfield <[email protected]>
Signed-off-by: Will Sackfield <[email protected]>
|
This has now been slimmed down to just the wait code, so is ready to be reviewed in its own right. |
meatballhat
approved these changes
Nov 14, 2024
| full_module_path = os.path.join( | ||
| pyenv_path, | ||
| "lib", | ||
| "python" + os.environ[PYTHON_VERSION_ENV_VAR], |
There was a problem hiding this comment.
Is it safe to assume PYTHON_VERSION_ENV_VAR is defined?
There was a problem hiding this comment.
Yeh, or at the very least I'd like it to hard fail if it isn't present, it would mean something is very wrong in our system configuration
8W9aG
dismissed
nickstenning’s stale review
PR has been atomised as much as possible.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
wait.pycontaining utilities that allow for waiting for specific files to appear, and for loading imports. These are defined by the environment variablesCOG_WAIT_FILEandCOG_EAGER_IMPORTS(CSV). If neither of these environment variables exist this is a no-op.BasePredictorandBaseInputinto their own class files now that they are accessed by multiple consumers (config and predictor).Configclass which is an abstraction aroundcog.yamlallowing for environment variables to be used as a substitute for certain attributes of thecog.yaml, in theory allowing us to load the cog environment without relying on this file provided we have set the right environment variables.create_appto take aConfigclass rather that a dictionary.env_propertyfunction decorator for reading from an environment variable before dropping into its wrapped function. This prevents the need for loadingcog.yamlif the property is guarded by this.Modeenumeration that delineates between the different modes ofpredictandtrainin a type safe way.setup.Configclass while injecting in some mock values if applicable.code_xformto make sense of its inputs and outputs.This allows cog to begin running and do as much work as possible with as little information as possible while it waits for a file system to appear around it. Currently cog requires
cog.yamland the prediction/training python to appear before setting up the HTTP client, in this case we can setup the HTTP client while we load these files in the background and get a signal that the file loading is finished and continue loading the predictors setup, in addition to this we perform any pre-emptive work we can while we are waiting for these files to load (such as eagerly importing modules).