ci updates #18
Merged
ci updates #18
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
agrski
pushed a commit
that referenced
this pull request
Dec 2, 2022
* Initial commit for REST reverse proxy * refactor and use v2client * initial cache manager * add LRU functionality * fix LRU test * wire up cache * Add locking and tests * add edge case tests * tidy up test * tidy up tests and add concurrent update * add a test for extract model name from path * initial state manager strucutre * rename file * adding locking * unload model when we evict * go mod changes * fix test * retun priority when evict as well * integrate cache in reverse proxy * wire up state manager in client * tidy up get model details * wire up reverse proxy at start * Change model version to unit32 * Initial Agent version changes. Code refactor. * lint and ensure model name is changed * Add unload model functionality to agent to handle versions * Add version cleanup to scheduler * Updates for k8s * lint * small updates * change download to use hash for rclone folder to eventually allow caching of copy sync * Modify protos to remove version and restructure * k8s testing via grpc * Update operator * ensure k8s generation works * Allow multiple vesions in ModelStatus call * Handle terminate and k8s updates * lint * add retry for status update and fix event creation * lint * fix retries for client -> scheduler * Add triton server handler to agent * grpc test * Add Triton * fix server http port typos * Fix Envoy resource bug and add server memory notebook tests * Add triton xgboost example and always copy triton config * lint * Initial Manager reconcile for Server * Add servers to kustomize * Updates for agent start from env and updates to serverconfigs * Add Status Handling for Servers * handle deleted servers and consistent ordering of replicas * Add logging * sync model reload with concurrent infer requests * Add Server Notification and events * buggy wg * review comments changes * lint * change minReplicas to 1 if not specified * remove unnecesary get model * fix test * wait on concurrent model reload * fix model reschedule to server on server delete * better reverse proxy lifecycle management * updated to fix delete server tests in k8s notebook * lint * review comments for client state * implement delete item using the heap Remove util * fix test after merge * Move channel comms to central hub * improve agent server tests * lint * review comment updates * Fix hub channel close safety * review comment updates * Make addListener signal type of channel * Set channel as send only for config updates * update Makefile to allow servers to be deployed * Update scheduler server deployment (as opposed to operator server deployment) and add debug for filters when they fail in scheduling * lint * Updates from review * fix typos * Agent Service for debugging * makefile changes * return more state in status endpoint * changes to notebook * further changes post-merge * tidy up logic for concurrency management * tidy up interface for clientservices * tidy up test * serialise actions on same model (control plane) * Add a test for concurrent unload * tidy up code * if there is an error in makeroom try next model * add test data and control plane * tidy up some of the concurrency * deal with concurrency parallel unload * having locks per model across entire stack * sorting out concurrency issues * fix deadlock * tidy up test * check memory server capacity before update * return 404 if the model cannot be found / reloaded * fix reverse proxy test * notebook changes * notebook changes * post merge #2 * fix lint * notebook changes * export reverse proxy ports * refactor function Co-authored-by: Clive Cox <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.