fix: Fix the CUDAGraphs C++ runtime implementation #3067
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
component: tests
narendasan
force-pushed
the
fix_cuda_graph_cpp
branch
7 times, most recently
from
37ea7c2
to
a6e3234
Compare
narendasan
force-pushed
the
fix_cuda_graph_cpp
branch
6 times, most recently
from
8bd7f78
to
0c8f280
Compare
narendasan
commented
Aug 7, 2024
| @@ -219,7 +219,7 @@ jobs: | |||
| export USE_HOST_DEPS=1 | |||
| pushd . | |||
| cd tests/py/dynamo | |||
| python -m pytest -n 4 --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_dynamo_core_runtime_test_results.xml runtime/ | |||
| python -m pytest --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_dynamo_core_runtime_test_results.xml runtime/ | |||
There was a problem hiding this comment.
Update windows tests as well
narendasan
force-pushed
the
fix_cuda_graph_cpp
branch
6 times, most recently
from
43f446f
to
d7ee6e7
Compare
narendasan
force-pushed
the
fix_cuda_graph_cpp
branch
7 times, most recently
from
06241d2
to
96810f3
Compare
narendasan
commented
Aug 12, 2024
| pushd . | ||
| cd tests/py/dynamo | ||
| python -m pytest -ra --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_dynamo_core_runtime_cudagraphs_cpp_test_results.xml runtime/test_002_cudagraphs_cpp.py | ||
| #python -m pytest -ra --junitxml=${RUNNER_TEST_RESULTS_DIR}/tests_py_dynamo_core_runtime_cudagraphs_py_test_results.xml runtime/test_002_cudagraphs_py.py |
There was a problem hiding this comment.
Uncomment the py tests
peri044
approved these changes
Aug 13, 2024
| export USE_HOST_DEPS=1 | ||
| pushd . | ||
| cd tests/py/dynamo | ||
| nvidia-smi |
Signed-off-by: Naren Dasan <[email protected]> Signed-off-by: Naren Dasan <[email protected]>
narendasan
force-pushed
the
fix_cuda_graph_cpp
branch
from
96810f3
to
8971333
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes the implementation intra-engine CUDAGraph support in the C++ runtime. Cleans up the implementation to distinctly handle creating and maintaining persistent buffers as well as adds explicit concurrency management between the calling stream and the engine execution stream.
Fixes #
Type of change
Please delete options that are not relevant and/or add your own.
Checklist: