[V1] Integrate Piecewise CUDA graphs

[WoosukKwon]

This PR integrates the piecewise CUDA graphs into the V1 model runner.

• Set VLLM_TORCH_COMPILE_LEVEL=3 to enable the feature.
• The compilation + capturing time takes at most ~17 secs.
• Currently, piecewise CUDA graphs are not compatible with custom ops. Therefore, we rely on Torch Inductor to optimize the model.
• Consequently, FP8 or other quantizations are not supported with piecewise CUDA graphs.

Benchmarks (benchmark_latency.py)

Model	Version	CUDA Graphs	Time (s)
OPT-125M (fp16)	V1	w/o Piecewise CUDA Graphs	0.41
	V1	w/ Piecewise CUDA Graphs	0.21
	V0	w/o CUDA Graphs	0.47
	V0	w/ CUDA Graphs	0.21
Llama 3.1 8B (bf16)	V1	w/o Piecewise CUDA Graphs	1.26
	V1	w/ Piecewise CUDA Graphs	1.04
	V0	w/o CUDA Graphs	1.10
	V0	w/ CUDA Graphs	0.99

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. @WoosukKwon please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. @WoosukKwon please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[youkaichao]

glad to see the integration working!