-
-
Notifications
You must be signed in to change notification settings - Fork 4.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Speculative Decoding] Support draft model on different tensor-parallel size than target model #5414
Merged
Commits on Jun 10, 2024
-
Configuration menu - View commit details
-
Copy full SHA for f5b5f94 - Browse repository at this point
Copy the full SHA f5b5f94View commit details -
Configuration menu - View commit details
-
Copy full SHA for 709de21 - Browse repository at this point
Copy the full SHA 709de21View commit details
Commits on Jun 12, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 0eacc96 - Browse repository at this point
Copy the full SHA 0eacc96View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2011ed0 - Browse repository at this point
Copy the full SHA 2011ed0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2e16c4e - Browse repository at this point
Copy the full SHA 2e16c4eView commit details -
Configuration menu - View commit details
-
Copy full SHA for b412a51 - Browse repository at this point
Copy the full SHA b412a51View commit details -
Configuration menu - View commit details
-
Copy full SHA for 593ccfa - Browse repository at this point
Copy the full SHA 593ccfaView commit details -
Configuration menu - View commit details
-
Copy full SHA for c5d3476 - Browse repository at this point
Copy the full SHA c5d3476View commit details -
Configuration menu - View commit details
-
Copy full SHA for 44e623b - Browse repository at this point
Copy the full SHA 44e623bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 98caf17 - Browse repository at this point
Copy the full SHA 98caf17View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7fc4ff5 - Browse repository at this point
Copy the full SHA 7fc4ff5View commit details -
Configuration menu - View commit details
-
Copy full SHA for a96e720 - Browse repository at this point
Copy the full SHA a96e720View commit details -
Configuration menu - View commit details
-
Copy full SHA for db39576 - Browse repository at this point
Copy the full SHA db39576View commit details -
Configuration menu - View commit details
-
Copy full SHA for b2e8595 - Browse repository at this point
Copy the full SHA b2e8595View commit details -
Configuration menu - View commit details
-
Copy full SHA for 756442a - Browse repository at this point
Copy the full SHA 756442aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 32094f1 - Browse repository at this point
Copy the full SHA 32094f1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7890191 - Browse repository at this point
Copy the full SHA 7890191View commit details -
Configuration menu - View commit details
-
Copy full SHA for 53b2ea9 - Browse repository at this point
Copy the full SHA 53b2ea9View commit details -
Configuration menu - View commit details
-
Copy full SHA for a29c9c5 - Browse repository at this point
Copy the full SHA a29c9c5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 52ba09d - Browse repository at this point
Copy the full SHA 52ba09dView commit details -
Configuration menu - View commit details
-
Copy full SHA for d26ef08 - Browse repository at this point
Copy the full SHA d26ef08View commit details -
Configuration menu - View commit details
-
Copy full SHA for 80c4994 - Browse repository at this point
Copy the full SHA 80c4994View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0f16f3f - Browse repository at this point
Copy the full SHA 0f16f3fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 140f478 - Browse repository at this point
Copy the full SHA 140f478View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3fd7e91 - Browse repository at this point
Copy the full SHA 3fd7e91View commit details -
Configuration menu - View commit details
-
Copy full SHA for 495aa30 - Browse repository at this point
Copy the full SHA 495aa30View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3a5a47f - Browse repository at this point
Copy the full SHA 3a5a47fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 07ddbb8 - Browse repository at this point
Copy the full SHA 07ddbb8View commit details -
Configuration menu - View commit details
-
Copy full SHA for b0a677d - Browse repository at this point
Copy the full SHA b0a677dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 96782a2 - Browse repository at this point
Copy the full SHA 96782a2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9998b9c - Browse repository at this point
Copy the full SHA 9998b9cView commit details -
Configuration menu - View commit details
-
Copy full SHA for e92ecdc - Browse repository at this point
Copy the full SHA e92ecdcView commit details -
Configuration menu - View commit details
-
Copy full SHA for b421607 - Browse repository at this point
Copy the full SHA b421607View commit details -
Configuration menu - View commit details
-
Copy full SHA for 386ab9b - Browse repository at this point
Copy the full SHA 386ab9bView commit details -
Configuration menu - View commit details
-
Copy full SHA for b25f74e - Browse repository at this point
Copy the full SHA b25f74eView commit details
Commits on Jun 13, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 8b51f08 - Browse repository at this point
Copy the full SHA 8b51f08View commit details -
Configuration menu - View commit details
-
Copy full SHA for d4b283c - Browse repository at this point
Copy the full SHA d4b283cView commit details -
Configuration menu - View commit details
-
Copy full SHA for dfc90cb - Browse repository at this point
Copy the full SHA dfc90cbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9bef5e4 - Browse repository at this point
Copy the full SHA 9bef5e4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 85d087d - Browse repository at this point
Copy the full SHA 85d087dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9af36b7 - Browse repository at this point
Copy the full SHA 9af36b7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5a0bf45 - Browse repository at this point
Copy the full SHA 5a0bf45View commit details -
Configuration menu - View commit details
-
Copy full SHA for 531c9f0 - Browse repository at this point
Copy the full SHA 531c9f0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 287da20 - Browse repository at this point
Copy the full SHA 287da20View commit details -
Configuration menu - View commit details
-
Copy full SHA for 08d1b2a - Browse repository at this point
Copy the full SHA 08d1b2aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 237c966 - Browse repository at this point
Copy the full SHA 237c966View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0bb38c2 - Browse repository at this point
Copy the full SHA 0bb38c2View commit details -
Configuration menu - View commit details
-
Copy full SHA for c097d6c - Browse repository at this point
Copy the full SHA c097d6cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 957a325 - Browse repository at this point
Copy the full SHA 957a325View commit details
Commits on Jun 14, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3ec8cb5 - Browse repository at this point
Copy the full SHA 3ec8cb5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8a8a1e4 - Browse repository at this point
Copy the full SHA 8a8a1e4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7f06f64 - Browse repository at this point
Copy the full SHA 7f06f64View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1e87579 - Browse repository at this point
Copy the full SHA 1e87579View commit details -
Configuration menu - View commit details
-
Copy full SHA for abc546c - Browse repository at this point
Copy the full SHA abc546cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7880cb0 - Browse repository at this point
Copy the full SHA 7880cb0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2ebe6f3 - Browse repository at this point
Copy the full SHA 2ebe6f3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 90d46ee - Browse repository at this point
Copy the full SHA 90d46eeView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7e1426c - Browse repository at this point
Copy the full SHA 7e1426cView commit details -
Configuration menu - View commit details
-
Copy full SHA for ad52d93 - Browse repository at this point
Copy the full SHA ad52d93View commit details -
Configuration menu - View commit details
-
Copy full SHA for 355475b - Browse repository at this point
Copy the full SHA 355475bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9cfdb5b - Browse repository at this point
Copy the full SHA 9cfdb5bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6a6c5ff - Browse repository at this point
Copy the full SHA 6a6c5ffView commit details -
Configuration menu - View commit details
-
Copy full SHA for ddef229 - Browse repository at this point
Copy the full SHA ddef229View commit details -
Configuration menu - View commit details
-
Copy full SHA for 965f648 - Browse repository at this point
Copy the full SHA 965f648View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1bb5534 - Browse repository at this point
Copy the full SHA 1bb5534View commit details -
Configuration menu - View commit details
-
Copy full SHA for ea6b8f5 - Browse repository at this point
Copy the full SHA ea6b8f5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 71977d2 - Browse repository at this point
Copy the full SHA 71977d2View commit details -
Configuration menu - View commit details
-
Copy full SHA for bc5f77a - Browse repository at this point
Copy the full SHA bc5f77aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5655a49 - Browse repository at this point
Copy the full SHA 5655a49View commit details -
Configuration menu - View commit details
-
Copy full SHA for eabc16a - Browse repository at this point
Copy the full SHA eabc16aView commit details
Commits on Jun 17, 2024
-
Configuration menu - View commit details
-
Copy full SHA for f748edf - Browse repository at this point
Copy the full SHA f748edfView commit details -
Configuration menu - View commit details
-
Copy full SHA for c099c94 - Browse repository at this point
Copy the full SHA c099c94View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4b74a45 - Browse repository at this point
Copy the full SHA 4b74a45View commit details -
Configuration menu - View commit details
-
Copy full SHA for c9786ad - Browse repository at this point
Copy the full SHA c9786adView commit details -
Configuration menu - View commit details
-
Copy full SHA for a42664a - Browse repository at this point
Copy the full SHA a42664aView commit details -
Configuration menu - View commit details
-
Copy full SHA for ac7701a - Browse repository at this point
Copy the full SHA ac7701aView commit details -
Configuration menu - View commit details
-
Copy full SHA for eea6a7e - Browse repository at this point
Copy the full SHA eea6a7eView commit details -
Configuration menu - View commit details
-
Copy full SHA for a648f5d - Browse repository at this point
Copy the full SHA a648f5dView commit details -
Configuration menu - View commit details
-
Copy full SHA for f23ba8c - Browse repository at this point
Copy the full SHA f23ba8cView commit details -
Configuration menu - View commit details
-
Copy full SHA for aa9af93 - Browse repository at this point
Copy the full SHA aa9af93View commit details
Commits on Jun 18, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 56c8927 - Browse repository at this point
Copy the full SHA 56c8927View commit details -
Configuration menu - View commit details
-
Copy full SHA for 385b4f8 - Browse repository at this point
Copy the full SHA 385b4f8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 43f37eb - Browse repository at this point
Copy the full SHA 43f37ebView commit details -
Configuration menu - View commit details
-
Copy full SHA for 99350e2 - Browse repository at this point
Copy the full SHA 99350e2View commit details -
Configuration menu - View commit details
-
Copy full SHA for a9f3e23 - Browse repository at this point
Copy the full SHA a9f3e23View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6ba250d - Browse repository at this point
Copy the full SHA 6ba250dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3e78613 - Browse repository at this point
Copy the full SHA 3e78613View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6532af7 - Browse repository at this point
Copy the full SHA 6532af7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6839797 - Browse repository at this point
Copy the full SHA 6839797View commit details
Commits on Jun 19, 2024
-
Configuration menu - View commit details
-
Copy full SHA for aac586b - Browse repository at this point
Copy the full SHA aac586bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 98e584d - Browse repository at this point
Copy the full SHA 98e584dView commit details
Commits on Jun 20, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 2d5e64d - Browse repository at this point
Copy the full SHA 2d5e64dView commit details -
Configuration menu - View commit details
-
Copy full SHA for ba88bd4 - Browse repository at this point
Copy the full SHA ba88bd4View commit details
Commits on Jun 21, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 46e5274 - Browse repository at this point
Copy the full SHA 46e5274View commit details -
Configuration menu - View commit details
-
Copy full SHA for 85f4f25 - Browse repository at this point
Copy the full SHA 85f4f25View commit details -
Configuration menu - View commit details
-
Copy full SHA for c1b5373 - Browse repository at this point
Copy the full SHA c1b5373View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4a58617 - Browse repository at this point
Copy the full SHA 4a58617View commit details -
Configuration menu - View commit details
-
Copy full SHA for b09e7be - Browse repository at this point
Copy the full SHA b09e7beView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7168d78 - Browse repository at this point
Copy the full SHA 7168d78View commit details -
Configuration menu - View commit details
-
Copy full SHA for fe0bd5b - Browse repository at this point
Copy the full SHA fe0bd5bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2e0d170 - Browse repository at this point
Copy the full SHA 2e0d170View commit details -
Configuration menu - View commit details
-
Copy full SHA for 36f8aa5 - Browse repository at this point
Copy the full SHA 36f8aa5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 54bf514 - Browse repository at this point
Copy the full SHA 54bf514View commit details -
Configuration menu - View commit details
-
Copy full SHA for bfd7d2f - Browse repository at this point
Copy the full SHA bfd7d2fView commit details -
Configuration menu - View commit details
-
Copy full SHA for f337428 - Browse repository at this point
Copy the full SHA f337428View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4654b9f - Browse repository at this point
Copy the full SHA 4654b9fView commit details -
Configuration menu - View commit details
-
Copy full SHA for e39926e - Browse repository at this point
Copy the full SHA e39926eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1c6eefd - Browse repository at this point
Copy the full SHA 1c6eefdView commit details -
Configuration menu - View commit details
-
Copy full SHA for f2d2ee5 - Browse repository at this point
Copy the full SHA f2d2ee5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 302955c - Browse repository at this point
Copy the full SHA 302955cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3d4754e - Browse repository at this point
Copy the full SHA 3d4754eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 620b224 - Browse repository at this point
Copy the full SHA 620b224View commit details -
Configuration menu - View commit details
-
Copy full SHA for b245d3c - Browse repository at this point
Copy the full SHA b245d3cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1e71e98 - Browse repository at this point
Copy the full SHA 1e71e98View commit details -
Configuration menu - View commit details
-
Copy full SHA for a01c00d - Browse repository at this point
Copy the full SHA a01c00dView commit details
Commits on Jun 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for debffc2 - Browse repository at this point
Copy the full SHA debffc2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 39fe67f - Browse repository at this point
Copy the full SHA 39fe67fView commit details -
Configuration menu - View commit details
-
Copy full SHA for af1b0be - Browse repository at this point
Copy the full SHA af1b0beView commit details -
Configuration menu - View commit details
-
Copy full SHA for 834c6e0 - Browse repository at this point
Copy the full SHA 834c6e0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5bc2bc3 - Browse repository at this point
Copy the full SHA 5bc2bc3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8740369 - Browse repository at this point
Copy the full SHA 8740369View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4d82ca1 - Browse repository at this point
Copy the full SHA 4d82ca1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7bf831c - Browse repository at this point
Copy the full SHA 7bf831cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3fccc76 - Browse repository at this point
Copy the full SHA 3fccc76View commit details
Commits on Jun 25, 2024
-
Configuration menu - View commit details
-
Copy full SHA for e8d0e93 - Browse repository at this point
Copy the full SHA e8d0e93View commit details -
Configuration menu - View commit details
-
Copy full SHA for 91c2e43 - Browse repository at this point
Copy the full SHA 91c2e43View commit details -
Configuration menu - View commit details
-
Copy full SHA for fac7e68 - Browse repository at this point
Copy the full SHA fac7e68View commit details -
Configuration menu - View commit details
-
Copy full SHA for 271822e - Browse repository at this point
Copy the full SHA 271822eView commit details -
Configuration menu - View commit details
-
Copy full SHA for ae0d7f1 - Browse repository at this point
Copy the full SHA ae0d7f1View commit details -
Configuration menu - View commit details
-
Copy full SHA for b84a070 - Browse repository at this point
Copy the full SHA b84a070View commit details -
Configuration menu - View commit details
-
Copy full SHA for 86fda24 - Browse repository at this point
Copy the full SHA 86fda24View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.