Add missing arguments in pipeline parallel generate method #12142
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add two arguments:
negative_prompt_ids
andnegative_prompt_attention_mask
to thegenerate
method inpipeline_parallel.py
. These two arguments are available since transformers 4.32.0.1. Why the change?
Hello, I'm using both OpenVINO and ipex-llm in one project, and found the
generate
method in ipex-llm'spipeline-parallel.py
has two arguments less than thegenerate
method in transformers, causing a weird error in an OpenVINO-only code path:With this PR, the error is gone.
2. User API changes
No change
3. Summary of the change
Add the two missing arguments to the
generate
method inpipeline_parallel.py
4. How to test?
Install
optimum[openvino]==1.22.0
, then use the following script to reproduce and check:This does not happen on
optimum[openvino]<=1.20.0
.