Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forward input dims in Reshape ShapeHelper #2828

Merged
merged 8 commits into from
Jun 12, 2024

Conversation

tungld
Copy link
Collaborator

@tungld tungld commented May 15, 2024

In this reshape op in a quantized bert model, the first two dimensions of the output are the same as the ones in the input, so we would like to forward IndexExprs of these dimensions from the input to the output so that DimAnalysis works. But it failed because of using IndexExpr.selectOrSelf.

%184 = onnx.Constant dense<[0, 0, 12, 64]> : tensor<4xi64>
%494 = "onnx.Reshape"(%491, %184) {allowzero = 0 : si64, onnx_node_name = "/bert/encoder/layer.0/attention/self/Reshape"} : (tensor<?x?x768xf32>, tensor<4xi64>) -> tensor<?x?x12x64xf32>

This patch updates selectOrSelf to forward the input QuestionMark when the condition is literal.

@tungld
Copy link
Collaborator Author

tungld commented May 15, 2024

@AlexandreEichenberger if you know how to modify selectOrSelf to fix this issue, please let me know or please feel free to create a PR for that. Thanks!

Signed-off-by: Tung D. Le <[email protected]>
outputDims[i] = outputDims[i].selectOrSelf(
outputDims[i] == -1, numOfElements.floorDiv(numOfElementsFromShape));
dimShape == -1, numOfElements.floorDiv(numOfElementsFromShape));
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed this to make the compare expression literal.

@tungld
Copy link
Collaborator Author

tungld commented May 15, 2024

It looks like I found the way to change selectOrSelf but it only works if the condition is literal at compile time.

@AlexandreEichenberger
Copy link
Collaborator

@tungld is there still a case to look into?

@tungld
Copy link
Collaborator Author

tungld commented Jun 5, 2024

@tungld is there still a case to look into?

I changed selectOrSelf to handle the case where the comparison is literal. It works for the quantized bert model. I hope it's enough for most cases.

@tungld
Copy link
Collaborator Author

tungld commented Jun 5, 2024

@AlexandreEichenberger It's ready for review.

Copy link
Collaborator

@AlexandreEichenberger AlexandreEichenberger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, make sense. Not 100% sure why I added a deepCopy at that time, it makes sense not to generate a new question mark when we have questions marks.

@AlexandreEichenberger AlexandreEichenberger merged commit 713fc2e into onnx:main Jun 12, 2024
7 checks passed
@jenkins-droid
Copy link
Collaborator

Jenkins Linux amd64 Build #14969 [push] Forward input dims in Re... started at 10:54

@jenkins-droid
Copy link
Collaborator

Jenkins Linux s390x Build #14974 [push] Forward input dims in Re... started at 11:54

@jenkins-droid
Copy link
Collaborator

Jenkins Linux ppc64le Build #13999 [push] Forward input dims in Re... started at 12:03

@jenkins-droid
Copy link
Collaborator

Jenkins Linux amd64 Build #14969 [push] Forward input dims in Re... passed after 1 hr 7 min

@jenkins-droid
Copy link
Collaborator

Jenkins Linux s390x Build #14974 [push] Forward input dims in Re... passed after 1 hr 31 min

@jenkins-droid
Copy link
Collaborator

Jenkins Linux ppc64le Build #13999 [push] Forward input dims in Re... passed after 1 hr 59 min

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants