feat: rewriting param to a Constant if it's a introduced input #1298
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Bo Wang <[email protected]>
|
@bowang007 seems like linting is failing, can you set up the pre-commit system? |
narendasan
reviewed
Aug 23, 2022
narendasan
reviewed
Aug 23, 2022
core/compiler.cpp
Outdated
| @@ -434,6 +451,9 @@ torch::jit::Module CompileGraph(const torch::jit::Module& mod, CompileSpec cfg) | |||
| (!(cfg.lower_info.forced_fallback_modules.size() == 0 && | |||
| cfg.partition_info.forced_fallback_operators.size() == 0 && isBlockConvertible) || | |||
| outputIsCollection)) { | |||
| if (!static_params.empty()) { | |||
| RewriteInputsWithParams(g, params); | |||
There was a problem hiding this comment.
Should this be a lowering pass?
There was a problem hiding this comment.
It could be a lowering pass. However, I'm not so sure about what happens with conversion stage if the input are stored in params. Any thought on this? @narendasan
There was a problem hiding this comment.
At this point static params is usually empty, it is just legacy behavior we still exploit for test so it should be able to handle it.
There was a problem hiding this comment.
cool let me change it to a lowering pass.
Signed-off-by: Bo Wang <[email protected]>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
There are some changes that do not conform to C++ style guidelines:
diff --git a/home/runner/work/TensorRT/TensorRT/core/conversion/converters/impl/select.cpp b/tmp/changes.txt
index 20a03f6..6c8f5dd 100644
--- a/home/runner/work/TensorRT/TensorRT/core/conversion/converters/impl/select.cpp
+++ b/tmp/changes.txt
@@ -484,7 +484,7 @@ auto select_registrations TORCHTRT_UNUSED =
auto layer = ctx->net->addScatter(*self, *index, *value_tensor, nvinfer1::ScatterMode::kELEMENT);
layer->setAxis(dim);
-
+
TORCHTRT_CHECK(layer, "Unable to create layer for aten::scatter.value");
layer->setName(util::node_info(n).c_str());
@@ -503,7 +503,7 @@ auto select_registrations TORCHTRT_UNUSED =
auto layer = ctx->net->addScatter(*self, *index, *src, nvinfer1::ScatterMode::kELEMENT);
layer->setAxis(dim);
-
+
TORCHTRT_CHECK(layer, "Unable to create layer for aten::scatter.src");
layer->setName(util::node_info(n).c_str());
diff --git a/home/runner/work/TensorRT/TensorRT/core/conversion/converters/impl/element_wise.cpp b/tmp/changes.txt
index 0ad4c12..32c7050 100644
--- a/home/runner/work/TensorRT/TensorRT/core/conversion/converters/impl/element_wise.cpp
+++ b/tmp/changes.txt
@@ -25,8 +25,6 @@ nvinfer1::ITensor* clamp_util(
return clamp_layer_out;
}
-
-
auto element_wise_registrations TORCHTRT_UNUSED =
RegisterNodeConversionPatterns()
.pattern(
ERROR: Some files do not conform to style guidelinesSigned-off-by: Bo Wang <[email protected]>
github-actions
bot
added
the
component: lowering
There was a problem hiding this comment.
There are some changes that do not conform to C++ style guidelines:
diff --git a/home/runner/work/TensorRT/TensorRT/core/lowering/passes/rewrite_inputs_with_params.cpp b/tmp/changes.txt
index a05a905..c06fcd6 100644
--- a/home/runner/work/TensorRT/TensorRT/core/lowering/passes/rewrite_inputs_with_params.cpp
+++ b/tmp/changes.txt
@@ -1,13 +1,11 @@
-#include "torch/csrc/jit/ir/constants.h"
#include "core/util/prelude.h"
-
+#include "torch/csrc/jit/ir/constants.h"
namespace torch_tensorrt {
namespace core {
namespace lowering {
namespace passes {
-
void RewriteInputsWithParams(std::shared_ptr<torch::jit::Graph>& g, std::vector<torch::jit::IValue>& params) {
auto input_size = g->inputs().size();
auto param_it = params.rbegin();
diff --git a/home/runner/work/TensorRT/TensorRT/core/conversion/converters/impl/select.cpp b/tmp/changes.txt
index 20a03f6..6c8f5dd 100644
--- a/home/runner/work/TensorRT/TensorRT/core/conversion/converters/impl/select.cpp
+++ b/tmp/changes.txt
@@ -484,7 +484,7 @@ auto select_registrations TORCHTRT_UNUSED =
auto layer = ctx->net->addScatter(*self, *index, *value_tensor, nvinfer1::ScatterMode::kELEMENT);
layer->setAxis(dim);
-
+
TORCHTRT_CHECK(layer, "Unable to create layer for aten::scatter.value");
layer->setName(util::node_info(n).c_str());
@@ -503,7 +503,7 @@ auto select_registrations TORCHTRT_UNUSED =
auto layer = ctx->net->addScatter(*self, *index, *src, nvinfer1::ScatterMode::kELEMENT);
layer->setAxis(dim);
-
+
TORCHTRT_CHECK(layer, "Unable to create layer for aten::scatter.src");
layer->setName(util::node_info(n).c_str());
diff --git a/home/runner/work/TensorRT/TensorRT/core/conversion/converters/impl/element_wise.cpp b/tmp/changes.txt
index 0ad4c12..32c7050 100644
--- a/home/runner/work/TensorRT/TensorRT/core/conversion/converters/impl/element_wise.cpp
+++ b/tmp/changes.txt
@@ -25,8 +25,6 @@ nvinfer1::ITensor* clamp_util(
return clamp_layer_out;
}
-
-
auto element_wise_registrations TORCHTRT_UNUSED =
RegisterNodeConversionPatterns()
.pattern(
ERROR: Some files do not conform to style guidelinesSigned-off-by: Bo Wang <[email protected]>
There was a problem hiding this comment.
There are some changes that do not conform to C++ style guidelines:
diff --git a/home/runner/work/TensorRT/TensorRT/core/lowering/passes/rewrite_inputs_with_params.cpp b/tmp/changes.txt
index a05a905..c06fcd6 100644
--- a/home/runner/work/TensorRT/TensorRT/core/lowering/passes/rewrite_inputs_with_params.cpp
+++ b/tmp/changes.txt
@@ -1,13 +1,11 @@
-#include "torch/csrc/jit/ir/constants.h"
#include "core/util/prelude.h"
-
+#include "torch/csrc/jit/ir/constants.h"
namespace torch_tensorrt {
namespace core {
namespace lowering {
namespace passes {
-
void RewriteInputsWithParams(std::shared_ptr<torch::jit::Graph>& g, std::vector<torch::jit::IValue>& params) {
auto input_size = g->inputs().size();
auto param_it = params.rbegin();
ERROR: Some files do not conform to style guidelinesSigned-off-by: Bo Wang <[email protected]>
There was a problem hiding this comment.
There are some changes that do not conform to C++ style guidelines:
diff --git a/home/runner/work/TensorRT/TensorRT/tests/core/lowering/test_rewrite_inputs_with_params.cpp b/tmp/changes.txt
index 2f0341c..b0d8045 100644
--- a/home/runner/work/TensorRT/TensorRT/tests/core/lowering/test_rewrite_inputs_with_params.cpp
+++ b/tmp/changes.txt
@@ -25,7 +25,7 @@ TEST(LoweringPasses, RewriteInputsWithParamsCorrectly) {
torch::jit::IValue param0 = torch::jit::IValue(0);
std::vector<torch::jit::IValue> params{param0};
torch_tensorrt::core::lowering::passes::RewriteInputsWithParams(sg, params);
-
+
auto tg = std::make_shared<torch::jit::Graph>();
torch::jit::parseIR(target_graph, &*tg);
ERROR: Some files do not conform to style guidelines
There was a problem hiding this comment.
There are some changes that do not conform to C++ style guidelines:
diff --git a/home/runner/work/TensorRT/TensorRT/tests/core/lowering/test_rewrite_inputs_with_params.cpp b/tmp/changes.txt
index 2f0341c..b0d8045 100644
--- a/home/runner/work/TensorRT/TensorRT/tests/core/lowering/test_rewrite_inputs_with_params.cpp
+++ b/tmp/changes.txt
@@ -25,7 +25,7 @@ TEST(LoweringPasses, RewriteInputsWithParamsCorrectly) {
torch::jit::IValue param0 = torch::jit::IValue(0);
std::vector<torch::jit::IValue> params{param0};
torch_tensorrt::core::lowering::passes::RewriteInputsWithParams(sg, params);
-
+
auto tg = std::make_shared<torch::jit::Graph>();
torch::jit::parseIR(target_graph, &*tg);
ERROR: Some files do not conform to style guidelinesSigned-off-by: Bo Wang <[email protected]>
|
This PR should be good to merge. @narendasan @peri044 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Bo Wang [email protected]
Description
In some cases, some parameters are graphs' input. We can rewrite them into constants to make sure it works.
Fixes #1190
Type of change
Checklist: