[Layer] add "add layer" #2710
[Layer] add "add layer" #2710
Conversation
baek2sm
requested review from
myungjoo,
jijoongmoon,
again4you,
jaeyun-jung,
leemgs,
wooksong,
helloahn,
kparichay,
gichan-jang,
anyj0527,
zhoonit,
lhs8928,
songgot,
jihochu,
DonghakPark,
SeoHyungjun,
skykongkong8,
djeong20,
EunjuYang and
a team
as code owners
|
📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2710. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/. |
baek2sm
force-pushed
the
n_op_add_layer
branch
from
28f887a
to
d116647
Compare
nntrainer/layers/add_layer.cpp
Outdated
| Tensor &hidden_ = context.getOutput(SINGLE_INOUT_IDX); | ||
|
|
||
| const Tensor &input0 = context.getInput(0); | ||
| const Tensor &input1 = context.getInput(1); |
There was a problem hiding this comment.
Does add layer only has 2 inputs?
Is there any possibility of having more than 2 inputs?
There was a problem hiding this comment.
@lhs8928 Good point! The addition layer, to be precise, implements a summation operation. This layer corresponds to four basic operations that take two inputs.
nntrainer/layers/add_layer.cpp
Outdated
| const Tensor &input0 = context.getInput(0); | ||
| const Tensor &input1 = context.getInput(1); | ||
|
|
||
| TensorDim input_dim = input0.getDim(); | ||
| TensorDim input_step_dim = input_dim; | ||
| input_step_dim.batch(1); | ||
| input_step_dim.height(to - from); |
There was a problem hiding this comment.
Do we need to add this code inside the batch loop?
What about moving the codes to the outside of the loop ?
baek2sm
force-pushed
the
n_op_add_layer
branch
from
d16c4bb
to
4eb6d12
Compare
cibot:
|
|
baek2sm
force-pushed
the
n_op_add_layer
branch
from
4eb6d12
to
cc20fb9
Compare
|
|
baek2sm
force-pushed
the
n_op_add_layer
branch
from
cc20fb9
to
6d958ff
Compare
|
|
baek2sm
force-pushed
the
n_op_add_layer
branch
from
6d958ff
to
3685021
Compare
baek2sm
force-pushed
the
n_op_add_layer
branch
3 times, most recently
from
d9416db
to
8600446
Compare
nntrainer/layers/operation_layer.h
Outdated
| * @brief Type of layer depending on the number of inputs. | ||
| * | ||
| */ | ||
| enum class OperationType { NONE, UNARY, BINARY }; |
There was a problem hiding this comment.
It is important to consider whether the operation can be performed in-place to save memory. I guess it might be good to give the user an option to set it when the operation layer is created.
There was a problem hiding this comment.
A new pull request will be uploaded concerning inplace operations. This implementation has been updated based on the UnaryOperationLayer and BinaryOperationLayer classes! Thanks
baek2sm
force-pushed
the
n_op_add_layer
branch
from
8600446
to
8c108e6
Compare
|
|
- added "add layer" - added a "model unit test" for add layer. Since 'model-level unit test' havn't been run for a long time, I diabled some test cases that causing issues when running model unit test. Many people gave great feedback, so I've improved the structure accordingly. - An upper class called "OperationLayer" was added to reduce redundant code. - Based on the number of input tensors, the behavior of "OperationLayer" has been classified into two types: unary and binary operations. - Various additional code cleanups have also taken place. **Self evaluation:** 1. Build test: [X]Passed [X]Failed [ ]Skipped 2. Run test: [X]Passed [X]Failed [ ]Skipped Signed-off-by: Seungbaek Hong <[email protected]>
baek2sm
force-pushed
the
n_op_add_layer
branch
from
8c108e6
to
f7e2646
Compare
baek2sm
changed the title
baek2sm
changed the title
baek2sm
changed the title
havn't been run for a long time, I diabled some test cases that causing
issues when running model unit test.
Many people gave great feedback, so I've improved the structure accordingly.
An upper class called "OperationLayer" was added to reduce redundant code.
Based on the number of input tensors, the behavior of "OperationLayer" has been classified into two types: unary and binary operations.
Various additional code cleanups have also taken place.
there is an issue where committing compressed files containing golden data for unit test prevents pushing changes to the remote server. (I've confirmed that all unit tests pass locally using that golden data.)
Self evaluation:
Signed-off-by: Seungbaek Hong [email protected]