Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[§] Generic eltwise ported to nGraph #2824

Closed
wants to merge 65 commits into from
Closed

[§] Generic eltwise ported to nGraph #2824

wants to merge 65 commits into from

Conversation

geexie
Copy link
Contributor

@geexie geexie commented Oct 26, 2020 •
edited
Loading

I would like to propose yet another step for eltwise operator generalization: possibility to fuse forks in eltwise chains with snippets.

Design is described here

Functionality

  • starts from multi-output layout-oblivious operation to handle GELU-like patterns
  • ISA support: AVX2
  • Precisions support: FP32
  • "Inplace" support
  • Layouts support
    • ncsp(x,nc,ncw,nchw,ncdhw, …etc)
    • nspc(nwc,nhwc,ndhwc, ... etc)
    • nCspXc(nCwXc, nChwXc, nCdhwXc, …etc) for X = {8, 16}
  • Only equal layouts on input and outputs

Requirements

  • FP32 models
    • DLB scope – no regressions (functional, performance)
    • OMZ scope – no regressions (functional, accuracy)
    • VDP scope – no regressions (functional, accuracy, performance)
  • Operations coverage – no regressions
  • First inference time – no regressions
  • Binary size – no regressions

@geexie geexie added category: CPU OpenVINO CPU plugin optimization labels Oct 26, 2020
@geexie geexie requested review from a team October 26, 2020 12:14
@geexie geexie self-assigned this Oct 26, 2020
@openvino-pushbot openvino-pushbot added category: inference OpenVINO Runtime library - Inference category: IE Tests OpenVINO Test: plugins and common category: Core OpenVINO Core (aka ngraph) labels Oct 26, 2020
@rblaczkowski
Copy link
Contributor

Please update to the newest master to fetch the newest CI state - currently the branch is a few months before current master

@geexie geexie requested a review from a team October 27, 2020 09:54
@geexie geexie added this to the 2021.3 milestone Dec 2, 2020
@geexie geexie changed the title [Draft] Generic eltwise ported to ngraph Generic eltwise ported to nGraph. Part #1 Dec 2, 2020
@geexie
Copy link
Contributor Author

geexie commented Dec 2, 2020

@slyalin, @GlebKazantaev, @dmitry-gorokhov, please share your suggestions about this PR, interfaces and everything. Please advice me on formal PR flow, what builds should I run etc

@geexie geexie changed the title Generic eltwise ported to nGraph. Part #1 Generic eltwise ported to nGraph Jan 12, 2021
@geexie geexie removed request for a team January 13, 2021 17:52
@geexie
Copy link
Contributor Author

geexie commented Jan 13, 2021

@dmitry-gorokhov, @slyalin, @GlebKazantaev, @ilyachur, could you please review this PR.

dmitry-gorokhov
dmitry-gorokhov reviewed Jan 14, 2021
View reviewed changes
inference-engine/src/mkldnn_plugin/nodes/subgraph.cpp Outdated Show resolved Hide resolved
inference-engine/src/mkldnn_plugin/nodes/subgraph.cpp Outdated Show resolved Hide resolved
inference-engine/src/mkldnn_plugin/nodes/subgraph.cpp Outdated Show resolved Hide resolved
inference-engine/src/mkldnn_plugin/mkldnn_plugin.cpp Outdated Show resolved Hide resolved
inference-engine/src/mkldnn_plugin/mkldnn_graph_optimizer.cpp Outdated Show resolved Hide resolved
inference-engine/src/mkldnn_plugin/emitters/cpu_generator.cpp Outdated Show resolved Hide resolved
inference-engine/src/mkldnn_plugin/emitters/jit_mkldnn_emitters.hpp Outdated Show resolved Hide resolved
inference-engine/src/mkldnn_plugin/mkldnn_graph_optimizer.cpp Outdated Show resolved Hide resolved
inference-engine/src/mkldnn_plugin/nodes/subgraph.cpp Outdated Show resolved Hide resolved
inference-engine/src/transformations/src/ngraph_ops/subgraph.cpp Outdated Show resolved Hide resolved
ilyachur
ilyachur reviewed Jan 25, 2021
View reviewed changes
inference-engine/src/legacy_api/src/convert_function_to_cnn_network.cpp Outdated Show resolved Hide resolved
inference-engine/src/mkldnn_plugin/emitters/emitter.hpp Outdated Show resolved Hide resolved
inference-engine/src/mkldnn_plugin/emitters/emitter.hpp Outdated Show resolved Hide resolved
inference-engine/src/transformations/include/ngraph_ops/broadcastload.hpp Outdated Show resolved Hide resolved
inference-engine/src/transformations/include/ngraph_ops/broadcastload.hpp Outdated Show resolved Hide resolved
inference-engine/src/transformations/include/transformations/snippets/generator.hpp Outdated Show resolved Hide resolved
inference-engine/src/transformations/src/ngraph_ops/broadcastload.cpp Outdated Show resolved Hide resolved
inference-engine/src/transformations/src/transformations/rt_info/register_info.cpp Outdated Show resolved Hide resolved
inference-engine/src/transformations/src/transformations/snippets/assign_registers_pass.cpp Outdated Show resolved Hide resolved
...e-engine/src/transformations/include/transformations/snippets/insert_explicit_loads_pass.hpp Outdated Show resolved Hide resolved
IvanNovoselov and others added 28 commits October 6, 2021 16:54
@IvanNovoselov
[§] Fixed segfault for 1-elem consts (smoke_FakeQuantizeNoneBroadcast)
582b79b
@IvanNovoselov
[§] Fixed CompareWithRefs_MVN* by adding MVN to fusable parents in f…
f28631b 
…ilter_fused
@IvanNovoselov
[§] Fixed *NormalizeL2LayerCPUTest* by adding NormalizeL2 to filter_f…
cdbabf0 
…used
@IvanNovoselov
[§] Added FullyConnected to filter_fused
3e00fac
@IvanNovoselov
[§] Fixed several cpuFuncTests by adding approproate ops to filter_fused
8885372
@a-sidorova @IvanNovoselov
[§] fixed android build
d931d34
@a-sidorova @IvanNovoselov
[§] added subgraph support in CanProcessDynBatch
bccd376
@a-sidorova @IvanNovoselov
[§] fixed x32 systems
5ea5bd8
@IvanNovoselov
[§] Align as_type_ptr<> with changes on master
ee2b049
@IvanNovoselov
[§] Enable DUMP_TOKENIZATION to xml on demand
4badd45
@IvanNovoselov
[§] Fixed convert power->PowerStatic in canonicalize()
11dee3e
@a-sidorova @IvanNovoselov
[§] fixed registers conflict on Win systems
0dc0c3d
@IvanNovoselov
[§] Avoid redundant push-pop for scalar emitters
10c9850
@IvanNovoselov
[§] Align with {[CPU] Dynamic shapes support using fallback on refere…
61d998c 
…nce}

hash: 5096fe1
@IvanNovoselov
[§] FilterFused now accounts for fusing through MaxPool
ea8b788
@IvanNovoselov
[§] Support MVN:6. Change to is_type. Avoid single-node subgraphs.
bc3ed59
@IvanNovoselov
[§] Enable snippets before Result nodes
3169408
@IvanNovoselov
[§] Allow single-node subgraphs (Considered to be a workaround).
e4443bf
@IvanNovoselov
[§] Bring back postCommonPassManager.register_pass<ReshapePRelu>() t…
c2af4da 
…o fix CI after rebase
@IvanNovoselov
[§] Clear control_dependencies when creating a body_function
40340e0
@IvanNovoselov
[§] Update isSuitableMiscParent to account for new fusing parents.
260848b
@IvanNovoselov
[§] Remove the op evaluate() method.
a725c74
@IvanNovoselov
[§] Restore old comparator for jitters, since the new one is not tran…
33ea7b9 
…sitive
@IvanNovoselov
[§] Commented SnippetsTests, need to add Snippets to evaluates_map
c24fb8c
@IvanNovoselov
[§] Disable snippets if BF16 is enabled
e0bd832
@IvanNovoselov
[§] Fix segfault in LoadMoveBroadcastToBroadcastLoad
af946fe
@IvanNovoselov
[§] Implemented HSwish support
960787c
@IvanNovoselov
[§] Fine tuning of filter_fused to better mimic FusingConvolutionWith…
b3a1ac2 
…SumActivation
@geexie geexie closed this Oct 11, 2021
@geexie geexie deleted the generic_eltwise_ported_to_ngraph branch October 11, 2021 09:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ilyachur ilyachur ilyachur approved these changes

@ilya-lavrenov ilya-lavrenov ilya-lavrenov approved these changes

@dmitry-gorokhov dmitry-gorokhov Awaiting requested review from dmitry-gorokhov

Assignees

@geexie geexie

Labels
category: CPU OpenVINO CPU plugin
Projects
None yet
Milestone
2022.1
Development

Successfully merging this pull request may close these issues.
8 participants
@geexie @rblaczkowski @ilya-lavrenov @dmitry-gorokhov @ilyachur @openvino-pushbot @IvanNovoselov @a-sidorova