[Snippets] Brgemm blocking by KN dims at the LIR level #19335

v-Golubev · 2023-08-22T09:40:27Z

Details:

Brgemm KN blocking removed from BrgemmCPU emitter
Brgemm blocking implemented on Linear IR level
InsertTailLoop rework: added a mechanism for subtensor propagation via shape inference infrastructure
dim_idx moved from LoopInfo to LoopPort
Several fixes

Tickets:

115164
112122

Prerequisites:

[Snippets] Changed BrgemmCopyB shape inference [Snippets] Changed BrgemmCopyB shape inference #19957
[Snippets] Deep copy interface [Snippets] Deep copy interface #20242
[Snippets] Refactored Buffer memory allocation [Snippets] Refactored Buffer memory allocation #19644

src/common/snippets/src/op/subgraph.cpp

src/common/snippets/src/lowered/pass/insert_tail_loop.cpp

src/plugins/intel_cpu/src/transformations/snippets/x64/pass/lowered/brgemm_blocking.cpp

v-Golubev · 2023-11-22T09:22:14Z

@IvanNovoselov @a-sidorova as we agreed offline, I also separated snippets operation Buffer on 2 ones. Could you please take a look on the 2 latest commits (AMX scratchpad buffer is moved inside blocking loops and Buffer operation separation)? Thanks in advance

src/common/snippets/src/op/buffer.cpp

v-Golubev · 2023-11-23T09:56:05Z

It was decided to merge the PR after #19644

v-Golubev · 2023-11-30T14:24:25Z

@dmitry-gorokhov could you please take a look?

dmitry-gorokhov

No major concerns
@v-Golubev Please answer remaining questions

dmitry-gorokhov · 2023-12-08T05:44:57Z

src/common/snippets/include/snippets/op/loop.hpp

- * @param ptr_increments specifies i/o pointer increment performed on every iteration. This is an alternative to
- * apply_increments, which enables more flexibility.
+ * @param is_incremented describes which data pointers attributed to the loop should be incremented on every iteration.
+ * @param ptr_increments specifies i/o pointer increment performed on every iteration if the following is_incremented[i] is true


Just wondering why is_incremented[i] cannot be expressed with ptr_increments[i] == 0;

Actually it can be expressed. Need to revert this change - I will do that in follow up PRs

I think that it might be useful for dynamic loops (for dynamic loop end emitter) when you need this port as LoopPort but it shouldn't be incremented.

I think that it might be useful for dynamic loops (for dynamic loop end emitter) when you need this port as LoopPort but it shouldn't be incremented.

Lets refactor in bounds of DynamicLoop enablement then.

dmitry-gorokhov · 2023-12-08T07:27:37Z

src/plugins/intel_cpu/src/transformations/snippets/x64/op/brgemm_cpu.hpp

@@ -32,19 +32,19 @@ class BrgemmCPU : public snippets::op::Brgemm {
    BrgemmCPU(const Output<Node>& A, const Output<Node>& B, const Type type,
              const size_t offset_a = 0, const size_t offset_b = 0, const size_t offset_c = 0,
              std::vector<size_t> layout_a = {}, std::vector<size_t> layout_b = {}, std::vector<size_t> layout_c = {},
-              const size_t blk_size_m = 0, const size_t blk_size_k = 0, const size_t blk_size_n = 0);
+              const size_t blk_size_m = 0, const size_t blk_size_k = 0, const size_t blk_size_n = 0, const float beta = 0.f);


I haven't analyzed the code in details, but was is the purpose of having blk size as part of Brgemm semantics?
It sounds like everything should be done with transformations now. As far as I remember blocking heuristics are also expressed with separate transformation.

blk size parameters are used to propagate information about preferred block sizes between ov::Model transformation (which applies blocking heuristics) and LIR blocking markup transformation.
We have already discussed this topic with @IvanNovoselov: we came to the conclusion that these 2 transformations can be easily combined into one: in this case, there will be no need to store block sizes as node fields.

…it#19335)

### Details: - *Introduced loop handlers which define how to handle first/last loop iteration* - *Changed loop markup logic: loops with `increment > work_amount` are not created anymore: in this case `increment` is set equal to `work_amount`* ### Tickets: - *CVS-119851* ### Prerequisites: - #19335 - #21382

github-actions bot added category: IE Tests OpenVINO Test: plugins and common category: CPU OpenVINO CPU plugin labels Aug 22, 2023

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch 2 times, most recently from 3ac018c to 20b2a91 Compare August 24, 2023 15:39

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch 2 times, most recently from 73ae818 to ef6b38b Compare September 4, 2023 15:44

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch from ef6b38b to 11a2a01 Compare September 11, 2023 17:05

github-actions bot removed the category: IE Tests OpenVINO Test: plugins and common label Sep 11, 2023

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch 2 times, most recently from cf8a1de to 9defd95 Compare September 12, 2023 14:20

v-Golubev added the do_not_review label Sep 12, 2023

v-Golubev marked this pull request as ready for review September 12, 2023 17:27

v-Golubev requested review from a team as code owners September 12, 2023 17:27

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch from 9defd95 to 3f554be Compare September 12, 2023 17:27

v-Golubev commented Sep 15, 2023

View reviewed changes

src/common/snippets/src/op/subgraph.cpp Show resolved Hide resolved

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch 2 times, most recently from 3e052a2 to d027a05 Compare September 29, 2023 12:24

v-Golubev requested review from a team as code owners September 29, 2023 12:24

github-actions bot added the category: IE Tests OpenVINO Test: plugins and common label Sep 29, 2023

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch 4 times, most recently from 51a567a to 85c7b3c Compare October 4, 2023 10:57

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch from d7199f7 to 6b0346a Compare October 9, 2023 07:42

v-Golubev removed the do_not_review label Oct 9, 2023

v-Golubev assigned a-sidorova and IvanNovoselov Oct 9, 2023

v-Golubev added this to the 2023.2 milestone Oct 9, 2023

IvanNovoselov approved these changes Nov 17, 2023

View reviewed changes

src/common/snippets/src/lowered/pass/insert_tail_loop.cpp Outdated Show resolved Hide resolved

a-sidorova reviewed Nov 21, 2023

View reviewed changes

src/plugins/intel_cpu/src/transformations/snippets/x64/pass/lowered/brgemm_blocking.cpp Show resolved Hide resolved

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch from 2c7622b to 0fda262 Compare November 21, 2023 17:29

v-Golubev commented Nov 22, 2023

View reviewed changes

src/common/snippets/src/op/buffer.cpp Outdated Show resolved Hide resolved

a-sidorova approved these changes Nov 22, 2023

View reviewed changes

IvanNovoselov approved these changes Nov 23, 2023

View reviewed changes

v-Golubev mentioned this pull request Nov 27, 2023

[Snippets] Specific loop iterations handler #21303

Merged

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch from 7fcdce2 to aedbdf3 Compare November 30, 2023 14:23

v-Golubev assigned dmitry-gorokhov and unassigned a-sidorova and IvanNovoselov Nov 30, 2023

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch 3 times, most recently from f731831 to 8dfb8e8 Compare December 1, 2023 16:26

v-Golubev mentioned this pull request Dec 1, 2023

[Snippets] Added PassConfig and PositionedPasses for PassPipeline #21382

Merged

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch from 8dfb8e8 to 68b48a0 Compare December 5, 2023 09:10

a-sidorova mentioned this pull request Dec 5, 2023

[Snippets] Fixed IdentifyBuffers for nested loop case #21470

Merged

v-Golubev added 5 commits December 5, 2023 13:20

[Snippets] Brgemm blocking moved from emitter to LIR

3390716

Alexandra's comments applied

372757e

Ivan's comments applied

1d256ec

Buffer operation separation

01f4a1a

Post rebase fixes

2699590

v-Golubev force-pushed the vg/snippets/mha/lir_kn_blocking branch from 68b48a0 to 2699590 Compare December 5, 2023 13:11

dmitry-gorokhov approved these changes Dec 8, 2023

View reviewed changes

dmitry-gorokhov merged commit 261cf4e into openvinotoolkit:master Dec 8, 2023
73 checks passed

akuporos pushed a commit to akuporos/openvino that referenced this pull request Dec 8, 2023

[Snippets] Brgemm blocking by KN dims at the LIR level (openvinotoolk…

a1e1924

…it#19335)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Snippets] Brgemm blocking by KN dims at the LIR level #19335

[Snippets] Brgemm blocking by KN dims at the LIR level #19335

v-Golubev commented Aug 22, 2023 •

edited by mg-intel

Loading

v-Golubev commented Nov 22, 2023

v-Golubev commented Nov 23, 2023

v-Golubev commented Nov 30, 2023

dmitry-gorokhov left a comment •

edited

Loading

dmitry-gorokhov Dec 8, 2023

v-Golubev Dec 8, 2023

a-sidorova Dec 8, 2023 •

edited

Loading

dmitry-gorokhov Dec 8, 2023

dmitry-gorokhov Dec 8, 2023

v-Golubev Dec 8, 2023

[Snippets] Brgemm blocking by KN dims at the LIR level #19335

[Snippets] Brgemm blocking by KN dims at the LIR level #19335

Conversation

v-Golubev commented Aug 22, 2023 • edited by mg-intel Loading

Details:

Tickets:

Prerequisites:

v-Golubev commented Nov 22, 2023

v-Golubev commented Nov 23, 2023

v-Golubev commented Nov 30, 2023

dmitry-gorokhov left a comment • edited Loading

Choose a reason for hiding this comment

dmitry-gorokhov Dec 8, 2023

Choose a reason for hiding this comment

v-Golubev Dec 8, 2023

Choose a reason for hiding this comment

a-sidorova Dec 8, 2023 • edited Loading

Choose a reason for hiding this comment

dmitry-gorokhov Dec 8, 2023

Choose a reason for hiding this comment

dmitry-gorokhov Dec 8, 2023

Choose a reason for hiding this comment

v-Golubev Dec 8, 2023

Choose a reason for hiding this comment

v-Golubev commented Aug 22, 2023 •

edited by mg-intel

Loading

dmitry-gorokhov left a comment •

edited

Loading

a-sidorova Dec 8, 2023 •

edited

Loading