Code

[CPU]whisper readvalue optimize#26130

Code

Merged

dmitry-gorokhov merged 166 commits intoopenvinotoolkit:masteropenvinotoolkit/openvino:masterfrom xipingyan:xp/whisper_readvalue_optimizexipingyan/openvino:xp/whisper_readvalue_optimizeCopy head branch name to clipboardDec 24, 2024

+1,165-50

Commits

Commits on Aug 19, 2024

• Add profiler for CPU plugin.

  Show description for 2916414
  xipingyan

  committed
  2916414

Commits on Aug 20, 2024

• Mark ReadValue's inputs and corresponding Assign.

  Show description for 451c76d
  xipingyan

  committed
  451c76d

• Only mark: ReadValue->Assign pairs.

  Show description for 137beee
  xipingyan

  committed
  137beee

Commits on Aug 21, 2024

• Optimize pattern match.

  Show description for 737fe5c
  xipingyan

  committed
  737fe5c

Commits on Sep 3, 2024

• transformation test pass.

  Show description for 6b05005
  xipingyan

  committed
  6b05005

Commits on Sep 6, 2024

• Test pass.

  Show description for 58d9f6f
  xipingyan

  committed
  58d9f6f

• Fix error: one param link to mulitple ReadValueWithSubgraphNode

  Show description for d54dc25
  xipingyan

  committed
  d54dc25

• Add submodel infer to MemoryInput::runDynamic

  Show description for a533d73
  xipingyan

  committed
  a533d73

• Debug code

  xipingyan

  committed
  f7339e3

• Merge remote-tracking branch 'origin/master' into xp/whisper_readvalue_optimize

  xipingyan

  committed
  e142a06

• fix merge error

  xipingyan

  committed
  4a2dba0

• Dynamic shape test pass

  Show description for bf7e493
  xipingyan

  committed
  bf7e493

• test whisper pass

  xipingyan

  committed
  d90144e

Commits on Sep 9, 2024

• Disable debug log to test performance. Got expected result:

  Show description for 5a98e7b
  xipingyan

  committed
  5a98e7b

• Add test.

  xipingyan

  committed
  577721d

Commits on Sep 10, 2024

• Remove stateName in ov::Node

  Show description for c23062e
  xipingyan

  committed
  c23062e

• Add env: ENABLE_RV for comprison test.

  xipingyan

  committed
  592919b

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  f134307

• Merge branch 'xp/whisper_readvalue_optimize' of https://github.com/xipingyan/openvino into xp/whisper_readvalue_optimize

  xipingyan

  committed
  258c3c8

• rm debug log

  xipingyan

  committed
  5c771b0

• [CPU] Introduce SubModel op and Composite node

  Show description for 17b8ce3
  

  EgorDuplensky

  authored and

  xipingyan

  committed
  17b8ce3

• Apply review comments

  

  EgorDuplensky

  authored and

  xipingyan

  committed
  3a6b83a

• Merge branch 'egor/introduce_subgraph_node' into xp/whisper_readvalue_optimize

  xipingyan

  committed
  f57851d

• Integrate SubModel, WIP.

  Show description for ca7dde8
  xipingyan

  committed
  ca7dde8

Commits on Sep 11, 2024

• Remove not used variable.

  Show description for df7bb70
  xipingyan

  committed
  df7bb70

Commits on Sep 12, 2024

• Merge remote-tracking branch 'origin/master' into xp/whisper_readvalue_optimize

  xipingyan

  committed
  aa33812

• Remove USE_SUBMODEL

  xipingyan

  committed
  62772a9

Commits on Sep 13, 2024

• Fix search gap.

  Show description for cebed6f
  xipingyan

  committed
  cebed6f

• Tmp version, test integrate new interface of Graph, Init and Activate.

  Show description for d0f7986
  xipingyan

  committed
  d0f7986

Commits on Sep 16, 2024

• After calling new interface of graph, fix memory inPlace issue.

  Show description for 47f436c
  xipingyan

  committed
  47f436c

• remove tmp code

  xipingyan

  committed
  dbb9a3e

• "remove debug log"

  Show description for 2ed69c0
  xipingyan

  committed
  2ed69c0

Commits on Sep 18, 2024

• Remove profiling code.

  Show description for 44c8fb2
  xipingyan

  committed
  44c8fb2

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  87664d4

• Revert unchanged code.

  xipingyan

  committed
  153b4b8

• Simplify codes.

  Show description for e188276
  xipingyan

  committed
  e188276

• Add judge whether subGraph can be called.

  Show description for 2833602
  xipingyan

  committed
  2833602

• Fix test fail issue: readvalue have no any input.

  Show description for 0a6f13f
  xipingyan

  committed
  0a6f13f

Commits on Sep 19, 2024

• Remove get_body, m_subgraph, and update haveSubgraph.

  Show description for 7ae318f
  xipingyan

  committed
  7ae318f

• Replace set_body with base class set_function

  Show description for 79b8272
  xipingyan

  committed
  79b8272

Commits on Sep 20, 2024

• remove debug env

  xipingyan

  committed
  9c0989d

Commits on Sep 25, 2024

• 1: Add check memoryNode null

  Show description for 3eaea83
  xipingyan

  committed
  3eaea83

• 1: Remove reset_prime_mem setter

  Show description for d3bf35c
  xipingyan

  committed
  d3bf35c

• Remove getSupportedDescriptors in cpp.

  xipingyan

  committed
  266cf38

Commits on Sep 26, 2024

• 1. Removed const MemoryPtr& prime_mem() const

  Show description for e94e67f
  xipingyan

  committed
  e94e67f

• 1: Remove redefine supportedPrimitiveDescriptors

  Show description for e5402f8
  xipingyan

  committed
  e5402f8

Commits on Sep 27, 2024

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  74147b7

Commits on Oct 8, 2024

• Simply codes.

  Show description for 8702a13
  xipingyan

  committed
  8702a13

• 1: Remove semicolon after {}

  Show description for fe6f9b4
  xipingyan

  committed
  fe6f9b4

Commits on Oct 11, 2024

• 1: visit_attributes add variable id info

  Show description for 5f14cc2
  xipingyan

  committed
  5f14cc2

• Remove is_in_subgraph, use hashset to check if exist.

  Show description for a4bc6cd
  xipingyan

  committed
  a4bc6cd

Commits on Oct 12, 2024

• Shorten final_successor_is_only_root to successor_is_root

  Show description for 2da8a3a
  xipingyan

  committed
  2da8a3a

Commits on Oct 14, 2024

• Add hashset variable visited to reduce time complexity.

  Show description for 942a10c
  xipingyan

  committed
  942a10c

Commits on Oct 16, 2024

• Take ReadValue Assign pair as stand feature.

  Show description for 6ce3e63
  xipingyan

  committed
  6ce3e63

Commits on Oct 17, 2024

• Move to MemoryInputSingle

  Show description for ef00d66
  xipingyan

  committed
  ef00d66

• Merge remote-tracking branch 'origin/master' into xp/whisper_readvalue_optimize

  xipingyan

  committed
  140862a

Commits on Oct 19, 2024

• dynamic test pass.

  Show description for f4a9dce
  xipingyan

  committed
  f4a9dce

Commits on Oct 21, 2024

• Remove resolveInPlaceEdges call in the Graph::Allocate.

  Show description for afaf42f
  xipingyan

  committed
  afaf42f

Commits on Oct 22, 2024

• Add shape inference.

  Show description for 00da6b7
  xipingyan

  committed
  00da6b7

• Merge remote-tracking branch 'origin/master' into xp/whisper_readvalue_optimize

  xipingyan

  committed
  bb006dc

• Remove search depth limit.

  Show description for 37bc18f
  xipingyan

  committed
  37bc18f

• Update node search function name, make it easy to understand.

  Show description for ad2a87a
  xipingyan

  committed
  ad2a87a

• Initial State doesn't work

  maxnick

  committed
  6ccd217

• Fix transformation

  maxnick

  committed
  f0142fc

• Fix isSuitable node check

  maxnick

  committed
  c3db59d

• Introduce a Single mode in the MemoryInputBase class

  maxnick

  committed
  0d01e93

• Make MemoryInput strictly non const

  maxnick

  committed
  8403952

• Add single buffer state implementation

  maxnick

  committed
  a34ab0c

• Merge remote-tracking branch 'origin/master' into Drop_Assign

  maxnick

  committed
  7319791

• Add commit call to the MemoryInputSingle

  maxnick

  committed
  dcb3603

• Enable LoRA test with random tensors

  maxnick

  committed
  2c61fb7

Commits on Oct 23, 2024

• Clean up LoRA test

  maxnick

  committed
  ced56d6

Commits on Oct 25, 2024

• 1: Merge branch 'maxsim/drop_assgin' into xp/whisper_readvalue_optimize

  Show description for 1074f19
  xipingyan

  committed
  1074f19

• remove memoryoutputsinglestub

  Show description for dd5dd8a
  xipingyan

  committed
  dd5dd8a

• Removed Assign node check when finding init_graph of ReadValue.

  Show description for 5e91806
  xipingyan

  committed
  5e91806

• Let test support Indirect ReadValue Assgin Pair.

  Show description for cf8a18f
  xipingyan

  committed
  cf8a18f

Commits on Oct 30, 2024

• Merge remote-tracking branch 'origin/master' into xp/whisper_readvalue_optimize

  xipingyan

  committed
  893888e

• Update OutputConfig desc

  xipingyan

  committed
  6fe7302

Commits on Nov 1, 2024

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  2f14480

Commits on Nov 4, 2024

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  786c59e

Commits on Nov 5, 2024

• 1: Remove InitGraphStatefulModelInplace, ReadValueAssignTest can cover this.

  Show description for a998273
  xipingyan

  committed
  a998273

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  4abc1b0

Commits on Nov 6, 2024

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  5d5ae39

Commits on Nov 11, 2024

• create a separate input memory objects instead of share them in subgraph input memory.

  Show description for 388c72a
  xipingyan

  committed
  388c72a

Commits on Nov 13, 2024

• Move subGraph init into MemoryInput::initOptimalPrimitiveDescriptor()

  Show description for d0fb105
  xipingyan

  committed
  d0fb105

Commits on Nov 14, 2024

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  de904bb

• Temporarily skip this pattern. If MemoryInputSDPA supports Subgraph in the future, it may be deleted.

  Show description for b621364
  xipingyan

  committed
  b621364

Commits on Nov 15, 2024

• 1: Remove "using MemoryInputBase::MemoryInputBase;inheritance "

  Show description for 63bb470
  xipingyan

  committed
  63bb470

• Remove assignedMem->redefineDesc(...

  Show description for 5d6c9de
  xipingyan

  committed
  5d6c9de

• update: std::shared_ptr<ov::Model> func = nullptr,

  xipingyan

  committed
  e2371d5

• Simplify code about ov::optional init.

  Show description for 800bca3
  xipingyan

  committed
  800bca3

• Correct grammar for test description.

  xipingyan

  committed
  84cddcd

• Recover remove parent edges code, because it will trigger fail for mulitply parents edges.

  Show description for 128e3ab
  xipingyan

  committed
  128e3ab

Commits on Nov 16, 2024

• Just compare node with pointer,

  Show description for 5bbf247
  xipingyan

  committed
  5bbf247

• All visited nodes should be flagged.

  xipingyan

  committed
  fa3ed85

• Use shared_ptr to wrapper subGraph.

  xipingyan

  committed
  1e9a327

Commits on Nov 19, 2024

• Call wrapped make_unique.

  Show description for 56c2da7
  xipingyan

  committed
  56c2da7

• Revert "Remove assignedMem->redefineDesc(..."

  Show description for 2055ea1
  xipingyan

  committed
  2055ea1

Commits on Nov 20, 2024

• Only init subGraph when haveSubgraph() is true.

  Show description for 305c4c8
  xipingyan

  committed
  305c4c8

• -graphOutputConfig.emplace_back(node::Input::OutputConfig{desc, false});

  Show description for 75147e0
  xipingyan

  committed
  75147e0

• Fix subGraph null for static shape model.

  Show description for 7060e85
  xipingyan

  committed
  7060e85

Commits on Nov 21, 2024

• Remove redefine the node port descriptors in MemoryInput::initOptimalPrimitiveDescriptor()

  Show description for 2bb7f7b
  xipingyan

  committed
  2bb7f7b

• Remove shape inference.

  Show description for f21b11a
  xipingyan

  committed
  f21b11a

• Move some parameters to constructor of ReadValueWithSubgraph

  Show description for 57fb1cf
  xipingyan

  committed
  57fb1cf

Commits on Nov 22, 2024

• Shape inference should keep. Just use "InternalDynShapeInferFactory" to replace my own implementation.

  Show description for 772fee4
  xipingyan

  committed
  772fee4

Commits on Nov 25, 2024

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  2533048

Commits on Nov 26, 2024

• Update src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/move_readvalue_inputs_to_subgraph.cpp

  Show description for aafeedd
  

  xipingyan

  and

  maxnick

  authored
  aafeedd

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  eb37689

• 1: remove subgraph_node_names;

  Show description for 61ef125
  xipingyan

  committed
  61ef125

• Add shape infer to constructor

  Show description for f051628
  xipingyan

  committed
  f051628

• Revert shape infer: InternalDynShapeInferFactory, still adopt validate_nodes_and_infer_types to get subgraph's real shape.

  Show description for 8313cb8
  xipingyan

  committed
  8313cb8

• Don't need to algin precison.

  Show description for 1e211ec
  xipingyan

  committed
  1e211ec

• Remove left codes about old version shape infer.

  Show description for bb64cd3
  xipingyan

  committed
  bb64cd3

• move to stateful_sdpa_fusion.cpp

  Show description for 473db5b
  xipingyan

  committed
  473db5b

Commits on Nov 27, 2024

• Update src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp

  Show description for 6925866
  

  xipingyan

  and

  maxnick

  authored
  6925866

• Rename runtime flag from DisableWithSubgraph to DisableInitSubgraphFusing

  Show description for a7ff187
  xipingyan

  committed
  a7ff187

• Revert "Revert shape infer: InternalDynShapeInferFactory, still adopt validate_nodes_and_infer_types to get subgraph's real shape."

  Show description for 4ca6e1d
  xipingyan

  committed
  4ca6e1d

• Still adopt "InternalDynShapeInfer" to do shape inference, because "InternalDynShapeInfer" to do no thing, we have to update memory after inner graph execution.

  Show description for 4c27b31
  xipingyan

  committed
  4c27b31

• Add transformation test for MoveReadValueInputsToSubgraph

  Show description for 64292c1
  xipingyan

  committed
  64292c1

Commits on Nov 28, 2024

• Use 2 hash table:

  Show description for 6fc3b9c
  xipingyan

  committed
  6fc3b9c

• Forget adding optional convert node in skip pattern:

  Show description for d59e455
  xipingyan

  committed
  d59e455

• Eliminate useless Convert, the patch is small. so add it to this PR.

  Show description for 4a7258d
  xipingyan

  committed
  4a7258d

Commits on Nov 29, 2024

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  e18fdaf

Commits on Dec 2, 2024

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  f819c6d

Commits on Dec 3, 2024

• Fix some test fail issue.

  Show description for 13f1649
  xipingyan

  committed
  13f1649

• Revert "Fix some test fail issue."

  Show description for ca28b36
  xipingyan

  committed
  ca28b36

• Revert "Eliminate useless Convert, the patch is small. so add it to this PR."

  Show description for 5fa614c
  xipingyan

  committed
  5fa614c

Commits on Dec 4, 2024

• Merge branch 'openvinotoolkit:master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  c0b494a

• Fix print average counter crash issue when -DENABLE_DEBUG_CAPS=ON -DENABLE_CPU_DEBUG_CAPS=ON,

  Show description for 1bd7088
  xipingyan

  committed
  1bd7088

Commits on Dec 5, 2024

• ReadValueWithSubmodel's input primitive should keep algin with subgraph input primitive.

  Show description for ea73474
  xipingyan

  committed
  ea73474

• Add new test to cover:

  Show description for 93f0c49
  xipingyan

  committed
  93f0c49

Commits on Dec 6, 2024

• Revert "ReadValueWithSubmodel's input primitive should keep algin with subgraph input primitive."

  Show description for 82e9e04
  xipingyan

  committed
  82e9e04

• Adopt more generic solution to fix ReadValueWithSubgraph's input and its subgraph's input privimtive diff issue.

  Show description for bff2aa0
  xipingyan

  committed
  bff2aa0

• Update src/plugins/intel_cpu/tests/functional/custom/subgraph_tests/src/common/stateful_init_graph.cpp

  Show description for b4d3512
  

  xipingyan

  and

  maxnick

  authored
  b4d3512

• Update src/plugins/intel_cpu/tests/functional/custom/subgraph_tests/src/common/stateful_init_graph.cpp

  Show description for 3f83093
  

  xipingyan

  and

  maxnick

  authored
  3f83093

• Update src/plugins/intel_cpu/tests/functional/custom/subgraph_tests/src/common/stateful_init_graph.cpp

  Show description for 1892ea3
  

  xipingyan

  and

  maxnick

  authored
  1892ea3

• Update src/plugins/intel_cpu/src/nodes/memory.cpp

  Show description for 2b1cfb5
  

  xipingyan

  and

  maxnick

  authored
  2b1cfb5

• simplify construct ov::optional

  Show description for 921733f
  xipingyan

  committed
  921733f

• Remove: bool MemoryInput::needShapeInfer() const

  Show description for 9f425d2
  xipingyan

  committed
  9f425d2

• Rename CREATE_CONST to static function create_const_node

  Show description for 5c0f444
  xipingyan

  committed
  5c0f444

• Update src/plugins/intel_cpu/src/nodes/memory.cpp

  Show description for 78192c5
  

  xipingyan

  and

  maxnick

  authored
  78192c5

• Merge branch 'xp/whisper_readvalue_optimize' of https://github.com/xipingyan/openvino into xp/whisper_readvalue_optimize

  xipingyan

  committed
  649625b

• Update src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/move_readvalue_inputs_to_subgraph.cpp

  Show description for d7a8779
  

  xipingyan

  and

  maxnick

  authored
  d7a8779

• Update src/plugins/intel_cpu/tests/functional/custom/subgraph_tests/src/common/stateful_init_graph.cpp

  Show description for a765763
  

  xipingyan

  and

  maxnick

  authored
  a765763

• only check node type: CheckNumberOfNodesWithType

  Show description for 6bf0d0a
  xipingyan

  committed
  6bf0d0a

• 1: Reduce a test for InitGraphStatefulDiffPrimitiveModel

  Show description for b8bfe45
  xipingyan

  committed
  b8bfe45

• Remove reset, it is enough to run 2 time for test.

  Show description for ef5962b
  xipingyan

  committed
  ef5962b

Commits on Dec 9, 2024

• Add sanity check for

  Show description for 73dcd80
  xipingyan

  committed
  73dcd80

• Revert "Remove reset, it is enough to run 2 time for test."

  Show description for aad6d83
  xipingyan

  committed
  aad6d83

• Add set_state to test.

  Show description for 85a711c
  xipingyan

  committed
  85a711c

• Format codes.

  Show description for db30c3b
  xipingyan

  committed
  db30c3b

• past_k_node, past_v_node should remain unchanged before/after replacenode.

  Show description for cf4339b
  xipingyan

  committed
  cf4339b

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  a9c5a3d

• Update src/plugins/intel_cpu/tests/unit/transformations/readvalue_subgraph.cpp

  Show description for 9e4f021
  

  xipingyan

  and

  maxnick

  authored
  9e4f021

• Update src/plugins/intel_cpu/src/transformations/cpu_opset/common/pass/stateful_sdpa_fusion.cpp

  Show description for e387e78
  

  xipingyan

  and

  maxnick

  authored
  e387e78

Commits on Dec 10, 2024

• Because convert node is sensitive to different platform for fp16 precision.

  Show description for 1536ece
  xipingyan

  committed
  1536ece

Commits on Dec 12, 2024

• Merge remote-tracking branch 'origin/master' into xp/whisper_readvalue_optimize

  Show description for fa41f51
  xipingyan

  committed
  fa41f51

• Revert "Because convert node is sensitive to different platform for fp16 precision."

  Show description for 46c506c
  xipingyan

  committed
  46c506c

• Add inference precision hint to keep align with different platform.

  Show description for 3f21087
  xipingyan

  committed
  3f21087

• Merge remote-tracking branch 'origin/master' into xp/whisper_readvalue_optimize

  Show description for 9dbb1df
  xipingyan

  committed
  9dbb1df

Commits on Dec 13, 2024

• Disable snippets for specific test for consistency across HW

  maxnick

  authored
  d9eeb16

Commits on Dec 17, 2024

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  ac7e9b9

• Fix merge master issue.

  Show description for 1021c8b
  xipingyan

  committed
  1021c8b

• Fix merge master issue. check if exist graphcontext before call getConfig.

  Show description for 0f5e46d
  xipingyan

  committed
  0f5e46d

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  1001888

Commits on Dec 18, 2024

• Merge branch 'master' into xp/whisper_readvalue_optimize

  xipingyan

  authored
  17f10b3