Code

Test transformers 41#12

Code

Open

SANKHA1 wants to merge 194 commits intoSANKHA1:test_transformers_41SANKHA1/ipex-llm:test_transformers_41from intel-analytics:test_transformers_41intel-analytics/ipex-llm:test_transformers_41Copy head branch name to clipboard

+11,749-2,425

Commits

Commits on Aug 8, 2024

• Add openai-whisper pytorch gpu (#11736)

  Show description for 9e65cf0
  lzivan

  authored
  9e65cf0

• enable inference mode for deepspeed tp serving (#11742)

  liu-shaojun

  authored
  107f7aa

• Add qwen2-1.5b-instruct into igpu performance (#11735)

  Show description for 27b4b10
  JinheTang

  authored
  27b4b10

Commits on Aug 9, 2024

• Fix vLLM CPU /chat endpoint (#11748)

  xiangyuT

  authored
  044e486

• Mistral apply_rotary_pos_emb_no_cache_xpu use rope_theta from config (#11747)

  Show description for d8808cc
  qiyuangong

  authored
  d8808cc

• Phi3 support compresskv (#11733)

  Show description for dd46c14
  cyita

  authored
  dd46c14

• fix gptq of llama (#11749)

  Show description for 7e917d6
  rnwang04

  authored
  7e917d6

• fix minicpm V 2.6 repeat output (#11753)

  MeouSker77

  authored
  93455aa

• Support compress kv with lookahead (#11752)

  Show description for 4b9c57c
  cyita

  authored
  4b9c57c

• initial support of IPEX_LLM_PERFORMANCE_MODE (#11754)

  Show description for 66fe2ee
  rnwang04

  authored
  66fe2ee

Commits on Aug 12, 2024

• Fix lightweight-serving codegeex error (#11759)

  hzjane

  authored
  245dba0

• Revert to use out-of-tree GPU driver (#11761)

  Show description for fac4c01
  liu-shaojun

  authored
  fac4c01

• optimize minicpm-v-2_6 repetition penalty (#11763)

  MeouSker77

  authored
  57d1777

• Update npu example and all in one benckmark (#11766)

  JinBridger

  authored
  05989ad

• optimize lookahead init time (#11769)

  rnwang04

  authored
  8db3405

• Set mistral fuse rope to false except fp6 & fp16 (#11765)

  Show description for 1b05cab
  

  ATMxsp01

  and

  ATMxsp01

  authored
  1b05cab

• Update all-in-one benchmark for continuation task input preparation (#11760)

  Show description for f97a77e
  Oscilloscope98

  authored
  f97a77e

• Fix compresskv with lookahead issue (#11767)

  Show description for 841dbcd
  cyita

  authored
  841dbcd

Commits on Aug 13, 2024

• optimize minicpm v 2_6 firs token perf (#11770)

  MeouSker77

  authored
  a1eb793

• Fix stdout in all-in-one benchmark to utf-8 (#11772)

  Oscilloscope98

  authored
  81824ff

• Update npu multimodal example (#11773)

  JinBridger

  authored
  c28b338

• Add experimental support of fused decoder layer for llama2 (#11768)

  plusbang

  authored
  23d3acd

• use new fp32 softmax kernel (#11776)

  MeouSker77

  authored
  aa861df

• Reduce Mistral softmax memory only in low memory mode (#11775)

  Show description for a88c132
  qiyuangong

  authored
  a88c132

• Add gemma-2-2b-it and gemma-2-9b-it to igpu nightly performance test (#11778)

  Show description for ec184af
  

  Oscilloscope98

  and

  ch1y0q

  authored
  ec184af

• fix minicpm-v 2.5 (#11780)

  MeouSker77

  authored
  a184b12

• deepspeed zero3 QLoRA finetuning (#11625)

  Show description for 70c828b
  Uxito-Ada

  authored
  70c828b

• Fix mistral forward_qkv in q4_0 (#11781)

  Show description for 3998de1
  qiyuangong

  authored
  3998de1

• MiniCPM-V support compresskv (#11779)

  Show description for 7cd6ec9
  cyita

  authored
  7cd6ec9

Commits on Aug 14, 2024

• refactor llama convert to fix minicpm-v 2.5 optimization (#11783)

  MeouSker77

  authored
  cb79dcd

• follow up on experimental support of fused decoder layer for llama2 (#11785)

  Show description for 51bcac1
  yangw1234

  authored
  51bcac1

• Troubleshoot for sycl not found (#11774)

  Show description for dbd1425
  JinheTang

  authored
  dbd1425

• fix gemma2 runtime error caused by sliding window (#11788)

  Show description for 43cca3b
  rnwang04

  authored
  43cca3b

• Further all-in-one benchmark update continuation task (#11784)

  Show description for 356281c
  Oscilloscope98

  authored
  356281c

• optimize minicpm v 2.5 (#11793)

  MeouSker77

  authored
  3d6cfa2

• added minicpm-v-2_6 (#11794)

  JinheTang

  authored
  d8d887e

• fix and optimize minicpm v 2 (#11799)

  MeouSker77

  authored
  9a93808

• Fix Windows Unit Test (#11801)

  Show description for e3c1dae
  liu-shaojun

  authored
  e3c1dae

• Fix performance tests (#11802)

  Show description for 016e840
  Oscilloscope98

  authored
  016e840

Commits on Aug 15, 2024

• fix: delete ipex extension import in ppl wikitext evaluation (#11806)

  Show description for 3ac83f8
  

  cranechu0131

  and

  JinheTang

  authored
  3ac83f8

• support and optimize qwen2-audio (#11809)

  MeouSker77

  authored
  07b7f13

• deletion of specification of transformers version (#11808)

  JinheTang

  authored
  f43da2d

• transformers==4.37, yi & yuan2 & vicuna (#11805)

  Show description for 2fbbb51
  JinheTang

  authored
  2fbbb51

• update transformers version for replit-code-v1-3b, `internlm2-chat-… (#11811)

  Show description for 447c8ed
  ch1y0q

  authored
  447c8ed

• rewrite minicpmv optimization (#11816)

  MeouSker77

  authored
  4e178f0

• fix phi3 and minicpmv cpu (#11818)

  MeouSker77

  authored
  828ab16

• add mixed_precision argument on ppl wikitext evaluation (#11813)

  Show description for 28d1c97
  

  cranechu0131

  and

  JinheTang

  authored
  28d1c97

• Remove 4k igpu perf on gemma-2-9b-it (#11820)

  Oscilloscope98

  authored
  6543321

• fix minicpm-v-2 fp16 (#11819)

  MeouSker77

  authored
  750d4ad

• Fix vLLM not convert issues (#11817)

  Show description for e70ae06
  gc-fu

  authored
  e70ae06

Commits on Aug 16, 2024

• Fix lightweight-serving no streaming resp on mtl (#11822)

  hzjane

  authored
  5a80fd2

• Update IPEX_LLM_PERFORMANCE_MODE (#11823)

  Oscilloscope98

  authored
  9e9086c

• Update README.md (#11824)

  jason-dai

  authored
  6a8d07d

• optimize qwen2-audio again (#11825)

  MeouSker77

  authored
  17a0beb

• fix: add run oneAPI instruction for the example of codeshell (#11828)

  Show description for f463268
  

  cranechu0131

  and

  JinheTang

  authored
  f463268

• Reorganize MiniCPM-V-2_6 example & update others MiniCPM-V-2 exmaples (#11815)

  Show description for adfbb91
  

  JinheTang

  and

  ivy-lv11

  authored
  adfbb91

• added link to minicpm-v-2_6 example (#11829)

  JinheTang

  authored
  a508b0a

• Codegeex2 tokenization fix (#11831)

  Show description for e07a556
  JinheTang

  authored
  e07a556

• updated ppl README (#11807)

  Show description for 3b630fb
  

  RyuKosei

  and

  jenniew

  authored
  3b630fb

• force lm_head optimization in any model if set environment variable (#11830)

  MeouSker77

  authored
  e966e85

• Update all-in-one benchmark prompts for continuation task & lookup update for minicpmv (#11827)

  Show description for 96796f9
  Oscilloscope98

  authored
  96796f9

• Add MiniCPM-V-2_6 to iGPU Perf (#11810)

  Show description for 9f17234
  

  JinBridger

  and

  Oscilloscope98

  authored
  9f17234

Commits on Aug 17, 2024

• Remove gemma-2-9b-it 3k input from igpu-perf (#11834)

  Oscilloscope98

  authored
  580c94d

Commits on Aug 19, 2024

• feat: add mixed_precision argument on ppl longbench evaluation (#11837)

  Show description for 46a1cbf
  

  cranechu0131

  and

  JinheTang

  authored
  46a1cbf

• Fixes regarding utf-8 in all-in-one benchmark (#11839)

  Oscilloscope98

  authored
  cfc959d

• fix load low bit com dtype (#11832)

  leonardozcm

  authored
  6841a9a

• Support compress KV with quantize KV (#11812)

  Show description for 3cd4e87
  cyita

  authored
  3cd4e87

• optimize phi3 memory usage again (#11848)

  MeouSker77

  authored
  9490781

• add MiniCPM-Llama3-V-2_5 into all-in-one benchmark (#11849)

  rnwang04

  authored
  a0fbda5

• delete transformers version requirement (#11845)

  Show description for da3d7a3
  JinheTang

  authored
  da3d7a3

• separate prefill into a process (#11787)

  Show description for 99b05ba
  

  yangw1234

  and

  sgwhat

  authored
  99b05ba

• Update Llama2 multi-processes example (#11852)

  Show description for 7380823
  sgwhat

  authored
  7380823

Commits on Aug 20, 2024

• add minicpmv 2.6 load_low_bit workaround (#11856)

  MeouSker77

  authored
  2946420

• Fix typo (#11862)

  Uxito-Ada

  authored
  ee6852c

• Add ipex-llm npu option in setup.py (#11858)

  Show description for 5b83493
  sgwhat

  authored
  5b83493

• optimize phi3 memory usage (#11867)

  MeouSker77

  authored
  d4ee0a8

• Update ipex-llm default transformers version to 4.37.0 (#11859)

  Show description for 5e8286f
  Oscilloscope98

  authored
  5e8286f

• Update performance test regarding updated default transformers==4.37.0 (#11869)

  Show description for 0d58c2f
  

  5 peopleauthored
  0d58c2f

• Pytorch models transformers version update (#11860)

  Show description for 3ee194d
  JinheTang

  authored
  3ee194d

• Update compresskv model forward type logic (#11868)

  Show description for c3c0583
  cyita

  authored
  c3c0583

• Update local import for ppl (#11866)

  Show description for 5df0086
  

  RyuKosei

  and

  jenniew

  authored
  5df0086

• feat: update readme for ppl test (#11865)

  Show description for 32f0a77
  

  10 peopleauthored
  32f0a77

• Fix run_decoders bug (#11871)

  yangw1234

  authored
  bdaeee1

• igpu performance test smal fix (#11872)

  Oscilloscope98

  authored
  37106a8

Commits on Aug 21, 2024

• Spr perf small fix (#11874)

  Oscilloscope98

  authored
  eab6f6d

• fix phi3 (#11878)

  MeouSker77

  authored
  bd1e490

• fix vllm qwen2 models (#11879)

  gc-fu

  authored
  537c0d2

• Refactor npu mp to make it easier to integrate new models (#11873)

  Show description for 209d42a
  yangw1234

  authored
  209d42a

• Update doc for running npu generate example with ipex-llm[npu] (#11876)

  Show description for 8c5c7f3
  sgwhat

  authored
  8c5c7f3

• set IPEX_LLM_LAST_LM_HEAD=1 as default (#11885)

  cyita

  authored
  0236de3

• support chatglm4 in lookup (#11855)

  cyita

  authored
  cc27321

Commits on Aug 22, 2024

• Update README.md (#11889)

  Show description for bdbe995
  lzivan

  authored
  bdbe995

• add compress_kv for baichuan2

  hxsz1997

  committed
  86248b0

• fix typos

  hxsz1997

  committed
  6bb9035

• Support qwen2-1.5b with fused decoderlayer optimization on NPU (#11888)

  plusbang

  authored
  72a7bf6

• fix typoes

  hxsz1997

  committed
  6a5ca17

• Make performance test install specific ipex-llm version from pypi (#11892)

  Oscilloscope98

  authored
  bac98ba

• fix typos

  hxsz1997

  committed
  4adaddd

• fix typos

  hxsz1997

  committed
  2a0aa92

• fix typos

  hxsz1997

  committed
  c6ed1c4

• fix typos

  hxsz1997

  committed
  01ed397

• fix typos

  hxsz1997

  committed
  8a5df93

• fix typos

  hxsz1997

  committed
  48a827a

• add comment

  hxsz1997

  committed
  42398a0

• add comment of change in model forward

  hxsz1997

  committed
  ce7de77

• added tokenization file for codegeex2-6b in pytorch-models(#11875)

  Show description for a8e2573
  JinheTang

  authored
  a8e2573

• add comment of compress kv in attention forward

  hxsz1997

  committed
  a2be3d7

• add comment

  hxsz1997

  committed
  eb1e65f

• Add lightweight-serving whisper asr example (#11847)

  Show description for 5c4ed00
  hzjane

  authored
  5c4ed00

• change 5 pytorch/huggingface models to fp16 (#11894)

  JinheTang

  authored
  18662dc

• Update pypi tag to 2.2.0.dev0 (#11895)

  liu-shaojun

  authored
  c5b51d4

• Fix optimize lm head error (#11899)

  gc-fu

  authored
  278b191

• update npu-readme (#11900)

  lzivan

  authored
  794abe2

• update baichuan-7b

  hxsz1997

  committed
  4cf03d6

• Fix non-stop at eos token problem for lookup generation (#11896)

  Show description for 420ce7d
  Oscilloscope98

  authored
  420ce7d

• update mlp of llama (#11897)

  Show description for 4a61f7d
  rnwang04

  authored
  4a61f7d

Commits on Aug 23, 2024

• Merge pull request #11891 from hxsz1997/baichuan2-compresskv

  Show description for 650e6e6
  hxsz1997

  authored
  650e6e6

• update docker image tag to 2.2.0-SNAPSHOT (#11904)

  liu-shaojun

  authored
  4cf640c

• disable lm_head opt for baichuan2-13b (#11905)

  cyita

  authored
  23631cd

• Add lm_head optimization on NPU (#11903)

  plusbang

  authored
  303a090

• Update IPEX_LLM_PERFORMANCE_MODE with input length threshold (#11908)

  Show description for 24c279e
  Oscilloscope98

  authored
  24c279e

• disable outdated scheduled workflow (#11915)

  liu-shaojun

  authored
  e5dc4e9

Commits on Aug 26, 2024

• Add troubleshooting about transpose value setting

  plusbang

  authored
  dd30377

• [NPU] Add support for running mp minicpm model on npu (#11909)

  Show description for 019f725
  sgwhat

  authored
  019f725

• All-in-one benchmark update regarding performance mode for input length threshold (#11920)

  Show description for a0bbd8e
  Oscilloscope98

  authored
  a0bbd8e

• Support streaming for lookup generation (#11922)

  Show description for c1d07bc
  Oscilloscope98

  authored
  c1d07bc

• update troubleshooting for llama.cpp and ollama (#11890)

  Show description for 5a8fc1b
  ch1y0q

  authored
  5a8fc1b

Commits on Aug 27, 2024

• LLM: Fix vLLM CPU convert error (#11926)

  xiangyuT

  authored
  7ca557a

• refactor from_pretrained API for NPU (#11927)

  lzivan

  authored
  6c3eb1e

• Update NPU example readme (#11931)

  plusbang

  authored
  14dddfc

• update llama3 npu example (#11933)

  cyita

  authored
  e246f1e

• update CORE_XE_VERSION to 2.6.0 (#11929)

  liu-shaojun

  authored
  b11b28e

• Add Qwen2-audio example (#11835)

  Show description for 730d9ec
  ch1y0q

  authored
  730d9ec

• Update benchmark script for NPU (#11932)

  plusbang

  authored
  7c8c9a0

• [NPU] Add example for NPU multi-processing minicpm-1b model (#11935)

  Show description for a81a329
  sgwhat

  authored
  a81a329

• update minicpm to meet latest refactor (#11937)

  sgwhat

  authored
  e211a5b

• NPU Baichuan2 Multi- Process example (#11928)

  jenniew

  authored
  b4b6ddf

• Quick fix benchmark script (#11938)

  plusbang

  authored
  7f7f6c8

• Update npu baichuan2 (#11939)

  lzivan

  authored
  90f6929

• Improve baichuan2 NPU performance (#11942)

  plusbang

  authored
  bec00e2

• update version of llama.cpp / ollama (#11930)

  Show description for 460bc96
  rnwang04

  authored
  460bc96

Commits on Aug 28, 2024

• update tag to 2.2.0-SNAPSHOT (#11947)

  liu-shaojun

  authored
  23f51f8

• test main 3072-384

  songhappy

  committed
  db0a280

• main 3k test

  songhappy

  committed
  27f75f5

• log conf

  songhappy

  committed
  12865dd

• Update llamaindex examples (#11940)

  Show description for e23549f
  hxsz1997

  authored
  e23549f

• 41 new run

  songhappy

  committed
  6a6549f

• added accelerate version specification in open webui quickstart(#11948)

  JinheTang

  authored
  ec67ee7

• [NPU] lm head to cpu (#11943)

  Show description for b38fb67
  cyita

  authored
  b38fb67

• Add vllm awq loading logic (#11950)

  Show description for 0a7bd27
  gc-fu

  authored
  0a7bd27

• use sdp_causal to reduce internvl2-4b memory usage if set environment variable (#11953)

  MeouSker77

  authored
  0fbb102

• [NPU] Add minicpm-2b support for npu multi-processing (#11949)

  Show description for 5ca7390
  sgwhat

  authored
  5ca7390

• Refactor NPU baichuan multiple-process (#11945)

  Show description for 63ac5f6
  jenniew

  authored
  63ac5f6

Commits on Aug 29, 2024

• Support qwen2-7b with fused decoderlayer optimization on NPU (#11912)

  plusbang

  authored
  71f03dc

• Add lnl npu driver recommend version and enable cpu_lm_head on llama3 (#11952)

  Show description for 882f4a5
  cyita

  authored
  882f4a5

• update troubleshooting (#11960)

  cyita

  authored
  5f7ff76

• restore ollama webui quickstart (#11955)

  JinheTang

  authored
  6fc9340

• Update MiniCPM-V-2_6 Example (#11958)

  Show description for 7abe17d
  Oscilloscope98

  authored
  7abe17d

• Update qwen2-7b example script (#11961)

  plusbang

  authored
  14b2c8d

• Update README.md (#11964)

  jason-dai

  authored
  431affd

• Further fix for MiniCPM-V-2_6 example (#11965)

  Oscilloscope98

  authored
  2e49e1f

• Support MiniCPM-V-2_6 multi-modal benchmarking with latency text streamer (#11963)

  Show description for a9e485e
  Oscilloscope98

  authored
  a9e485e

• remove obselete npu code (#11967)

  yangw1234

  authored
  fbf088f

Commits on Aug 30, 2024

• add notes for SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS (#11936)

  Show description for 77b04ef
  ch1y0q

  authored
  77b04ef

• Fix glm4-9b-chat nan error on vllm 0.3.3 (#11970)

  Show description for 7d10341
  hzjane

  authored
  7d10341

• Disable lm head (#11972)

  plusbang

  authored
  cd07788

• modification on llamacpp readme after Ipex-llm latest update (#11971)

  Show description for e895e1b
  JinheTang

  authored
  e895e1b

• fix model path (#11973)

  liu-shaojun

  authored
  1e8c870

• add gptq option for ppl test (#11921)

  Show description for ae7302a
  cranechu0131

  authored
  ae7302a

• [NPU] Add initial support for minicpm-llama-v2.5 (#11962)

  Show description for 158289d
  sgwhat

  authored
  158289d

• Initial NPU support for MiniCPM-V-2_6 (#11966)

  Show description for 60aa1a2
  rnwang04

  authored
  60aa1a2

• fix npu lm_head cpu condition (#11976)

  Show description for 573c20b
  rnwang04

  authored
  573c20b

• small fix (#11978)

  Show description for 4811a49
  rnwang04

  authored
  4811a49

• update npu multimodal readme (#11979)

  Show description for 79978e6
  rnwang04

  authored
  79978e6

• Merge branch 'main' of https://github.com/intel-analytics/ipex-llm into test_transformers_41

  songhappy

  committed
  9e5518e

• for cpu

  songhappy

  committed
  f325660

Commits on Sep 2, 2024

• Add MiniCPM-V cpu example (#11975)

  Show description for 65e281b
  JinBridger

  authored
  65e281b

• Support Qwen2-7b MLP in int4 and transpose_value_cache=True (#11968)

  yangw1234

  authored
  c48817b

• Fix AttributeError of qwen2-1.5B (#11990)

  plusbang

  authored
  a40ea70

• hotfix qwen2-7b weight setting (#11991)

  plusbang

  authored
  2f3d1bd

• Fix wrong attention mask and garbage output for inputs_embeds inputs during lookup generation (#11989)

  Show description for 659d15d
  Oscilloscope98

  authored
  659d15d

Commits on Sep 3, 2024

• Revert prefill logic of qwen2-7b (#11992)

  plusbang

  authored
  01099f0

• Update GraphRAG QuickStart (#11995)

  Show description for 643458d
  Oscilloscope98

  authored
  643458d

• Rename MiniCPM-V-2_6 CPU example (#11998)

  JinBridger

  authored
  2e54f44

• MiniCPM-V-2 & MiniCPM-Llama3-V-2_5 example updates (#11988)

  Show description for 164f47a
  JinheTang

  authored
  164f47a

• Performance mode strategy update for input_embeds input (#11997)

  Oscilloscope98

  authored
  6eb5565

• add save & load support for NPU optimized model (#11999)

  Show description for 9eaff5e
  rnwang04

  authored
  9eaff5e

Commits on Sep 4, 2024

• vllm update for glm-4 model automatic not_convert (#12003)

  hzjane

  authored
  2b993ad

• fix dependabot alerts (#12006)

  Show description for 77cb348
  liu-shaojun

  authored
  77cb348

• fix UT (#12005)

  MeouSker77

  authored
  b1408a1

• Update action.yml (#12016)

  liu-shaojun

  authored
  c6348a4

• revert actions/download-artifact version to 3 (#12017)

  liu-shaojun

  authored
  75b19f8

• update

  songhappy

  committed
  428e62b

• new main Merge branch 'main' of https://github.com/intel-analytics/ipex-llm into test_transformers_41

  songhappy

  committed
  7d8f3a0