ggml : AVX2 support for Q4_0_8_8 (#8713) · arthw/llama.cpp@e4b85cd · GitHub

Commit

ggml : AVX2 support for Q4_0_8_8 (ggerganov#8713)

Browse files

* Add AVX2 based implementations for quantize_q8_0_4x8, ggml_gemv_q4_0_8x8_q8_0 and ggml_gemm_q4_0_8x8_q8_0 functions

* Update code to fix issues occuring due to non alignment of elements to be processed as multiple of 16 in MSVC

* Update comments and indentation

* Make updates to reduce number of load instructions

Loading branch information

Srihari-mcw authored and arthw committed Nov 18, 2024

1 parent cac4f55 commit e4b85cd