From 9dc4b89f5ad6be7e66beae6f30dfd954a928582d Mon Sep 17 00:00:00 2001 From: "Pan, Yujie" Date: Tue, 16 Jul 2024 00:24:30 +0800 Subject: [PATCH] mention dejavu for acceleration example --- .../torch/sparsify_activations/ActivationSparsity.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/nncf/experimental/torch/sparsify_activations/ActivationSparsity.md b/nncf/experimental/torch/sparsify_activations/ActivationSparsity.md index a7aceb0d1fb..96292321963 100644 --- a/nncf/experimental/torch/sparsify_activations/ActivationSparsity.md +++ b/nncf/experimental/torch/sparsify_activations/ActivationSparsity.md @@ -1,6 +1,6 @@ ### Activation Sparsity (experimental feature) -The `sparsify_activations` algorithm is a post-training method designed to introduce sparsity into the activations of a neural network. This process reduces the number of active neurons during inference by masking out neurons based on their magnitude relative to a calibrated static threshold. +The `sparsify_activations` algorithm is a post-training method designed to introduce sparsity into the activations of a neural network. This process reduces the number of active neurons during inference by masking out neurons based on their magnitude relative to a calibrated static threshold. Typically this can help accelerate inference for Transformer-based Large Language Models on edge devices; one such example is [Liu et al., 2023](https://arxiv.org/abs/2310.17157). The algorithm sparsifies the input of a layer by applying the following function: