Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
-
Updated
Oct 25, 2024 - Python
#
efficient-attention
Here are 9 public repositories matching this topic...
Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
-
Updated
Sep 6, 2024 - Python
Efficient Infinite Context Transformers with Infini-attention Pytorch Implementation + QwenMoE Implementation + Training Script + 1M context keypass retrieval
-
Updated
May 9, 2024 - Python
Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".
deep-learning
neural-network
pytorch
artificial-intelligence
transformer
attention-mechanism
iclr
efficient-attention
iclr2022
-
Updated
Oct 29, 2021 - Jupyter Notebook
The official PyTorch implementation for CascadedGaze: Efficiency in Global Context Extraction for Image Restoration, TMLR'24.
-
Updated
Oct 8, 2024 - Python
Pytorch implementation of "Compact Global Descriptor for Neural Networks" (CGD).
efficient
pytorch
convolutional-neural-networks
attention-mechanism
attention-model
efficient-attention
-
Updated
Nov 24, 2019 - Python
Implementation of: Hydra Attention: Efficient Attention with Many Heads (https://arxiv.org/abs/2209.07484)
-
Updated
Jan 8, 2023 - Python
Nonparametric Modern Hopfield Models
efficient-transformers
efficient-attention
modern-hopfield-networks
modern-hopfield-model
efficient-hopfield-models
efficient-hopfield-networks
-
Updated
Jan 8, 2024 - Jupyter Notebook
Official Implementation of SEA: Sparse Linear Attention with Estimated Attention Mask (ICLR 2024)
-
Updated
Oct 24, 2024 - Python
Improve this page
Add a description, image, and links to the efficient-attention topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the efficient-attention topic, visit your repo's landing page and select "manage topics."