Some length extrapolation results on algorithmic/arithmetic tasks

[HanseulJo]

Thank you for your contribution to listing the papers about long-context LLMs.
Let me bring three recent papers about Transformer's length extrapolation on algorithmic/arithmetic tasks, one of which is already on the list but its title is changed a bit:

1. Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure
   • https://openreview.net/forum?id=5cIRdGM1uG
   • The title is changed (originally, it was "Position Coupling: Leveraging Task Structure for Improved Length Generalization of Transformers")
   • Published at NeurIPS 2024
2. Transformers Can Do Arithmetic with the Right Embeddings
   • https://openreview.net/forum?id=cBFsFt1nDW
   • Published at NeurIPS 2024
3. Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count
   • https://arxiv.org/abs/2410.15787

[Xnhyacinth]

Sure! Thank you very much for your contribution! I will improve it in today's update.