TY  - UNPB
ID  - dl3-a01-1
TI  - Attention Is All You Need
AU  - Ashish Vaswani
AU  - Noam Shazeer
AU  - Niki Parmar
AU  - Jakob Uszkoreit
AU  - Llion Jones
AU  - Aidan N. Gomez
AU  - Lukasz Kaiser
AU  - Illia Polosukhin
PY  - 2017
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/1706.03762
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  - 

TY  - UNPB
ID  - dl3-a01-2
TI  - Low-Rank Bottleneck in Multi-head Attention Models
AU  - Srinadh Bhojanapalli
AU  - Chulhee Yun
AU  - Ankit Singh Rawat
AU  - Sashank J. Reddi
AU  - Sanjiv Kumar
PY  - 2020
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/2002.07028
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  - 

TY  - UNPB
ID  - dl3-a01-3
TI  - Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth
AU  - Yihe Dong
AU  - Jean-Baptiste Cordonnier
AU  - Andreas Loukas
PY  - 2021
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/2103.03404
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  - 

TY  - UNPB
ID  - dl3-a01-4
TI  - Are Transformers universal approximators of sequence-to-sequence functions?
AU  - Chulhee Yun
AU  - Srinadh Bhojanapalli
AU  - Ankit Singh Rawat
AU  - Sashank J. Reddi
AU  - Sanjiv Kumar
PY  - 2019
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/1912.10077
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  - 

TY  - UNPB
ID  - dl3-a01-5
TI  - Rethinking Attention with Performers
AU  - Krzysztof Choromanski
AU  - Valerii Likhosherstov
AU  - David Dohan
AU  - Xingyou Song
AU  - Andreea Gane
AU  - Tamas Sarlos
AU  - Peter Hawkins
AU  - Jared Davis
AU  - Afroz Mohiuddin
AU  - Lukasz Kaiser
AU  - David Belanger
AU  - Lucy Colwell
AU  - Adrian Weller
PY  - 2020
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/2009.14794
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  - 

TY  - UNPB
ID  - dl3-a01-6
TI  - Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
AU  - Angelos Katharopoulos
AU  - Apoorv Vyas
AU  - Nikolaos Pappas
AU  - François Fleuret
PY  - 2020
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/2006.16236
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  -