TY  - UNPB
ID  - dl3-a02-1
TI  - RoFormer: Enhanced Transformer with Rotary Position Embedding
AU  - Jianlin Su
AU  - Yu Lu
AU  - Shengfeng Pan
AU  - Ahmed Murtadha
AU  - Bo Wen
AU  - Yunfeng Liu
PY  - 2021
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/2104.09864
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  - 

TY  - UNPB
ID  - dl3-a02-2
TI  - Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
AU  - Ofir Press
AU  - Noah A. Smith
AU  - Mike Lewis
PY  - 2021
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/2108.12409
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  - 

TY  - UNPB
ID  - dl3-a02-3
TI  - Extending Context Window of Large Language Models via Positional Interpolation
AU  - Shouyuan Chen
AU  - Sherman Wong
AU  - Liangjian Chen
AU  - Yuandong Tian
PY  - 2023
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/2306.15595
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  - 

TY  - UNPB
ID  - dl3-a02-4
TI  - YaRN: Efficient Context Window Extension of Large Language Models
AU  - Bowen Peng
AU  - Jeffrey Quesnelle
AU  - Honglu Fan
AU  - Enrico Shippole
PY  - 2023
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/2309.00071
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  - 

TY  - UNPB
ID  - dl3-a02-5
TI  - Lost in the Middle: How Language Models Use Long Contexts
AU  - Nelson F. Liu
AU  - Kevin Lin
AU  - John Hewitt
AU  - Ashwin Paranjape
AU  - Michele Bevilacqua
AU  - Fabio Petroni
AU  - Percy Liang
PY  - 2023
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/2307.03172
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  - 

TY  - UNPB
ID  - dl3-a02-6
TI  - Exploring Length Generalization in Large Language Models
AU  - Cem Anil
AU  - Yuhuai Wu
AU  - Anders Andreassen
AU  - Aitor Lewkowycz
AU  - Vedant Misra
AU  - Vinay Ramasesh
AU  - Ambrose Slone
AU  - Guy Gur-Ari
AU  - Ethan Dyer
AU  - Behnam Neyshabur
PY  - 2022
T2  - arXiv primary author manuscript
UR  - https://arxiv.org/abs/2207.04901
N1  - Original arXiv bibliographic record and abstract inspected on 2026-10-28; full proof and source experiments not independently audited.
ER  -