TY  - UNPB
ID  - dl-15-1
TI  - Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
AU  - Shazeer, Noam
AU  - Mirhoseini, Azalia
AU  - Maziarz, Krzysztof
AU  - Davis, Andy
AU  - Le, Quoc
AU  - Hinton, Geoffrey
AU  - Dean, Jeff
T2  - arXiv preprint
PY  - 2017
DO  - 10.48550/arXiv.1701.06538
UR  - https://arxiv.org/abs/1701.06538
ER  - 


TY  - JOUR
ID  - dl-15-2
TI  - Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
AU  - Fedus, William
AU  - Zoph, Barret
AU  - Shazeer, Noam
JO  - Journal of Machine Learning Research, 23(120):1-39
PY  - 2022
UR  - https://jmlr.org/papers/v23/21-0998.html
ER  - 


TY  - UNPB
ID  - dl-15-3
TI  - GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
AU  - Lepikhin, Dmitry
AU  - Lee, HyoukJoong
AU  - Xu, Yuanzhong
AU  - Chen, Dehao
AU  - Firat, Orhan
AU  - Huang, Yanping
AU  - Krikun, Maxim
AU  - Shazeer, Noam
AU  - Chen, Zhifeng
T2  - arXiv preprint
PY  - 2020
DO  - 10.48550/arXiv.2006.16668
UR  - https://arxiv.org/abs/2006.16668
ER  -