DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

2024153 citationsJournal Articlegold Open Access

Authors

Damai Dai

Chengqi Deng

Abstract

Damai Dai, Chengqi Deng, Chenggang Zhao, R.x. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y.k. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, Wenfeng Liang. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Topics & Keywords

Topic Modeling Expert finding and Q&A systems Machine Learning in Healthcare

Publication Details

DOI: 10.18653/v1/2024.acl-long.70

Field-Weighted Citation Impact: 52.85

Command Palette

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

Authors

Abstract

Topics & Keywords

Publication Details