DeepSeek-V3 Technical Report

Autor: DeepSeek-AI, Liu, Aixin, Feng, Bei, Xue, Bing, Wang, Bingxuan, Wu, Bochao, Lu, Chengda, Zhao, Chenggang, Deng, Chengqi, Zhang, Chenyu, Ruan, Chong, Dai, Damai, Guo, Daya, Yang, Dejian, Chen, Deli, Ji, Dongjie, Li, Erhang, Lin, Fangyun, Dai, Fucong, Luo, Fuli, Hao, Guangbo, Chen, Guanting, Li, Guowei, Zhang, H., Bao, Han, Xu, Hanwei, Wang, Haocheng, Zhang, Haowei, Ding, Honghui, Xin, Huajian, Gao, Huazuo, Li, Hui, Qu, Hui, Cai, J. L., Liang, Jian, Guo, Jianzhong, Ni, Jiaqi, Li, Jiashi, Wang, Jiawei, Chen, Jin, Chen, Jingchang, Yuan, Jingyang, Qiu, Junjie, Li, Junlong, Song, Junxiao, Dong, Kai, Hu, Kai, Gao, Kaige, Guan, Kang, Huang, Kexin, Yu, Kuai, Wang, Lean, Zhang, Lecong, Xu, Lei, Xia, Leyi, Zhao, Liang, Wang, Litong, Zhang, Liyue, Li, Meng, Wang, Miaojun, Zhang, Mingchuan, Zhang, Minghua, Tang, Minghui, Li, Mingming, Tian, Ning, Huang, Panpan, Wang, Peiyi, Zhang, Peng, Wang, Qiancheng, Zhu, Qihao, Chen, Qinyu, Du, Qiushi, Chen, R. J., Jin, R. L., Ge, Ruiqi, Zhang, Ruisong, Pan, Ruizhe, Wang, Runji, Xu, Runxin, Zhang, Ruoyu, Chen, Ruyi, Li, S. S., Lu, Shanghao, Zhou, Shangyan, Chen, Shanhuang, Wu, Shaoqing, Ye, Shengfeng, Ma, Shirong, Wang, Shiyu, Zhou, Shuang, Yu, Shuiping, Zhou, Shunfeng, Pan, Shuting, Wang, T., Yun, Tao, Pei, Tian, Sun, Tianyu, Xiao, W. L., Zeng, Wangding, Zhao, Wanjia, An, Wei, Liu, Wen, Liang, Wenfeng, Gao, Wenjun, Yu, Wenqin, Zhang, Wentao, Li, X. Q., Jin, Xiangyue, Wang, Xianzu, Bi, Xiao, Liu, Xiaodong, Wang, Xiaohan, Shen, Xiaojin, Chen, Xiaokang, Zhang, Xiaokang, Chen, Xiaosha, Nie, Xiaotao, Sun, Xiaowen, Wang, Xiaoxiang, Cheng, Xin, Liu, Xin, Xie, Xin, Liu, Xingchao, Yu, Xingkai, Song, Xinnan, Shan, Xinxia, Zhou, Xinyi, Yang, Xinyu, Li, Xinyuan, Su, Xuecheng, Lin, Xuheng, Li, Y. K., Wang, Y. Q., Wei, Y. X., Zhu, Y. X., Zhang, Yang, Xu, Yanhong, Huang, Yanping, Li, Yao, Zhao, Yao, Sun, Yaofeng, Li, Yaohui, Wang, Yaohui, Yu, Yi, Zheng, Yi, Zhang, Yichao, Shi, Yifan, Xiong, Yiliang, He, Ying, Tang, Ying, Piao, Yishi, Wang, Yisong, Tan, Yixuan, Ma, Yiyang, Liu, Yiyuan, Guo, Yongqiang, Wu, Yu, Ou, Yuan, Zhu, Yuchen, Wang, Yuduan, Gong, Yue, Zou, Yuheng, He, Yujia, Zha, Yukun, Xiong, Yunfan, Ma, Yunxian, Yan, Yuting, Luo, Yuxiang, You, Yuxiang, Liu, Yuxuan, Zhou, Yuyang, Wu, Z. F., Ren, Z. Z., Ren, Zehui, Sha, Zhangli, Fu, Zhe, Xu, Zhean, Huang, Zhen, Zhang, Zhen, Xie, Zhenda, Zhang, Zhengyan, Hao, Zhewen, Gou, Zhibin, Ma, Zhicheng, Yan, Zhigang, Shao, Zhihong, Xu, Zhipeng, Wu, Zhiyu, Zhang, Zhongyu, Li, Zhuoshu, Gu, Zihui, Zhu, Zijia, Liu, Zijun, Li, Zilin, Xie, Ziwei, Song, Ziyang, Gao, Ziyi, Pan, Zizheng

We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token. To achieve efficient inference and cost-effective training, DeepSeek-V3 adopts Multi-head Latent Attention (MLA) an

Externí odkaz: http://arxiv.org/abs/2412.19437

A Silver Bullet or a Compromise for Full Attention? A Comprehensive Study of Gist Token-based Context Compression

Autor: Deng, Chenlong, Zhang, Zhisong, Mao, Kelong, Li, Shuaiyi, Huang, Xinting, Yu, Dong, Dou, Zhicheng

In this work, we provide a thorough investigation of gist-based context compression methods to improve long-context processing in large language models. We focus on two key questions: (1) How well can these methods replace full attention models? and

Externí odkaz: http://arxiv.org/abs/2412.17483

Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models

Autor: Liu, Wenhan, Ma, Xinyu, Zhu, Yutao, Zhao, Ziliang, Wang, Shuaiqiang, Yin, Dawei, Dou, Zhicheng

Large Language Models (LLMs) have shown exciting performance in listwise passage ranking. Due to the limited input length, existing methods often adopt the sliding window strategy. Such a strategy, though effective, is inefficient as it involves repe

Externí odkaz: http://arxiv.org/abs/2412.14574

Searching for Neutrinoless Double-Beta Decay of $^{136}$Xe with PandaX-4T

Autor: PandaX Collaboration, Zhang, Shu, Bo, Zihao, Chen, Wei, Chen, Xun, Chen, Yunhua, Cheng, Zhaokan, Cui, Xiangyi, Fan, Yingjie, Fang, Deqing, Gao, Zhixing, Geng, Lisheng, Giboni, Karl, Guo, Xunan, Guo, Xuyuan, Guo, Zichao, Han, Chencheng, Han, Ke, He, Changda, He, Jinrong, Huang, Di, Huang, Houqi, Huang, Junting, Hou, Ruquan, Hou, Yu, Ji, Xiangdong, Ji, Xiangpan, Ju, Yonglin, Li, Chenxiang, Li, Jiafu, Li, Mingchuan, Li, Shuaijie, Li, Tao, Li, Zhiyuan, Lin, Qing, Liu, Jianglai, Lu, Congcong, Lu, Xiaoying, Luo, Lingyin, Luo, Yunyang, Ma, Wenbo, Ma, Yugang, Mao, Yajun, Meng, Yue, Ning, Xuyang, Pang, Binyu, Qi, Ningchun, Qian, Zhicheng, Ren, Xiangxiang, Shan, Dong, Shang, Xiaofeng, Shao, Xiyuan, Shen, Guofang, Shen, Manbin, Sun, Wenliang, Tao, Yi, Wang, Anqing, Wang, Guanbo, Wang, Hao, Wang, Jiamin, Wang, Lei, Wang, Meng, Wang, Qiuhong, Wang, Shaobo, Wang, Siguang, Wang, Wei, Wang, Xiuli, Wang, Xu, Wang, Zhou, Wei, Yuehuan, Wu, Weihao, Wu, Yuan, Xiao, Mengjiao, Xiao, Xiang, Xiong, Kaizhi, Xu, Yifan, Yao, Shunyu, Yan, Binbin, Yan, Xiyu, Yang, Yong, Ye, Peihua, Yu, Chunxu, Yuan, Ying, Yuan, Zhe, Yun, Youhui, Zeng, Xinning, Zhang, Minzhen, Zhang, Peng, Zhang, Shibo, Zhang, Tao, Zhang, Wei, Zhang, Yang, Zhang, Yingxin, Zhang, Yuanyuan, Zhao, Li, Zhou, Jifang, Zhou, Jiaxu, Zhou, Jiayi, Zhou, Ning, Zhou, Xiaopeng, Zhou, Yubo, Zhou, Zhizhen

We report the search for neutrinoless double-beta decay of $^{136}$Xe from the PandaX-4T experiment with a 3.7-tonne natural xenon target. The data reconstruction and the background modeling are optimized in the MeV energy region. A blind analysis is

Externí odkaz: http://arxiv.org/abs/2412.13979

FocusChat: Text-guided Long Video Understanding via Spatiotemporal Information Filtering

Autor: Cheng, Zheng, Wang, Rendong, Wang, Zhicheng

Recently, multi-modal large language models have made significant progress. However, visual information lacking of guidance from the user's intention may lead to redundant computation and involve unnecessary visual noise, especially in long, untrimme

Externí odkaz: http://arxiv.org/abs/2412.12833