中国宏观经济月报：DeepSeek的抄袭争议——蒸馏技术的使用

Group 1: DeepSeek Controversy - DeepSeek's rapid rise has led to accusations of "plagiarism" regarding its model's reliance on knowledge from other advanced models[2] - Microsoft security researchers indicated that DeepSeek employees accessed significant data via OpenAI's API, suggesting potential training data for models V3 and R1[2] Group 2: Distillation Technology - Distillation technology is a common optimization method in machine learning, transferring knowledge from large models (teacher) to smaller models (student) to enhance efficiency[3] - The concept of distillation was introduced by Geoffrey Hinton in 2015, emphasizing knowledge transfer rather than copying model architecture or code[3] Group 3: DeepSeek's Implementation - DeepSeek generated 800,000 training samples from its R1 series to enhance the V3 model's training efficiency[5] - The integration of reasoning chains from R1 into V3 significantly improved its reasoning performance, demonstrating the effectiveness of distillation[5] Group 4: Advantages and Challenges of Distillation - Distillation can lower data construction costs and expand the application range of AI models, particularly benefiting small enterprises[7] - However, student models often face performance limitations due to inherent constraints of the teacher models, especially in complex tasks[7]