计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (6A): 250800005-9.doi: 10.11896/jsjkx.250800005

• 计算机软件&体系结构 • 上一篇    下一篇

基于动态调节交叉熵损失的MCU驱动代码PID-Dynamic LSTM生成模型

刘子璇1,2, 唐小勇1   

  1. 1 长沙理工大学计算机学院 长沙 410114
    2 上海芯钛信息科技有限公司 长沙 410005
  • 出版日期:2026-06-16 发布日期:2026-06-12
  • 通讯作者: 唐小勇(tangxy@csust.edu.cn)
  • 作者简介:(1594751922@qq.com)
  • 基金资助:
    国家自然科学基金(62372064,62472151)

PID-Dynamic LSTM Generation Model for MCU Driver Code Based on Dynamically-tuned Cross-entropy Loss

LIU Zixuan1,2, TANG Xiaoyong1   

  1. 1 School of Computer and Communications Engineering,Changsha University of Science & Technology,Changsha 410114,China
    2 Shanghai ThinkTech Information Technology Co.,Ltd.,Changsha 410005,China
  • Published:2026-06-16 Online:2026-06-12
  • About author:LIU Zixuan,born in 1999,M.S.candidate.His main research interests include deep learning,MCU-driven code generation,and research on code genera-tion models combined with control theory,etc.
    TANG Xiaoyong,born in 1973,Ph.D,professor,is a premium member of CCF(No.33420S).His main research in-terests include parallel distributed computing and big data,etc.
  • Supported by:
    National Natural Science Foundation of China(62372064,62472151).

PDF (PC)

33

摘要： 针对深度学习中噪声数据导致的模型过拟合与训练不稳定问题,首次将控制理论中的动态误差补偿机制引入代码生成任务,提出一种基于动态调节交叉熵损失函数(PID-CE Loss)的代码生成模型(PID-Dynamic LSTM)。传统交叉熵损失函数在噪声场景下易受异常样本干扰,导致梯度更新方向偏移与收敛速度下降。通过融合比例(P)、积分(I)、微分(D)控制项,构建动态误差补偿机制:1)比例项保留交叉熵的即时误差响应特性;2)积分项引入指数移动平均(EMA)差分捕捉损失变化的长期趋势,修正累积偏差;3)微分项通过约束相邻训练步概率分布的均方误差(MSE),抑制噪声引发的预测突变。实验表明,在500轮含噪声训练中,所提方法在测试数据集上达到96.28%验证准确率(较基线提升3.42%),首次达到80%准确率所需训练轮次减少31.7%(224→153轮),过拟合间隙降低6.4%,损失波动下降18.5%。消融实验进一步验证了PID-CE对噪声抑制的关键作用以及参数特性。该方法为噪声鲁棒性优化提供了理论可解释、工程易部署的新范式,在噪声敏感场景中具有相应的应用潜力。

关键词: PID控制, 深度学习, 交叉熵损失函数, PID-CE Loss, PID-Dynamic LSTM, MCU驱动代码生成关键词, TTA8

Abstract: To address the issues of model overfitting and training instability caused by noisy data in deep learning,this paper introduces,for the first time,a dynamic error compensation mechanism from control theory into code generation tasks.It proposes a code generation model named PID-Dynamic LSTM,based on a dynamically-tuned cross-entropy loss function(PID-CE Loss).Traditional cross-entropy loss is vulnerable to interference from anomalous samples under noisy conditions,leading to deviations in gradient updates and reduced convergence speed.To mitigate this,it integrates proportional(P),integral(I),and derivative(D) control terms to construct a dynamic error compensation mechanism.1)Proportional term preserves the immediate error response characteristic of cross-entropy.2)Integral term incorporates exponential moving average(EMA) differentialto capture long-term trends in loss variation,thereby correcting accumulated bias.3)Derivative term suppresses prediction fluctuations induced by noise by constraining the mean squared error(MSE) of probability distributions between adjacent training steps.Experimental results demonstrate that during 500 epochs of noisy training,the proposed method achieves an 96.28% validation accuracy on the test dataset(+3.42% improvement over baselines).Critically,it reduces the number of epochs required to first reach 80% accuracy by 31.7%(from 224 to 153 epochs).Furthermore,it reduces the overfitting gap by 6.4% and decreases loss fluctuation by 18.5%.Ablation experiment further verifies the key role and parameter characteristics of PID-CE in noise suppression.This method establishes a theoretically interpretable and engineering-friendly paradigm for noise-robust optimization,demonstrating significant application potential in noise-sensitive scenarios.

Key words: PID control, Deep learning, Cross-entropy loss, PID-CE Loss, PID-Dynamic LSTM, MCU driver code generation, TTA8

中图分类号: 

  • TP311
[1] LIU K,YIN C,ZHAO H,et al.Prediction of sand body thickness based on a mixed density network constrained by a spatially approximated probability[J].Geophysical Prospecting for Petroleum,2020,59(4):596-606,615.
[2] AO B Q,JIANG X J,LIU X Y,et al.Design of Unmanned Surface Vehicle Control System Based onNeural Network-PID Control[J].Control Engineering of China,2024,31(7):1178-1184.
[3] QIN D H,LI Z T,BAI F B,et al.A Review of Parameter-Efficient Fine-Tuning Technology for Large Language Models[J].Computer Engineering and Applications,2025,61(16):38-63.
[4] TIPIRNENI S,ZHU M,REDDY C K.Structcoder:Structure-aware transformer for code generation[J].ACM Transactions on Knowledge Discovery from Data,2024,18(3):1-20.
[5] FARHADI A,MIRZAREZAEE M,SHARIFI A,et al.Domain adaptation in reinforcement learning:a comprehensive and systematic study[J].Frontiers of Information Technology & Electronic Engineering,2024,25(11):1446-1465.
[6] LE H,WANG Y,GOTMARE A D,et al.CodeRL:Mastering code generation through pretrained models and deep reinforcement learning[J].arXiv:2207.01780,2022.
[7] ZHANG T Y,YU T,HASHIMOTO T B,et al.Coder reviewer reranking for code generation[J].arXiv:2211.16490,2022.
[8] HUANG Y,GU H,YU Z,et al.Enhancing low-resource cross-lingual summarization from noisy data with fine-grained reinforcement learning[J].Frontiers of Information Technology & Electronic Engineering,2024,25(1):121-134.
[9] XU F F,JIANG Z B,YIN P C,et al.Incorporating externalknowledge through pre-training for natural language to code generation[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Association for Computational Linguistics,2020:6045-6052.
[10] YIN P C,NEUBIG G.TRANX:A transition-based neural abstract syntax parser for semantic parsing and code generation[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing:System Demonstrations.Association for Computational Linguistics,2018:7-12.
[11] SOW D,WOISETSCHLÄGER H,BULUSU S,et al.Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining[J].arXiv:2502.06733,2025.
[12] TERVEN J,CORDOVA-ESPARZA D M,RAMIREZ-PEDRAZA A,et al.Loss functions and metrics in deep learning[J].ar-Xiv:2307.02694,2023.
[13] CIAMPICONI L,ELWOOD A,LEONARDI M,et al.A survey and taxonomy of loss functions in machine learning[J].arXiv:2301.05579,2023.
[14] DUAN F,WANG H M,ZHANG C.Cauchy Non-negative Matrix Factorization for Data Representation[J].Computer Science,2021,48(6):96-102.
[15] GONG Z Y.Robust Principal Component Analysis Based onWelsch Loss and Half-Quadratic Optimization[C]//2024 International Conference on Wavelet Analysis and Pattern Recognition(ICWAPR).IEEE,2024.
[16] PERETS K,OLEKSII K.Assessment of the impact of sparsity and geman-mcclure regularization on signal reconstruction accuracy[J].Management,Training and Communication System,2025,2:239-243.
[17] TERVEN J,CORDOVA-ESPARZA D M,RAMIREZ-PEDRAZA A,et al.Loss functions and metrics in deep learning[J].ar-Xiv:2307.02694,2023.
[18] WEN W,HU Z X,HAO Z F.Deep Exponential Moving Ave-rage Learning Method for Sequential Recommendation [J].Journal of Frontiers of Computer Science and Techno,2025,19(3):774-786.
[19] CHU Y L,ZHENG H,HOU X P.Chinese semantic similarity calculation based on dynamic semantic coding Bi-Directional LSTM[J].Computer Applications and Software,2020,37(6):224-229.
[20] XIAO S L,QIANG Z X,LI D Y,et al.Pedestrian detection combining fine-grained feature and attention mechanism[J].Computer Applications and Software,2025,42(4):166-173,207.
[21] QI Q R L G,SI Q T,WANG S G L.A Survey of Automatic Text Summarization Based on Deep Learning [J].Computer Engineering and Applications,2025,61(18):24-40.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发