计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (10): 285-294.doi: 10.11896/jsjkx.260500060

• 人工智能 • 上一篇    下一篇

基于双阶段触发的反向验证逻辑推理方法

李鹏1,2, 邱浙宇1, 胡素君1   

  1. 1 南京邮电大学计算机学院、软件学院、网络空间安全学院 南京 210023
    2 南京邮电大学网络安全与可信计算研究所 南京 210023
  • 收稿日期:2026-05-13 修回日期:2026-07-27 发布日期:2026-10-13
  • 通讯作者: 李鹏(lipeng@njupt.edu.cn)
  • 基金资助:
    江苏省六大才高峰高层次人才项(RJFW-111);江苏省研究生科研与实践创新计划项目(KYCX23_1048)

Reverse Verification Logical Reasoning Method Based on Two-stage Triggering

LI Peng1,2, QIU Zheyu1, HU Sujun1   

  1. 1 School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023, China
    2 Institute of Network Security and Trusted Computing,Nanjing University of Posts and Telecommunications,Nanjing 210023, China
  • Received:2026-05-13 Revised:2026-07-27 Online:2026-10-13
  • About author:LI Peng,born in 1979,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.48573M).His main research interests include computer communication networks,cloud computing and information security.
  • Supported by:
    Six Talent Peaks Project of Jiangsu Province(RJFW-111) and Postgraduate Research and Practice Innovation Program of Jiangsu Province(KYCX23_1048).

PDF (PC)

78

摘要： 针对大语言模型在复杂逻辑推理任务中存在的单向生成误差累积、高置信错误难以识别以及全量外部验证计算开销高的问题,提出一种基于反向验证的逻辑推理方法。以前向推理器输出的终端Token置信度和候选判别裕度为依据,构建两阶段动态触发机制,并利用受试者工作特征曲线与约登指数确定最优验证阈值,实现高风险样本筛选;复用前向生成的Top-K候选空间构造反事实假设,通过外部判题器多轮投票获得独立判别信号,完成逻辑一致性校验;进一步结合判题裕度设计动态决策门控,抑制不可靠反向信号造成的误纠错。LogiQA 2.0数据集上的实验结果表明:在候选空间大小 K=3、外部判题器独立投票次数 N=3,并结合双阶段置信度筛选的配置下,所提方法的准确率达到77.16%,较标准思维链(Standard CoT)基线提升6.94个百分点,触发验证比例为34.94%,平均验证耗时为1.3206s,体现出较好的精度与效率平衡。在进一步引入动态门控后,系统准确率为76.41%,修正精度提升至66.89%,破坏性翻转数由57降至33,表明该机制能够以少量整体精度代价换取更高的纠错可靠性与系统鲁棒性;在ReClor上也取得了一致增益,验证了方法的泛化性。结果表明,该方法能够在较低的附加开销下缓解大语言模型逻辑推理中的高置信错误问题。

关键词: 大语言模型, 逻辑推理, 反向验证, 置信度评估, 动态门控

Abstract: To address three major challenges of large language models (LLMs) in complex logical reasoning tasks,namely error accumulation in one-pass forward generation,the difficulty of detecting high-confidence errors,and the high computational cost of exhaustive external verification,this paper proposes a reverse verification logical reasoning framework.Specifically,a two-stage dynamic triggering mechanism is developed based on the confidence of the final token generated by the forward reasoner and the margin between candidate predictions.The receiver operating characteristic (ROC) curve and Youden’s index are further employed to determine the optimal verification threshold for high-risk sample selection.The Top-K candidate space produced during forward reasoning is then reused to construct counterfactual hypotheses,and an independent judgment distribution is obtained through multiple rounds of voting by an external evaluator,enabling logical consistency checking.In addition,a dynamic decision gating mechanism is introduced based on the evaluator margin to suppress miscorrections caused by unreliable backward signals.Experimental results on the LogiQA 2.0 dataset show that,under the configuration with a candidate space size of K=3,N=3 independent voting rounds by the external evaluator,and two-stage confidence filtering,the proposed method achieves an accuracy of 77.16%,outperforming the Standard CoT baseline by 6.94 percentage points,while maintaining a verification trigger rate of 34.94% and an average verification latency of 1.3206s,demonstrating a favorable trade-off between accuracy and efficiency.After further incorporating the dynamic gating mechanism,the overall system accuracy reaches 76.41%,correction precision improves to 66.89%,and the number of destructive flips decreases from 57 to 33.Consistent gains are also achieved on ReClor,which further validates the generalizability of the proposed method.These results indicate that the proposed method can effectively alleviate the high-confidence error problem in LLM-based logical reasoning with relatively low additional overhead.

Key words: Large language models, Logical reasoning, Reverse verification, Confidence evaluation, Dynamic gating

中图分类号: 

  • TP391
[1] VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems 30:Neural Information Processing Systems Foundation.2017:5998-6008.
[2] BROWN T B,MANN B,RYDER N,et al.Language models are few-shot learners[C]//Advances in Neural Information Processing Systems 33:Neural Information Processing Systems Foundation.2020:1877-1901.
[3] OUYANG L,WU J,JIANG X,et al.Training language models to follow instructions with human feedback[C]//Advances in Neural Information Processing Systems 35:Neural Information Processing Systems Foundation.2022:27730-27744.
[4] WANG W Q,GUO M F,YANG D X,et al.Survey on the deve-lopment and applications of large language models[J].Journal of Zhongyuan University of Technology,2025,36(2):1-8.
[5] RAFAILOV R,SHARMA A,MITCHELL E,et al.Direct pre-ference optimization:Your language model is secretly a reward model[J].Advances in Neural Information Processing Systems,2023,36:53728-53741.
[6] SHAO Z,WANG P,ZHU Q,et al.Deepseekmath:Pushing the limits of mathematical reasoning in open language models[J].arXiv:2402.03300,2024.
[7] WEI J,WANG X,SCHUURMANS D,et al.Chain-of-thoughtprompting elicits reasoning in large language models[C]//Advances in Neural Information Processing Systems 35:Neural Information Processing Systems Foundation.2022:24824-24837.
[8] WANG X,WEI J,SCHUURMANS D,et al.Self-consistencyimproves chain of thought reasoning in language models[C]//The Eleventh International Conference on Learning Representations.2023:12648-12671.
[9] ZENG B Q,LUO Z K,CHEN W H.Natural language inference via negation augmentation[J].Computer Systems & Applications,2025,34(9):104-111.
[10] ZHANG Z,ZHANG A,LI M,et al.Automatic chain of thought prompting in large language models[J].arXiv:2210.03493,2022.
[11] WANG L,XU W,LAN Y,et al.Plan-and-solve prompting:improving zero-shot chain-of-thought reasoning by large language models[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics.ACL,2023:2609-2634.
[12] CHEN W,MA X,WANG X,et al.Program of thoughts promp-ting:Disentangling computation from reasoning for numerical reasoning tasks[J].arXiv:2211.12588,2022.
[13] YAO S,YU D,ZHAO J,et al.Tree of thoughts:deliberateproblem solving with large language models[C]//Advances in Neural Information Processing Systems 36:Neural Information Processing Systems Foundation.2023.
[14] YAO S,ZHAO J,YU D,et al.ReAct:synergizing reasoning and acting in language models[C]//The Eleventh International Conference on Learning Representations.2023.
[15] SCHICK T,DWIVEDI-YU J,DESSÌ R,et al.Toolformer:Language models can teach themselves to use tools[J].Advances in Neural Information Processing Systems,2023,36:68539-68551.
[16] WENG Y,ZHU M,XIA F,et al.Large language models are bet-ter reasoners with self-verification[C]//Findings of the Asso-ciation for Computational Linguistics:EMNLP 2023.ACL,2023:2550-2575.
[17] ZHANG Y,KHALIFA M,LOGESWARAN L,et al.Small language models need strong verifiers to self-correct reasoning[C]//Findings of the Association for Computational Linguistics:ACL 2024.ACL,2024:15637-15653.
[18] TAN C H,JIA K B,WANG H Y.Text summarization qualityevaluation based on large language model[J].Computer Systems &Applications,2025,34(2):28-36.
[19] GOU Z,SHAO Z,GONG Y,et al.CRITIC:large language mo-dels can self-correct with tool-interactive critiquing[C]//International Conference on Learning Representations 2024:ICLR.2024.
[20] LIN Z,GOU Z,LIANG T,et al.CriticBench:benchmarkingLLMs for critique-correct reasoning[C]//Findings of the Association for Computational Linguistics:ACL 2024.ACL,2024:1585-1603.
[21] MEKALA D,WESTON J E,LANCHANTIN J,et al.TOOLVERIFIER:generalization to new tools via self-verification[C]//Findings of the Association for Computational Linguistics:EMNLP 2024.ACL,2024:5026-5041.
[22] LIU C,YUAN Y,YIN Y,et al.Safe:enhancing mathematical reasoning in large language models via retrospective step-aware formal verification[C]//Proceedings of the 63rd Annual Mee-ting of the Association for Computational Linguistics.ACL,2025:12171-12186.
[23] JANG H,JANG Y,LEE S,et al.Self-training large languagemodels with confident reasoning[C]//Findings of the Association for Computational Linguistics:EMNLP 2025.ACL,2025:14925-14939.
[24] LUO H K,GE Y F,LIU S.Research progress of large language models in mathematical reasoning[J].Computer Engineering,2024,50(9):1-17.
[25] ARIYANI N F,BOURAOUI Z,BOOTH R,et al.There’s no such thing as simple reasoning for LLMs[C]//Findings of the Association for Computational Linguistics:ACL 2025.ACL,2025:4503-4514.
[26] TANG Z,SHEN K,KEJRIWAL M.An evaluation of estimative uncertainty in large language models[J].NPJ Complexity,2026,3:8.
[27] PENG Z,LIU X,YANG G,et al.The uncertainty advantage:enhancing large language models’ reliability through chain of uncertainty reasoning[J].Pattern Recognition Letters,2026,200:30-36.
[28] SONG J,TANG X,HAO K,et al.An uncertainty-aware framework integrating large language model and fuzzy inference system for commonsense reasoning[J].Expert Systems with Applications,2026,310:131273.
[29] LIU H,LIU J,CUI L,et al.Logiqa 2.0—an improved dataset for logical reasoning in natural language understanding[J].IEEE/ACM Transactions on Audio,Speech,and Language Processing,2023,31:2947-2962.
[30] YU W H,ZHANG J,et al.ReClor:A Reading Comprehension Dataset Requiring Logical Reasoning[C]//ICLR.2020.
[31] MADAAN A,TANDON N,GUPTA P,et al.Self-refine:Iterative refinement with self-feedback[J].Advances in Neural Information Processing Systems,2023,36:46534-46594.
[32] SANYAL S,XIAO T,LIU J,et al.Are machines better at complex reasoning? unveiling human-machine inference gaps in entailment verification[C]//Findings of the Association for Computational Linguistics:ACL 2024.2024:10361-10386.
[33] XU X,CHEN T,ZHANG F,et al.Double-Checker:Enhancing Reasoning of Slow-Thinking LLMs via Self-Critical Fine-Tuning[J].arXiv:2506.21285,2025.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发