Computer Science

Computer Science ›› 2026, Vol. 53 ›› Issue (10): 285-294.doi: 10.11896/jsjkx.260500060

• Artificial Intelligence • Previous Articles     Next Articles

Reverse Verification Logical Reasoning Method Based on Two-stage Triggering

LI Peng1,2, QIU Zheyu1, HU Sujun1   

  1. 1 School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023, China
    2 Institute of Network Security and Trusted Computing,Nanjing University of Posts and Telecommunications,Nanjing 210023, China
  • Received:2026-05-13 Revised:2026-07-27 Published:2026-10-13
  • About author:LI Peng,born in 1979,Ph.D,professor,Ph.D supervisor,is a member of CCF(No.48573M).His main research interests include computer communication networks,cloud computing and information security.
  • Supported by:
    Six Talent Peaks Project of Jiangsu Province(RJFW-111) and Postgraduate Research and Practice Innovation Program of Jiangsu Province(KYCX23_1048).

PDF (PC)

63

Abstract: To address three major challenges of large language models (LLMs) in complex logical reasoning tasks,namely error accumulation in one-pass forward generation,the difficulty of detecting high-confidence errors,and the high computational cost of exhaustive external verification,this paper proposes a reverse verification logical reasoning framework.Specifically,a two-stage dynamic triggering mechanism is developed based on the confidence of the final token generated by the forward reasoner and the margin between candidate predictions.The receiver operating characteristic (ROC) curve and Youden’s index are further employed to determine the optimal verification threshold for high-risk sample selection.The Top-K candidate space produced during forward reasoning is then reused to construct counterfactual hypotheses,and an independent judgment distribution is obtained through multiple rounds of voting by an external evaluator,enabling logical consistency checking.In addition,a dynamic decision gating mechanism is introduced based on the evaluator margin to suppress miscorrections caused by unreliable backward signals.Experimental results on the LogiQA 2.0 dataset show that,under the configuration with a candidate space size of K=3,N=3 independent voting rounds by the external evaluator,and two-stage confidence filtering,the proposed method achieves an accuracy of 77.16%,outperforming the Standard CoT baseline by 6.94 percentage points,while maintaining a verification trigger rate of 34.94% and an average verification latency of 1.3206s,demonstrating a favorable trade-off between accuracy and efficiency.After further incorporating the dynamic gating mechanism,the overall system accuracy reaches 76.41%,correction precision improves to 66.89%,and the number of destructive flips decreases from 57 to 33.Consistent gains are also achieved on ReClor,which further validates the generalizability of the proposed method.These results indicate that the proposed method can effectively alleviate the high-confidence error problem in LLM-based logical reasoning with relatively low additional overhead.

Key words: Large language models, Logical reasoning, Reverse verification, Confidence evaluation, Dynamic gating

CLC Number: 

  • TP391
[1] VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems 30:Neural Information Processing Systems Foundation.2017:5998-6008.
[2] BROWN T B,MANN B,RYDER N,et al.Language models are few-shot learners[C]//Advances in Neural Information Processing Systems 33:Neural Information Processing Systems Foundation.2020:1877-1901.
[3] OUYANG L,WU J,JIANG X,et al.Training language models to follow instructions with human feedback[C]//Advances in Neural Information Processing Systems 35:Neural Information Processing Systems Foundation.2022:27730-27744.
[4] WANG W Q,GUO M F,YANG D X,et al.Survey on the deve-lopment and applications of large language models[J].Journal of Zhongyuan University of Technology,2025,36(2):1-8.
[5] RAFAILOV R,SHARMA A,MITCHELL E,et al.Direct pre-ference optimization:Your language model is secretly a reward model[J].Advances in Neural Information Processing Systems,2023,36:53728-53741.
[6] SHAO Z,WANG P,ZHU Q,et al.Deepseekmath:Pushing the limits of mathematical reasoning in open language models[J].arXiv:2402.03300,2024.
[7] WEI J,WANG X,SCHUURMANS D,et al.Chain-of-thoughtprompting elicits reasoning in large language models[C]//Advances in Neural Information Processing Systems 35:Neural Information Processing Systems Foundation.2022:24824-24837.
[8] WANG X,WEI J,SCHUURMANS D,et al.Self-consistencyimproves chain of thought reasoning in language models[C]//The Eleventh International Conference on Learning Representations.2023:12648-12671.
[9] ZENG B Q,LUO Z K,CHEN W H.Natural language inference via negation augmentation[J].Computer Systems & Applications,2025,34(9):104-111.
[10] ZHANG Z,ZHANG A,LI M,et al.Automatic chain of thought prompting in large language models[J].arXiv:2210.03493,2022.
[11] WANG L,XU W,LAN Y,et al.Plan-and-solve prompting:improving zero-shot chain-of-thought reasoning by large language models[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics.ACL,2023:2609-2634.
[12] CHEN W,MA X,WANG X,et al.Program of thoughts promp-ting:Disentangling computation from reasoning for numerical reasoning tasks[J].arXiv:2211.12588,2022.
[13] YAO S,YU D,ZHAO J,et al.Tree of thoughts:deliberateproblem solving with large language models[C]//Advances in Neural Information Processing Systems 36:Neural Information Processing Systems Foundation.2023.
[14] YAO S,ZHAO J,YU D,et al.ReAct:synergizing reasoning and acting in language models[C]//The Eleventh International Conference on Learning Representations.2023.
[15] SCHICK T,DWIVEDI-YU J,DESSÌ R,et al.Toolformer:Language models can teach themselves to use tools[J].Advances in Neural Information Processing Systems,2023,36:68539-68551.
[16] WENG Y,ZHU M,XIA F,et al.Large language models are bet-ter reasoners with self-verification[C]//Findings of the Asso-ciation for Computational Linguistics:EMNLP 2023.ACL,2023:2550-2575.
[17] ZHANG Y,KHALIFA M,LOGESWARAN L,et al.Small language models need strong verifiers to self-correct reasoning[C]//Findings of the Association for Computational Linguistics:ACL 2024.ACL,2024:15637-15653.
[18] TAN C H,JIA K B,WANG H Y.Text summarization qualityevaluation based on large language model[J].Computer Systems &Applications,2025,34(2):28-36.
[19] GOU Z,SHAO Z,GONG Y,et al.CRITIC:large language mo-dels can self-correct with tool-interactive critiquing[C]//International Conference on Learning Representations 2024:ICLR.2024.
[20] LIN Z,GOU Z,LIANG T,et al.CriticBench:benchmarkingLLMs for critique-correct reasoning[C]//Findings of the Association for Computational Linguistics:ACL 2024.ACL,2024:1585-1603.
[21] MEKALA D,WESTON J E,LANCHANTIN J,et al.TOOLVERIFIER:generalization to new tools via self-verification[C]//Findings of the Association for Computational Linguistics:EMNLP 2024.ACL,2024:5026-5041.
[22] LIU C,YUAN Y,YIN Y,et al.Safe:enhancing mathematical reasoning in large language models via retrospective step-aware formal verification[C]//Proceedings of the 63rd Annual Mee-ting of the Association for Computational Linguistics.ACL,2025:12171-12186.
[23] JANG H,JANG Y,LEE S,et al.Self-training large languagemodels with confident reasoning[C]//Findings of the Association for Computational Linguistics:EMNLP 2025.ACL,2025:14925-14939.
[24] LUO H K,GE Y F,LIU S.Research progress of large language models in mathematical reasoning[J].Computer Engineering,2024,50(9):1-17.
[25] ARIYANI N F,BOURAOUI Z,BOOTH R,et al.There’s no such thing as simple reasoning for LLMs[C]//Findings of the Association for Computational Linguistics:ACL 2025.ACL,2025:4503-4514.
[26] TANG Z,SHEN K,KEJRIWAL M.An evaluation of estimative uncertainty in large language models[J].NPJ Complexity,2026,3:8.
[27] PENG Z,LIU X,YANG G,et al.The uncertainty advantage:enhancing large language models’ reliability through chain of uncertainty reasoning[J].Pattern Recognition Letters,2026,200:30-36.
[28] SONG J,TANG X,HAO K,et al.An uncertainty-aware framework integrating large language model and fuzzy inference system for commonsense reasoning[J].Expert Systems with Applications,2026,310:131273.
[29] LIU H,LIU J,CUI L,et al.Logiqa 2.0—an improved dataset for logical reasoning in natural language understanding[J].IEEE/ACM Transactions on Audio,Speech,and Language Processing,2023,31:2947-2962.
[30] YU W H,ZHANG J,et al.ReClor:A Reading Comprehension Dataset Requiring Logical Reasoning[C]//ICLR.2020.
[31] MADAAN A,TANDON N,GUPTA P,et al.Self-refine:Iterative refinement with self-feedback[J].Advances in Neural Information Processing Systems,2023,36:46534-46594.
[32] SANYAL S,XIAO T,LIU J,et al.Are machines better at complex reasoning? unveiling human-machine inference gaps in entailment verification[C]//Findings of the Association for Computational Linguistics:ACL 2024.2024:10361-10386.
[33] XU X,CHEN T,ZHANG F,et al.Double-Checker:Enhancing Reasoning of Slow-Thinking LLMs via Self-Critical Fine-Tuning[J].arXiv:2506.21285,2025.
[1] LI Zhennan, QIAN Jiayan, WANG Xinzhi, ZHANG Hui. Technology Risk Structure Recognition Based on Multi-granularity Semantic Dual Reflection [J]. Computer Science, 2026, 53(9): 395-404.
[2] ZHANG Rongjie, PANG Xiongwen, WANG Fengling. Review of Large Language Model-based Time Series Modeling via Fine-tuning and AgentArchitecture [J]. Computer Science, 2026, 53(9): 55-70.
[3] GUO Yuyang, SHI Lei, LIU Huan, DONG Yixiang, LI Rui. Domain-adapted and Dynamically Retrieval-augmented Approach for Large-scale History Discipline Model Construction [J]. Computer Science, 2026, 53(9): 92-100.
[4] HE Jiaojun, LI Xin. Review of Graph Learning Based on Large Language Models:Methods,Benchmarks and Advances [J]. Computer Science, 2026, 53(9): 1-15.
[5] LI Luozheng, LI Lingbo, YUAN Quan. Survey of Chinese Datasets for LLM Safety Alignment:Landscape and Prospects [J]. Computer Science, 2026, 53(9): 16-23.
[6] LIU Jing. Review of Music Artificial Intelligence Driven by Large Language Models [J]. Computer Science, 2026, 53(8): 229-244.
[7] ZHANG Haoran, HAO Wenning, JIN Dawei, CHENG Kai, LIU Junyang. Agentic Retrieval Augmented Generation Framework Based on Retrieval Task Planning and Reflection Mechanism [J]. Computer Science, 2026, 53(8): 285-297.
[8] CHEN Zhixiang, XIE Zhipeng. Event Causal Data Augmentation Method Based on Large Language Model [J]. Computer Science, 2026, 53(7): 125-131.
[9] XU Rui, LIU Jin, LIU Xudong, GUAN Jian, DONG Wei. Exploring the Generalization Ability of Prompt-based Large Language Models for TextClassification [J]. Computer Science, 2026, 53(6A): 250400092-7.
[10] WEI Qing, ZHANG Yupeng, LIU Shaoxun, ZHANG Jinfeng, ZHANG Yuezhong, CHEN Haoyang. Fuzzing Driver Generation Based on Large Language Models [J]. Computer Science, 2026, 53(6A): 250400113-8.
[11] ZHANG Yongyu, GUO Chenjuan, FEI Xueqin, LI Feng. Study on Financial Text Sentiment Analysis Method Based on Large Language Models with Market Feedback Supervision [J]. Computer Science, 2026, 53(6A): 250500073-14.
[12] SHI Hongxu, LIU Yi, LIU Kun. Survey of Recommendation Systems Based on Large Language Models [J]. Computer Science, 2026, 53(6): 281-303.
[13] WANG Shenghui, LI Teng. Innovative Automated Scoring Based on Large Language Models [J]. Computer Science, 2026, 53(5): 90-98.
[14] LIU Suyi, LIU Qi, GAO Weibo. Agent4Stu:Efficient LLM-based Student Answer Behavior Simulation Agent [J]. Computer Science, 2026, 53(4): 347-355.
[15] HU Junjie, CHEN Yujie, HU Yikun, WEN Cheng, CAO Jialun, MA Zhi, SU Jie, SUN Weidi, TIAN Cong, QIN Shengchao. Formal Theorem Proving Empowered by Large Language Model:Survey and Perspectives [J]. Computer Science, 2026, 53(4): 1-23.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.