自然语言语义表示的范畴论建模:系统综述与组合机制分析

doi:10.11896/jsjkx.251000136

Abstract

Abstract: Semantic representation is a central challenge in natural language processing (NLP).Existing approaches can be broa-dly categorized into two paradigms:symbolic and connectionist methods.Although the latter have achieved remarkable practical success,they suffer from theoretical limitations-commonly referred to as the “compositionality crisis” in compositional modeling and semantic interpretability.In existing methods,categorical compositional distributional semantics provides a principled mathematical framework for unifying symbolic syntactic structure with distributed semantics via type-driven composition.From a categorical perspective,this paper surveys category-theoretic approaches to semantic representation along the conceptual line of “category theory-composition-quantum computation”.Unlike surveys organized by models or tasks,it focuses on semantic composition mechanisms,comparing sentence-level models from a compositional viewpoint,analyzing the limitations of distributed approaches,and outlining the theoretical shift toward compositional distributional semantics.Building on this,string diagram-based frameworks such as DisCoCat and DisCoCirc are presented,clarifying their formal properties and quantum extensions,offering a unified view of symbolic,connectionist,and quantum semantics.

Key words: Category theory, String diagrams, Compositional semantics, Quantum computing, Interpretability

CLC Number:

B819

LI Yidan, CUI Jianying, XIONG Minghui. Category-Theoretic Semantic Representation: Systematic Review and Compositional Mechanism Analysis[J].Computer Science, 2026, 53(4): 337-346.

References

[1]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[C]//Proceedings of the 1st International Conference on Learning Representations.2013.
[2]PENNINGTON J,SOCHER R,MANNING C D.GloVe:Global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing.2014:1532-1543.
[3]PETERS M E,NEUMANN M,IYYER M,et al.Deep contextualized word representations[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2018:2227-2237.
[4]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,2019:4171-4186.
[5]MITCHELL J,LAPATA M.Vector-based models of semantic composition[C]//Proceedings of ACL-08:Human Language Technologies.2008:236-244.
[6]BLACOE W,LAPATA M.A comparison of vector-based representations for semantic composition[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Lear-ning.2012:546-556.
[7]DE FELICE G.Categorical tools for natural language processing[J].arXiv:2212.06636,2022.
[8]TULL S,LORENZ R,CLARK S,et al.Towards compositional interpretability for XAI[J].arXiv:2406.17583,2024.
[9]LI R,ZHAO X,MO M F.A brief overview of universal sentence representation methods:A linguistic view[J].ACM Computing Surveys,2022,55(3):1-42.
[10]ARORA S,LIANG Y,MA T.A simple but tough-to-beat baseline for sentence embeddings[C]//Proceedings of the 5th International Conference on Learning Representations.2017.
[11]RÜCKLÉ A,EGER S,PEYRARD M,et al.Concatenated p-mean word embeddings as universal cross-lingual sentence representations[C]//Proceedings of the 2018 Conference.2018.
[12]LE Q V,MIKOLOV T.Distributed representations of sentences and documents[C]//Proceedings of the 31st International Conference on Machine Learning.2014:1188-1196.
[13]HILL F,CHO K,KORHONEN A.Learning distributed repre-sentations of sentences from unlabelled data[C]//Proceedings of the Conference of the North American Chapter of the Asso-ciation for Computational Linguistics:Human Language Techno-logies.2016:1367-1377.
[14]ZHANG M,WU Y,LI W K,et al.Learning universal sentence representations with mean-max attention autoencoder[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018.
[15]CHEN Q,WANG W,ZHANG Q L,et al.Ditto:A simple and efficient approach to improve sentence embeddings[C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing.2023.
[16]KIROS R,ZHU Y,SALAKHUTDINOV R,et al.Skip-thought vectors[C]//Proceedings of the Advances in Neural Information Processing Systems.2015.
[17]LOGESWARAN L,LEE H.An efficient framework for learning sentence representations[C]//Proceedings of the 6th International Conference on Learning Representations.2018.
[18]NIE A,BENNETT E D,GOODMAN N D.DisSent:Sentence representation learning from explicit discourse relations[C]//Proceedings of the 2017 Conference.2017.
[19]SILEO D,VAN DE CRUYS T,PRADEL C,et al.Mining discourse markers for unsupervised sentence representation lear-ning[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:3477-3486.
[20]KIROS J,CHAN W.InferLite:Simple universal sentence representations from natural language inference data[C]//Procee-dings of the Conference on Empirical Methods in Natural Language Processing.2018.
[21]REIMERS N,GUREVYCH I.Sentence-BERT:Sentence em-beddings using siamese BERT-networks[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).2019.
[22]JERNITE Y,BOWMAN S R,SONTAG D.Discourse-based objectives for fast unsupervised sentence representation learning[J].arXiv:1705.00557,2017.
[23]SUBRAMANIAN S,TRISCHLER A,BENGIO Y,et al.Lear-ning general purpose distributed sentence representations via large scale multi-task learning[C]//Proceedings of the 6th International Conference on Learning Representations.2018.
[24]CER D,YANG Y,KONG S Y,et al.Universal sentence encoder for English[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing.2018:169-174.
[25]SCHOPF T,SCHNEIDER D,MATTHES F.Efficient domainadaptation of sentence embeddings using adapters[J].arXiv:2307.03104,2023.
[26]SOCHER R,PERELYGIN A,WU J,et al.Recursive deep mo-dels for semantic compositionality over a sentiment treebank[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing.2013:1631-1642.
[27]TAI K S,SOCHER R,MANNING C D.Improved semantic representations from tree-structured long short-term memory networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7thInternatio-nal Joint Conference on Natural Language Processing.2015:1556-1566.
[28]GEHRING J,AULI M,GRANGIER D,et al.Convolutional sequence to sequence learning[C]//Proceedings of the 34th International Conference on Machine Learning.2017:1243-1252.
[29]COECKE B,SADRZADEH M,CLARK S.Mathematical foundations for a compositional distributional model of meaning[J].arXiv:1003.4394,2010.
[30]LAN Z,CHEN M,GOODMAN S,et al.ALBERT:A lite BERT for self-supervised learning of language representations[J].arXiv:1909.11942,2019.
[31]OPENAI.GPT-5 system card[EB/OL].https://openai.com/-index/gpt-5-system-card/.
[32]RUSSELL B.The principles of mathematics[M].London:Routledge,1903.
[33]AJDUKIEWICZ K.Die syntaktische Konnexität[J].StudiaPhilosophica,1935,1:1-27.
[34]BAR-HILLEL Y.Logical syntax and semantics[J].Language,1954,30(2):230.
[35]EILENBERG S,MAC LANE S.General theory of naturalequivalences[J].Transactions of the American Mathematical Society,1945,58:231-294.
[36]LAWVERE F W.Functorial semantics of algebraic theories[J].Proceedings of the National Academy of Sciences of the United States of America,1963,50(5):869-872.
[37]LAMBEK J.On the calculus of syntactic types[C]//Structure of Language and Its Mathematical Aspects.Providence:American Mathematical Society,1961:166-178.
[38]COECKE B,GREFENSTETTE E,SADRZADEH M.Lambekvs.Lambek:Functorial vector space semantics and string diagrams for Lambek calculus[J].Annals of Pure and Applied Logic,2013,164(11):1079-1100.
[39]JOYAL A,STREET R.The geometry of tensor calculus II[R].Unpublished manuscript,1995.
[40]KARTSAKLIS D,FAN I,YEUNG R,et al.Lambeq:An efficient high-level Python library for quantum NLP[J].arXiv:2110.04236,2021.
[41]DE FELICE G,DI LAVORE E,ROMÁN M,et al.Functorial language games for question answering[C]//Proceedings of the 3rd Annual International Applied Category Theory Conference(ACT 2020).2020:311-321.
[42]TOUMI A,DE FELICE G.Higher-order DisCoCat(Peirce-Lambek-Montague semantics)[J].arXiv:2311.17813,2023.
[43]ZENG W,COECKE B.Quantum algorithms for compositional natural language processing[J].Electronic Proceedings in Theo-retical Computer Science,2016,221:67-75.
[44]YEUNG R,KARTSAKLIS D.A CCG-based version of the DisCoCat framework[J].arXiv:2105.07720,2021.
[45]COECKE B.The mathematics of text structure[C]//The Interplay of Mathematics,Logic,and Linguistics.2021:181-217.
[46]WANG-MASCIANICA V,LIU J,COECKE B.Distilling textinto circuits[J].arXiv:2301.10595,2023.
[47]LAAKKONEN T,MEICHANETZIDIS K,COECKE B.Quantum algorithms for compositional text processing[J].Electronic Proceedings in Theoretical Computer Science,2024,406:162-196.
[48]LE DU S,HERNÁNDEZ SANTANA S,SCARPA G.A gentle introduction to quantum natural language processing[J].arXiv:2202.11766,2022.
[49]SORDONI A,NIE J Y,BENGIO Y.Modeling term dependen-cies with quantum language models for IR[C]//Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval.2013:653-662.
[50]XIE M,HOU Y,ZHANG P,et al.Modeling quantum entanglements in quantum language models[C]//Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence.2015.
[51]LI Q,MELUCCI M,TIWARI P.Quantum language model-based query expansion[C]//Proceedings of the 2018 ACM SIGIR International Conference on Theory of Information Retrieval.2018:183-186.
[52]JIANG Y,ZHANG P,GAO H,et al.A quantum interference inspired neural matching model for ad-hoc retrieval[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.2020:19-28.
[53]PASIN A,CUNHA W,GONÇALVES M A,et al.A quantum annealing instance selection approach for efficient and effective transformer fine-tuning[C]//Proceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval.2024:205-214.
[54]LI Q,WANG B,MELUCCI M.CNM:An interpretable com-plex-valued network for matching[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:4139-4148.
[55]MANSKY B,WÖRLE F,STEIN J K,et al.Adapting the DisCoCat-model for question answering in the Chinese language[C]//Proceedings of the IEEE International Conference on Quantum Computing and Engineering.2023:591-600.
[56]DUNEAU T,BRUHN S,MATOS G,et al.Scalable and interpretable quantum natural language processing:An implementation on trapped ions[J].arXiv:2409.08777,2024.
[57]ZHANG P,ZHANG J,MA X,et al.TextTN:Probabilistic encoding of language on tensor network[C]//Proceedings of the International Conference on Learning Representations.2021.
[58]RUSKANDA F Z,ABIWARDANI M R,AL BARI M A,et al.Quantum representation for sentiment classification[C]//Proceedings of the IEEE International Conference on Quantum Computing and Engineering.2022:67-78.
[59]QU Z,MENG Y,MUHAMMAD G,et al.QMFND:A quantum multimodal fusion-based fake news detection model for social media[J].Information Fusion,2024,104:102172.
[60]MEICHANETZIDIS K,TOUMI A,DE FELICE G,et al.Grammar-aware question-answering on quantum computers[J].ar-Xiv:2012.03756,2020.
[61]MEICHANETZIDIS K,TOUMI A,DE FELICE G,et al.Grammar-aware sentence classification on quantum computers[J].Quantum Machine Intelligence,2023,5(1):10.
[62]NIETO V.Towards machine translation with quantum computers[D].Stockholm:University of Stockholm,2021.
[63]BRADLEY T,TERILLA J,VLASSOPOULOS Y.An enriched category theory of language:from syntax to semantics[J].La Matematica,2022,1(2):551-580.
[64]LIU J,SHAIKH R A,RODATZ B,et al.A pipeline for discourse circuits from CCG[J].arXiv:2311.17892,2023.
[65]LIU T,WEI Y,WANG J.Research on distributional compositional categorical model in both classical and quantum natural language processing[C]//Proceedings of the SNPD 2024.2024:1-6.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Category-Theoretic Semantic Representation: Systematic Review and Compositional Mechanism Analysis

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	LI Hui, LIU Shujuan, JU Mingmei, WANG Jiepeng, JI Yingsong. High Frequency-Dense Quantum Gate Set Optimization Algorithm for Quantum Circuit in NISQ Era [J]. Computer Science, 2026, 53(4): 112-120.
[2]	ZHENG Yi, JIA Xinghao, ZHANG Junwen, REN Shuang. Image Classification Based on Hybrid Quantum-Classical Long-Short Range Feature Extension Network [J]. Computer Science, 2026, 53(4): 277-283.
[3]	WEN Zerui, JIANG Tian, HUANG Zijian, CUI Xiaohui. Section Sparse Attack:A More Powerful Sparse Attack Method [J]. Computer Science, 2026, 53(1): 323-330.
[4]	ZHANG Xinglan, RONG Xiaojun. Variational Quantum Algorithm for Solving Discrete Logarithms [J]. Computer Science, 2026, 53(1): 353-362.
[5]	JIANG Yunliang, JIN Senyang, ZHANG Xiongtao, LIU Kaining, SHEN Qing. Multi-scale Multi-granularity Decoupled Distillation Fuzzy Classifier and Its Application inEpileptic EEG Signal Detection [J]. Computer Science, 2025, 52(9): 37-46.
[6]	ZHANG Yaolin, LIU Xiaonan, DU Shuaiqi, LIAN Demeng. Hybrid Quantum-classical Compressed Generative Adversarial Networks Based on Matrix Product Operators [J]. Computer Science, 2025, 52(6): 74-81.
[7]	XIONG Qibing, MIAO Qiguang, YANG Tian, YUAN Benzheng, FEI Yangyang. Malicious Code Detection Method Based on Hybrid Quantum Convolutional Neural Network [J]. Computer Science, 2025, 52(3): 385-390.
[8]	CHEN Zigang, PAN Ding, LENG Tao, ZHU Haihua, CHEN Long, ZHOU Yousheng. Explanation Robustness Adversarial Training Method Based on Local Gradient Smoothing [J]. Computer Science, 2025, 52(2): 374-379.
[9]	RUAN Ning, LI Chun, MA Haoyue, JIA Yi, LI Tao. Review of Quantum-inspired Metaheuristic Algorithms and Its Applications [J]. Computer Science, 2025, 52(10): 190-200.
[10]	ZHU Fukun, TENG Zhen, SHAO Wenze, GE Qi, SUN Yubao. Semantic-guided Neural Network Critical Data Routing Path [J]. Computer Science, 2024, 51(9): 155-161.
[11]	XIN Bo, DING Zhijun. Interpretable Credit Evaluation Model for Delayed Label Scenarios [J]. Computer Science, 2024, 51(8): 45-55.
[12]	QIAO Fan, WANG Peng, WANG Wei. Multivariate Time Series Classification Algorithm Based on Heterogeneous Feature Fusion [J]. Computer Science, 2024, 51(2): 36-46.
[13]	WANG Baocai, WU Guowei. Feature-weighted Counterfactual Explanation Method:A Case Study in Credit Risk Control Scenarios [J]. Computer Science, 2024, 51(12): 259-268.
[14]	CHEN Chao, YAN Wenjie, XUE Guixiang. Parameterized Quantum Circuits Based Quantum Neural Networks for Data Classification [J]. Computer Science, 2024, 51(11A): 231200112-7.
[15]	WANG Dongli, YANG Shan, OUYANG Wanli, LI Baopu, ZHOU Yan. Explainability of Artificial Intelligence:Development and Application [J]. Computer Science, 2023, 50(6A): 220600212-7.