计算机科学 ›› 2026, Vol. 53 ›› Issue (6A): 250500104-12.doi: 10.11896/jsjkx.250500104

• 大数据&数据科学 • 上一篇    下一篇

面向关系数据库优化的轻量级内存映射数据缓存MDBCache

王海1,2, 刘中一1,3, 张晨阳1,2, 崔华1, 符丹1   

  1. 1 中国民航信息网络股份有限公司 北京 101318
    2 北京市民航大数据工程技术研究中心 北京 101318
    3 中国民用航空局民航旅客服务智能化应用技术重点实验室 北京 101318
  • 出版日期:2026-06-16 发布日期:2026-06-12
  • 通讯作者: 刘中一(liuzy@travelsky.com.cn)
  • 作者简介:(wanghai@travelsky.com.cn)
  • 基金资助:
    国家自然科学基金(U2433212);工信部大数据产业发展试点示范项目(工信厅信发函[2022]219号);工信部未来产业创新任务揭榜项目(工信厅科函[2024]220号)

MDBCache:Lightweight Data Caching Solution for Relational Databases Based on Memory- mapping

WANG Hai1,2, LIU Zhongyi1,3, ZHANG Chenyang1,2, CUI Hua1, FU Dan1   

  1. 1 TravelSky Technology Limited,Beijing 101318,China
    2 Beijing Engineering Research Center of Civil Aviation Big Data,Beijing 101318,China
    3 Key Laboratory of Intelligent Passenger Service of Civil Aviation,Civil Aviation Administration of China,Beijing 101318,China
  • Published:2026-06-16 Online:2026-06-12
  • About author:WANG Hai,born in 1979,master,senior engineer.His main research interests include middleware,cloud computing,and distributed systems.
    LIU Zhongyi,born in 1987,master,senior engineer,is a member of CCF(No.T8746M).His main research interests include digital intelligence in civil aviation,high-performance intensive computing and in-memory computing grid.
  • Supported by:
    National Natural Science Foundation of China(U2433212),Big Data Industry Development Pilot Demonstration Project of the Ministry of Industry and Information Technology of China(MIIT)(Letter[2022] No.219 from the Information Development Department,General Office of MIIT) and Future Industry Innovation Challenge Project of the Ministry of Industry and Information Technology of China(MIIT)(Letter[2024] No.220 from the Department of Science and Technology,General Office of MIIT).

摘要: 关系型数据库在高并发实时场景中存在读取效率低的问题,在传统数据库系统的高性能改造中,一般采用NoSQL(Not Only SQL)内存数据库、嵌入式数据库作为数据缓存层进行加速,读写效率提升上限有限,且由于存储结构与读写方式差异,迁移成本普遍较高。文中提出一种面向关系数据库的数据缓存方案MDBCache,通过使用定址内存映射缓存、多维度自定义索引与内存数据增量更新技术,实现了无需跨进程、跨用户空间的高效数据共享方式,大幅降低了数据请求响应时间及迁移成本。实验结果表明,与机制相近的内存数据库MMDB、嵌入式数据库Berkeley DB缓存方案相比,MDBCache读取效率在单线程模式上最高提升了1.49倍与6.88倍,数据更新效率最高提升了8.59倍与5.44倍。所提方案具有底层技术成熟、服务性能高、实施复杂度低、实用性高等优点,在高并发实时场景数据方案设计中具有一定参考价值。

关键词: 实时计算, 高并发计算, 高性能缓存, 定址内存映射, 多维度自定义索引, 内存数据增量更新

Abstract: Relational databases face issues of low read efficiency in high-concurrency real-time scenarios.In efforts to enhance the performance of traditional database systems,NoSQL(Not Only SQL) in-memory databases and embedded databases are often employed as data caching layers for acceleration.However,these approaches have limited potential for improving read and write efficiency,and generally incur high migration costs due to differences in storage structures and read/write methods.This paper proposes a data caching solution MDBCache tailored for relational databases.By leveraging fixed-address memory-mapped caching,multi-dimensional custom index reading,and incremental update technology for in-memory data,it achieves efficient data sharing without the need for cross-process or cross-user-space operations,significantly reducing data request response times and migration costs.Experimental results demonstrate that,compared to the caching solutions of the in-memory database MMDB and the embedded database Berkeley DB,MDBCache achieves a maximum improvement of 1.49× and 6.88× in read efficiency(single-thread mode),and 8.59× and 5.44× in data update efficiency,respectively.This solution boasts mature underlying technology,high-performance service,low implementation complexity,and high practicality,making it a valuable reference in the design of data solutions for high-concurrency real-time scenarios.

Key words: Real-time computing, High-concurrency computing, High-performance caching, Fixed-address memory mapping, Multi-dimensional custom indexing, In-memory data incremental update

中图分类号: 

  • TP311
[1] LIANG H F,LIU Z Y,YANG Y.High-Performance Civil Aviation Tariff System Architecture and Key Technologies [J/OL].Computer Applications and Software,1-8.[2024-11-10] .https://link.cnki.net/urlid/31.1260.tp.20240703.1142.002.
[2] TAIPALUS T.Database Management System Performance Com-parisons:A Systematic Literature Review [J].The Journal of Systems and Software,2024,208(2):111872.1-111872.16.
[3] KHAN W,AHMED E,SHAHZAD W.Predictive Performance Comparison Analysis of Relational & NoSQL Graph Databases [J].International Journal of Advanced Computer Science and Applications(IJACSA),2017,8(5).
[4] NAYAK A,PORIYA A,POOJARY D.Type of NOSQL Databases and its Comparison with Relational Databases [J].International Journal of Applied Information Systems,2013,5(4):16-19.
[5] EDDELBUETTEL D.A Brief Introduction to Redis [J].arXiv:2203.06559,2022.
[6] OLSON M A,BOSTIC K, SELTZER M I.Berkeley DB [C]//USENIX Annual Technical Conference,FREENIX Track,1999:183-191.
[7] BRADSHAW S,BRAZIL E,CHODOROW K.MongoDB:theDefinitive Guide:Powerful and Scalable Data Storage [M].O'Reilly Media,2019.
[8] ZAMFIR V A,CARABAS M,CARABAs C,et al.Systems Monitoring and Big Data Analysis Using the Elasticsearch System [C]//2019 22nd International Conference on Control Systems and Computer Science(CSCS).IEEE,2019:188-193.
[9] LAKSHMAN A,MALIK P.Cassandra:A Decentralized Structured Storage System [J].Operating Systems Review,2010,44(2):35-40.
[10] MILLER J J.Graph Database Applications and Concepts with Neo4j [C]//Proceedings of the Southern Association for Information Systems Conference.Atlanta:GA,USA,2324(36),2013:141-147.
[11] CHAUDHRY N,YOUSAF M M.Architectural Assessment of NoSQL and NewSQL Systems [J].Distributed and Parallel Databases,2020,38(4):881-926.
[12] HUANG X P,PENG M T,YANG Y K.Fast Reading Methodof Complex Objects Based on Memory-Mapped File [J].Computer Technology and Development,2020,30(3):82-87.
[13] WAN M N,GUAN Y,HAN X J.Study on Representative Embedded Database Technologies:SQLite and Berkeley DB [J].Microcomputer Information,2006(2):91-93,272.
[14] HUANG X P,PENG M T,YANG Y K.High-Performance In-ventory Caching System Based on Memory-Mapped File [J].Application of Electronic Technique,2020,46(7):113-117.
[15] JOSE B,ABRAHAM S.Performance Analysis of NoSQL andRelational Databases with MongoDB and MySQL [J].Materials Today:Proceedings,2020,24(3):2036-2043.
[16] GUPTA A,TYAGI S,PANWAR N,et al.NoSQL Databases:Critical Analysis and Comparison [C]//2017 International Conference on Computing and Communication Technologies for Smart Nation(IC3TSN).2017:293-299.
[17] KATHARE N,REDDY O V,PRABHU V.A ComprehensiveStudy of Elasticsearch [J].International Journal of Science and Research(IJSR),2021,10(6):716-720.
[18] KONONENKO O,BAYSAL O,HOLMES R,et al.MiningModern Repositories with Elasticsearch [C]//Proceedings of the 11th Working Conference on Mining Software Repositories.2014:328-331.
[19] CHENG Y J,DING P J,WANG T T,et al.Which Category Is Better:Benchmarking Relational and Graph Database Management Systems [J].Data Science and Engineering,2019,4(4):309-322.
[20] LUPU E,OLTEANU A,IONITA A D.Concurrent Access Performance Comparison Between Relational Databases and Graph NoSQL Databases for Complex Algorithms [J].Applied Sciences,2024,14(21):2076-3417.
[21] LIANG H F,YANG Y,LIU Z Y.High Performance Memory-Mapped Database MMDB for Complex Objects [J].Computer Engineering and Design,2023,44(3):937-944.
[22] LIN Z Y,KAHNG M,SABRIN K M,et al.Mmap:Fast Billion-Scale Graph Computation on A PC via Memory Mapping [C]//
2014 IEEE International Conference on Big Data(Big Data).2014:159-164.
[23] MAGALHAES A,BRAYNER A,MONTEIRO J M.MM-DI-RECT:Main Memory Database Instant Recovery with Tuple Consistent Checkpoint [J].VLDB Journal International Journal on Very Large Data Bases,2024,33(3).
[24] BARKER S,WOOD T,SHENOY P,et al.An Empirical Study of Memory Sharing in Virtual Machines [C]//2012 USENIX Annual Technical Conference(USENIX ATC 12).2012:273-284.
[25] SHAH A,BHATT N.A Systematic Review of In-Memory Database over Multi-Tenancy [J].International Journal of Electrical and Computer Engineering(IJECE),2024,14(2):1720-1729.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!