计算机科学 ›› 2025, Vol. 52 ›› Issue (1): 131-141.doi: 10.11896/jsjkx.231200079

• 数据库&大数据&数据科学 • 上一篇    下一篇

面向NewSQL数据库数据协同持久化的研究

左顺1, 李永坤2, 许胤龙2   

  1. 1 中国科学技术大学计算机科学与技术学院 合肥 230026
    2 安徽省高性能计算重点实验室 合肥 230026
  • 收稿日期:2023-12-12 修回日期:2024-05-22 出版日期:2025-01-15 发布日期:2025-01-09
  • 通讯作者: 李永坤(ykli@ustc.edu.cn)
  • 作者简介:(shunzuo@mail.ustc.edu.cn)
  • 基金资助:
    国家自然科学基金面上项目(62172382)

Study on Collaborative Data Persistence in NewSQL Databases

ZUO Shun1, LI Yongkun2, XU Yinlong2   

  1. 1 School of Computer Science and Technology,University of Science and Technology of China,Hefei 230026,China
    2 Anhui Province Key Laboratory of High Performance Computing,Hefei 230026,China
  • Received:2023-12-12 Revised:2024-05-22 Online:2025-01-15 Published:2025-01-09
  • About author:ZUO Shun,born in 1999,postgraduate.His main research interests include distributed databases and data persistence.
    LI Yongkun,born in 1986,professor,is a member of CCF(No.33184M).His main research interests include data storage and file systems,especially storage and system problems in application scenarios such as cloud computing and big data,virtualized memory management,key-value systems,flash,NVRAM.
  • Supported by:
    National Natural Science Foundation of China(62172382).

摘要: 现代NewSQL数据库为了提供数据的高可用性,通常会为数据提供多个副本,以便在某个副本不可用时,可以从其他的副本中获取数据。而在数据多副本的情况下,又需要考虑副本间的数据一致性问题,即在某一时刻不同客户端读取某个数据时得到的结果应该是相同的,因此引入了事务处理机制。在一个包含多个写操作的交互式事务处理过程中,由于数据存在多个副本,因此每个写入操作需要对所有的主备副本进行写入操作。然而主备副本通常分散在不同的机器上,因此会引入写远端副本的时延,其最终将会增大整个事务的处理时延。针对该问题,提出了数据协同持久化的方案,其主要思想是让客户端在本地缓存事务的写操作日志,在最终提交事务时,客户端首先将事务中的写操作日志进行持久化,并将该日志发送给事务的协调者节点,让协调者进行日志数据的分发处理,从而达到两者协同持久化事务数据的目的。实验结果表明,相较于同步持久化方案,协同持久化方案不仅能降低交互式事务处理的时延,还能提高约38%左右的系统极限吞吐率。

关键词: 分布式数据库, 并发控制, 数据持久化, 数据一致性, 高数据竞争负载

Abstract: To ensure high availability of data,modern NewSQL databases often create several copies of data so that it can be accessed from other copies in case one copy is not available.With multiple data copies,it is essential to consider data consistency between them.This means that the results should be the same when different clients read the same data at a particular moment.Therefore,a transaction processing mechanism is introduced.In the interactive transactional process with multiple write operations,each write operation must be performed on both the primary and backup copies of the data,since there are multiple copies.However,the primary and backup replicas are typically located on different machines,resulting in increased latency when writing to remote replicas,which in turn can ultimately lead to an increase in the processing latency of the entire transaction.In this paper,we present a collaborative data persistence scheme where the client caches the transaction write logs locally.When the transaction is finally committed,the client firstly persists the write logs of the transaction and sends the logs to the coordinator node of the transaction to allow the coordinator to distribute the log data,so as to achieve the purpose of the two cooperating in persistence of the transaction data.Experimental results show that in comparison to the synchronous persistence scheme,cooperative persistence scheme can not only reduce the latency of interactive transaction processing,but also improve the system limit throughput rate by roughly 38%.

Key words: Distributed database, Concurrency control, Data persistence, Data consistency, High-contention workload

中图分类号: 

  • TP311
[1]DANIELSEN A.The evolution of data models and approachesto persistence in database systems[J/OL].https://www.fing.edu.uy/inco/grupos/csi/esp/Cursos/cursos_act/2000/DAP_DisAvDB/documentacion/OO/Evol_DataModels.html.
[2]FATIMA H,WASNIK K.Comparison of SQL,NoSQL and NewSQL databases for internet of things[C]//2016 IEEE Bombay Section Symposium(IBSS).IEEE,2016:1-6.
[3]BINANI S,GUTTI A,UPADHYAY S.SQL vs.NoSQL vs.New-SQL-a comparative study[J].Database,2016,6(1):1-4.
[4]MONIRUZZAMAN A,HOSSAIN S.Nosql database:New era of databases for big data analytics-classification,characteristics and comparison[J].arXiv:1307.0191,2013.
[5]TAFT R,SHARIF I,MATEI A,et al.Cockroachdb:The resi-lient geo-distributed sql database[C]//Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data.2020:1493-1509.
[6]HUANG D,LIU Q,CUI Q,et al.TiDB:a Raft-based HTAPdatabase[J].Proceedings of the VLDB Endowment,2020,13(12):3072-3084.
[7]CORBETT J C,DEAN J,EPSTEIN M,et al.Spanner:Google’s globally distributed database[J].ACM Transactions on Computer Systems(TOCS),2013,31(3):1-22.
[8]Hewlett Packard Corporation.The machine:A new kind of computer[EB/OL].https://www.hpl.hp.com/research/systems-research/themachine/.
[9]Intel Corporation.Intel rack scale design architecture[EB/OL].https://www.intel.com/content/www/us/en/architecture-and-technology/rack-scale-design-overview.html,2021.
[10]MARCOS K A,NADAV A,IRINA C,et al.Remote regions:a simple abstraction for remote memory[C]//2018 USENIX Annual Technical Conference.2018:775-787.
[11]DU X Y,LI T,LU W,et al.Cross-domain Data Management[J].Computer Science,2024,51(1):4-12.
[12]YAN X,YANG L,ZHANG H,et al.Carousel:Low-latencytransaction processing for globally-distributed data[C]//Proceedings of the 2018 International Conference on Management of Data.2018:231-243.
[13]MU S,NELSON L,LLOYD W,et al.Consolidating concurrency control and consensus for commits under conflicts[C]//12th USENIX Symposium on Operating Systems Design and Implementation(OSDI 16).2016:517-532.
[14]REN K,LI D,ABADI D J.Slog:Serializable,low-latency,geo-replicated transactions[J].Proceedings of the VLDB Endowment,2019,12(11):1747-1761.
[15]ONGARO D,OUSTERHOUT J.In search of an understandable consensus algorithm[C]//2014 USENIX Annual Technical Conference(USENIX ATC 14).2014:305-319.
[16]LAMPORT L.Paxos made simple[J/OL].https://lamport.azurewebsites.net/pubs/paxos-simple.pdf.
[17]YIZHOU S,YUTONG H,YILUN C,et al.Legoos:A disseminated,distributed OS for hardware resource disaggregation[C]//13th USENIX Symposium on Operating Systems Design and Implementation.2018:69-87.
[18]PENGFEI Z,JIAZHAO S,LIU Y,et al.One-sided rdma-con-scious extendible hashing for disaggregated memory[C]//2021 USENIX Annual Technical Conference.2021:15-29.
[19]Futurewei Cloud.Chogori-Platform[EB/OL].https://github.com/futureweicloud/chogori-platform.
[20]General HBase tuning[EB/OL].https://www.ibm.com/docs/en/db2-big-sql/5.0.2?topic=performance-general-hbase-tu-ning.
[21]Cockroachlabs.How Pipelining consensus writes speeds up distributed SQL transactions[EB/OL].https://www.cockroach-
labs.com/blog/transaction-pipelining/.
[22]CARLSON J.Redis in action[M].Simon and Schuster,2013.
[23]DONG S,KRYCZKA A,JIN Y,et al.Rocksdb:Evolution of development priorities in a key-value store serving large-scale applications[J].ACM Transactions on Storage(TOS),2021,17(4):1-32.
[24]LevelDB[EB/OL].https://github.com/google/leveldb.
[25]THOMSON A,DIAMOND T,WENG S C,et al.Calvin:fastdistributed transactions for partitioned database systems[C]//Proceedings of the 2012 ACM SIGMOD International Confe-rence on Management of Data.2012:1-12.
[26]FunnaDB[EB/OL].https://fauna.com/blog/inside-faunas-distributed-transaction-engine-dte.
[27]CHEN Y,YU X,KOUTRIS P,et al.Plor:General transactions with predictable,low tail latency[C]//Proceedings of the 2022 International Conference on Management of Data.2022:19-33.
[28]ZHOU X,YU X,GRAEFE G,et al.Lotus:scalable multi-partition transactions on single-threaded partitioned databases[J].Proceedings of the VLDB Endowment,2022,15(11):2939-2952.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!